Beam profile analysis in combination with TOF sensors

Abstract

A method for determining if an object corresponds to a living organism comprising i) (132) illuminating the object with a pattern of light beams generated by at least one transmitter (116) of a time-of-flight system (114), wherein the transmitter (116) comprises at least one array of light emitters (118); ii) (134) generating at least one pattern image (122) of the object while the object is being illuminated by the pattern of light beams by using at least one detector (120) of the time-of-flight system (114); iii) (136) extracting liveness data from the pattern image (122) by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

Description

[0001] Beam Profile Analysis in Combination with ToF Sensors

[0002] Technical Field

[0003] The invention relates to a method for determining if an object corresponds to a living organism, a system for determining if an object corresponds to a living organism, a computer-implemented method for authenticating a user of a device, a system for authenticating a user, computer programs, computer-readable storage media, non-transient computer-readable media and several uses.

[0004] The devices, methods and uses according to the present invention specifically may be employed for example in various areas of daily life, security technology, gaming, traffic technology, production technology, photography such as digital photography or video photography for arts, documentation or technical purposes, safety technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology or in the sciences. However, other applications are also possible.

[0005] Background art

[0006] 3D detector comprising a Time-of-flight (TOF) sensor are generally known and may be used in mobile devices such as in smartphones, tablets and the like for distance estimation. Authentication systems of mobile devices may test for spoofing, e.g. determine if an object corresponds to a living organism, and require for these tests specific and additional sensors such as for material detection. There is still a need to save space and cost in hardware and reduce the number of elements for authentication systems for mobile devices.

[0007] WO 2023 / 287351 describes a three-dimensional (3D) face recognition system. The system comprises a structured light sensor or stereo camera sensor, a time- of-flight sensor and a processor. The processor is configured to run algorithms for face recognition on data from the structured light sensor or stereo camera sensor, wherein the processor is configured to use distance data from the time-of-flight sensor to optimize the algorithms for face recognition. Also disclosed is a method of 3D face recognition and an associated computer program product.

[0008] US 11 ,099,009 describes an imaging apparatus which includes a circuitry configured to perform depth analysis of a scene by time-of-flight imaging and to perform motion analysis in the scene by structured light imaging, wherein identical sensor data is used for both, the depth analysis and the motion analysis.

[0009] US10,613,228 describes a depth camera that uses structured light and modulated light to produce a depth image. The modulated and structured light allows a time-of-flight depth to be calculated for each unit (e.g., a single dot) of the reflected structured light image captured by the depth camera's image sensor. Once each unit of the reflected structured light image is identified, a structured light triangulation algorithm can be used to calculate a depth for each unit of the reflected structured light image.

[0010] Li Larry in "Time-of-Flight Camera -An Introduction", 31 May 2014, XP093183136, www.ti.com / lit / wp / sloa190b / sloa190b.pdf?ts=1720418472608 gives a general introduction to Time-of-Flight camera from Texas Instruments.

[0011] US 11 468 712 B2 describes a liveness detection device comprising a light source unit, image sensor unit, and data processing module and authentication method thereof. The light source unit comprises a substrate having a first inclined surface, whereby emitted light is reflected light from the first inclined surface. An application triggers an authentication process, which is indicated to a user. The light source unit begins illumination having a specific pattern and for a specific period and image signals are generated. Liveness detection signals are generated, via calculation of interference patterns, each, from more than one image signal, in sequence, for determination of liveness. When a liveness threshold is met, feature recognition data is generated, via calculation of interference patterns, each, from more than one image signal, in sequence, for matching. Then, the features are compared with previously enrolled data for locking or unlocking of the liveness detection device and / or system coupled thereto.

[0012] US 2021 / 181305 A1 describes a liveness test method and liveness test apparatus. The liveness test method includes determining a presence of a subject using a radar sensor, performing a first liveness test on the subject based on radar data obtained by the radar sensor, in response to the subject being present, acquiring image data of the subject using an image sensor, in response to a result of the first liveness test satisfying a first condition, and performing a second liveness test on the subject based on the image data.

[0013] Problem to be solved

[0014] It is therefore an object of the present invention to provide devices and methods facing the above-mentioned technical challenges of known devices and methods. Specifically, it is an object of the present invention to provide method and devices for authentication of a user, e.g. of mobile devices, allowing for reducing the number elements to save space and cost in hardware.

[0015] Summary

[0016] This problem is addressed by method for determining if an object corresponds to a living organism, a system for determining if an object corresponds to a living organism, a computer-implemented method for authenticating a user of a device, a system for authenticating a user, computer programs, computer-readable storage media, non-transient computer-readable media with the features of the independent claims. Advantageous embodiments which might be realized in an isolated fashion or in any arbitrary combinations are listed in the dependent claims as well as throughout the specification.

[0017] In a first aspect, a method for determining if an object corresponds to a living organism is disclosed.

[0018] The term “object” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary target, chosen from a living object and a non-living object. The object may be or may comprise one or more living beings and / or one or more parts thereof, such as one or more body parts of a human being, e.g. a user. The object may be a non-living object such as a silicon mask or a printed image of a human being. The term “living organism”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to any living body, in particular a living human. The term “living human”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an individual of the species homo sapiens, wherein the individual is currently alive.

[0019] Determining if the user corresponds to a living organism may comprise distinguishing between non-living organisms, e.g. between skin and non-skin material such as silicon or paper.

[0020] The method steps may be performed in the given order. A different order, however, may also be feasible. Further, two or more of the method steps may be performed simultaneously. Thereby the method steps may at least partly overlap in time. Further, the method steps may be performed once or repeatedly. Thus, one or more or even all of the method steps may be performed once or repeatedly. The method may comprise additional method steps, which are not listed herein.

[0021] The method comprises i) illuminating the object with a pattern of light beams generated by at least one transmitter of a time-of-flight system, wherein the transmitter comprises at least one array of light emitters; ii) generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams by using at least one detector of the time-of-flight system; iii) extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0022] The term “light” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term “ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO- 21348 in a valid version at the date of this document, the term “visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term “infrared spectral range” (IR) generally refers to electromagnetic radiation of 760 nm to 1000 pm, wherein the range of 760 nm to 1 .5 pm is usually denominated as “near infrared spectral range” (NIR) while the range from 1 .5 p to 15 pm is denoted as “mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as “far infrared spectral range” (FIR). Preferably, light used for the typical purposes of the present invention is light in the infrared (IR) spectral range, more preferred, in the near infrared (NIR) and / or the mid infrared spectral range (MidlR), especially the light having a wavelength of 1 pm to 5 pm, preferably of 1 pm to 3 pm. For example, the light beam may have a wavelength from 760 nm to 1 .5 pm, preferably 940 nm, 1140 nm or 1320 to 1380 nm. Other wavelength are possible, too.

[0023] The term “ray” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a line that is perpendicular to wavefronts of light which points in a direction of energy flow. The term “light beam” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a collection of rays. In the following, the terms “ray” and “beam” will be used as synonyms. The term “light beam” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an amount of light, specifically an amount of light traveling essentially in the same direction, including the possibility of the light beam having a spreading angle or widening angle.

[0024] The term “illuminate”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of exposing at least one element to light. The illuminating may comprise projecting a pattern onto a scene comprising the object, e.g. a face.

[0025] The term “pattern”, also denoted as “light pattern” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary pattern comprising a plurality of light beams of at least two light beams, preferably at least two light beams. The pattern may be projected onto the object. Projection of a light beam of the light pattern onto a surface may result in a light spot. A light beam may illuminate at least a part of the surface. A light spot may refer to a contiguous area of coherent electromagnetic radiation on at least a part of the surface. A light spot may refer to an arbitrarily shaped spot of coherent electromagnetic radiation. A light spot may be a result of the projection of a light beam associated with the light pattern.

[0026] The light spot may be at least partially spatially extended. The emitted light pattern may illuminate the surface by a light pattern comprising a plurality of light spots. The light spots may be overlapping at least partially. For example, the number of light spots may be equal to the number of light beams associated with the emitted light pattern. The intensity associated with a light spot may be substantially similar. Substantially similar may refer to intensity values associated with the light spot may differ by less than 50%, preferably less than 30%, more preferably less than 20%. Using patterned light may be advantageous since it can enable the sparing of lightsensitive regions such as the eyes. The pattern may comprise at least one point pattern.

[0027] The term "system" as used herein is a broad term and is to be given its ordinary and cus-tomary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary set of interacting or interdependent components parts forming a whole. Specifically, the components may interact with each other in order to fulfill at least one common function. The at least two components may be handled independently or may be coupled or connectable.

[0028] The term “time-of-flight (TOF) system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary system configured for measuring distances between a detector and the object based on time-of-flight. The TOF system may comprise at least one transmitter and at least one detector, in particular at least one phase detector imager.

[0029] The term “transmitter” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for generating or providing light in the sense of the above-mentioned definition. The transmitter may be and / or may comprise at least one illumination source. The transmitter comprises at least one array of light emitters.

[0030] The array may be a two-dimensional or one dimensional array. The array may comprise a plurality of light emitters arranged in a matrix. The light emitters of the array may be arranged in a one dimensional, e.g. row, array, or a two dimensional array, in particular in a matrix having m rows and n columns, with m, n, independently, being positive integers. The term “matrix” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arrangement of a plurality of elements in a predetermined geometrical order. The matrix specifically may be or may comprise a rectangular matrix having one or more rows and one or more columns. The rows and columns specifically may be arranged in a rectangular fashion. However, other arrangements are feasible, such as non- rectangular arrangements. The light emitters may be arranged such that the pattern is a hexagonal pattern. Thus, for example, the matrix may be a hexagonal matrix.

[0031] The term “light emitter”, also abbreviated as emitter, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing at least one light beam.

[0032] The emitters may be configured for emitting light pulses. Each of the light emitters may be configured for emitting light beams one after the other, e.g. in time intervals, e.g. controlled by at least one control unit of the TOF-system.

[0033] The control unit may be configured for controlling the transmitter and the detector of the TOF system. The controlling may comprise providing control signals, in particular high speed signals, to the transmitter and the detector. The controlling may comprise synchronizing the transmitter and the detector. The control unit may comprise driver electronics configured for performing the named operations.

[0034] The emitter may comprise one or more of at least one laser source, at least one light emitting diode or at least one laser diode. For example, the laser source may be at least one vertical cavity surface emitting laser (VCSEL). The term “vertical-cavity surface-emitting laser” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a semiconductor laser diode configured for laser beam emission perpendicular with respect to a top surface. Examples for VCSELs can be found e.g. in en.wikipedia.org / wiki / Verticalcavity_surface-emitting_laser.

[0035] For example, the transmitter may be or may comprise at least one Radio Frequency (RF)-modu- lated light source. In this case, the light emitter may be and / or may comprise at least one light emitting diode or at least one laser diode. The transmitter may be configured for modulating the light generated by the light emitter with an RF carrier. The transmitter may be configured for modulating the light with an RF carrier frequency from 100 kHz to 300 MHz, preferably from 80 MHz to 200 MHz. For example, the transmitter may be configured for modulating the light with high speeds up to 100 MHz.

[0036] For example, the TOF system may comprise at least one direct TOF imager. In this case, the light emitter may be and / or may comprise at least one laser source, e.g. at least one infrared laser source. This can allow making the illumination unobtrusive. A single pulse per frame from 10 to 100 Hz, preferably from 15 to 80 Hz, more preferably from 20 to 60 Hz may be used. For example, a single pulse per frame, e.g. 30 Hz, may be used. The transmitter may comprise at least one imaging and / or collimating optic comprising at least one optical element selected from the group consisting of: at least one refractive lens; at least one meta surface lens; and at least one diffractive optical element (DOE). The optical element is configured for collimating and / or replicating light. The term “collimating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to aligning a direction of motion of impinging, e.g. diverging, light beams into parallel rays. The term “replicating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to increasing, e.g. duplicating, the number of spots, e.g. the spots generated by the light emitters. The optical element may comprise at least one diffractive optical element (DOE) and / or at least one meta surface element. The DOE and / or the meta surface element may be configured for generating multiple light beams from a single incoming light beam. For example, the emitters may generate up to 2000 spots and an optical element comprising a plurality of meta surface elements may be used to duplicate the number of spots. Further arrangements, particularly comprising a different number of projecting emitters and / or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible.

[0037] Step i) comprises generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams by using at least one detector of the time-of-flight system.

[0038] The light beams illuminating the object may be reflected by the object and are imaged by the detector of the TOF system. The term “detector of the time-of-flight system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for detecting impinging light, in particular for TOF analysis. The detector may be and / or may comprise at least one image sensor. The detector comprises at least one pixelated imaging element comprising a plurality of optical sensors. The optical sensors may be arranged in a two dimensional matrix. For example, the detector may comprise at least one CMOS sensor or at least one CCD chip.

[0039] The present invention proposes a combination of two methods by using a ToF detector only, in particular without any further or additional light detector or light sensor. The detector of the TOF system is used for TOF detection and for light detection for generating the pattern image. Thus, the TOF detector may have two functions. The TOF detector may be configured for measuring the time the light has taken to travel from the transmitter to the object and from the object to detector and for detecting light for generating the pattern image.

[0040] Each of the pixels of the detector may be configured for measuring the time the light has taken to travel from the transmitter to the object and back to detector. For example, in case of using the RF-modulated light source, the detector of the TOF system may be configured for measuring a phase shift of the carrier. For example, in case of using a direct time-of-flight measurement principle, the detector may be configured for determining the time for a single pulse to leave the transmitter and reflect back to the detector. In particular, when running in the so-called trigger mode, the detector can generate a 3D image comprising spatial and temporal data. The detector, e.g. each of the pixels, may be configured for measuring intensity of impinging light, in particular an amplitude of the modulated light on the detector.

[0041] The term “image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to data recorded by using the detector, such as a plurality of electronic readings from the CMOS or CCD chip. The image may comprise raw image data or may be a pre-processed image. For example, the pre-processing may comprise applying at least one filter to the raw image data and / or at least one background correction and / or at least one background subtraction.

[0042] The optical sensors of the detector may be configured for measuring an intensity of light from the object under illumination by the pattern of light beams. Each of the pixels of the detector may be configured for measuring the intensity of the impinging light, in particular of the light beams reflected from the object under illumination by the light pattern. The detector is configured for generating a pattern image. The term “pattern image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an image generated by the detector of the time-of-flight system while the object is being illuminated by the pattern of light beams. The pattern image may comprise an image showing the object, in particular at least parts of the face of the user, while the user is being illuminated with the pattern, particularly on a respective area of interest comprised by the image. Thus, the detector may yield an intensity image of the object projected with the pattern. The pattern image may be generated by imaging and / or recording light reflected by an object, which is illuminated by the light pattern.

[0043] Thus, the present invention proposes using a TOF system for generating an image, in particular a 2D image, e.g. a grayscale image, to be evaluated for extracting liveness data by beam profile analysis.

[0044] This image can be processed in step iii), e.g. by using beam profile analysis, thereby obtaining material information about the object.

[0045] Step iii) comprises extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0046] For example, the pattern image may be reduced to a predefined size, e.g. by applying one or more image processing techniques. The reducing may comprise selecting at least one area of interest and cutting the pattern image to the area of the pattern image of the predefined size. The area of the pattern image of the predefined size may be associated with the object. The part of the image other than the area of the pattern image of the predefined size may be associated with background and / or may be independent of the object. The part of the pattern image useful for the subsequent analysis may be selected.

[0047] Determining if the object corresponds to a living organism based on the pattern image may comprise extracting liveness data from the pattern image. Particularly thereby, it may be determined that the object is a human.

[0048] The term “liveness data” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to data providing an indication that the object corresponds to a living organism. In particular, extracting liveness data comprises extracting material data and / or extracting blood perfusion data and / or extracting at least one surface roughness measure.

[0049] For example, determining if the object corresponds to a living organism based on the at least one pattern image may comprise extracting material data from the pattern image. Particularly thereby, it may be determined that the user is a human. The material data may comprise an item of information on the type of material of the user detected in the pattern image. Extracting material data from the pattern image may be or may comprise generating the material type and / or data derived from the material type. The material data may comprise an item of information on the type of material of the object detected in the pattern image.

[0050] The extracting of material data from the pattern image may comprise beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , WO 2020 / 187719 A1 , WO 2023 / 156469 A1 , WO 2023 / 156315 A1 , the full content of which is included by reference. Beam profile analysis can allow for providing a reliable classification of the object based on a few light spots. Each of the light spots of the pattern image may comprise a beam profile. As used herein, the term “beam profile” may generally refer to at least one intensity distribution of the light spot on the optical sensor as a function of the pixel. The beam profile may be selected from the group consisting of a trapezoid beam profile; a triangle beam profile; a conical beam profile and a linear combination of Gaussian beam profiles.

[0051] Determining if the object corresponds to a living organism based the at least one pattern image may comprise determining if the extracted material data corresponds a desired material data. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin. Comparing the material data with desired material data may comprise determining a similarity of the extracted material data and the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon.

[0052] For extracting the material data, the complete pattern image may be used. Alternatively, partial images may be used. Extracting of material data may include generating one or more partial images from the pattern image. For example, different regions of the pattern image may be selected as partial images. The partial images may be different from each other. In particular, the partial images may be non-overlapping. Using partial images may allow having material data from different areas of the object (e.g. having different light conditions) and / or comparing the extracted material data and / or generating of a material map and / or reducing an uncertainty on the obtained material data.

[0053] In an embodiment, extracting material data from the pattern image may comprise generating the material type and / or data derived from the material type. Preferably, extracting material data may be based on the pattern image.

[0054] Material data may be extracted by using at least one model. Extracting material data may comprise providing the pattern image to at least one model and / or receiving material data from the model. Extracting material data may include providing the image to a model and / or receiving material data from the model. Providing the image to a model may comprise and may be followed by receiving the image at an input layer of the model or via a model loss function.

[0055] The model may be a data-driven model. The data-driven model may comprise a convolutional neural network and / or an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy models, normalizing flows, GANs, vision transformers, or transformers used for natural language processing, autoregressive image modelling. GANs, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models, Vision Transformers. Supervised or unsupervised schemes may be applicable to generate representation, also embedding in e.g. cosine or Euclidian metric in in ML language.

[0056] The data-driven model may be parametrized according to a training data set including at least one image and material data, preferably at least one pattern image and material data. In particular, the training data set comprises a plurality of historical pattern images and corresponding material data. In another embodiment, extracting material data may include providing the image to a model and / or receiving material data from the model. In another embodiment, data-driven model may be trained according to a training data set including at least one image and material data. In another embodiment, data-driven model may be parametrized according to a training data set including at least one image and material data. The data-driven model may be parametrized according to a training data set to receive the image and provide material data based on the received image. The data-driven model may be trained according to a training data set to receive the image and provide material data as output based on the received image. The training data set may comprise at least one image and material data, preferably material data associated with the at least one image.

[0057] The extracting of material data may comprise generating a representation associated with the pattern image of the object such as at least one tensor representing the pattern image or a dimensionality-reduced representation of the pattern image. The image may comprise a representation of the image. The representation may be a lower dimensional representation of the image, e.g. a tensor. The representation may comprise at least a part of the data or the information associated with the image. Representation of an image may comprise a feature vector. In an embodiment, determining a representation, in particular a lower-dimensional representation may be based on principal component analysis (PCA) mapping or radial basis function (RBF) mapping. Determining a representation may also be referred to as generating a representation. Generating a representation based on PCA mapping may include clustering based on features in the pattern image and / or partial image. Additionally or alternatively, generating a representation may be based on neural network structures suitable for reducing dimensionality. Neural network structures suitable for reducing dimensionality may comprise encoder and / or decoder. In an example, the neural network structure may be an autoencoder. In an example, neural network structure may comprise a convolutional neural network (CNN). CNN may comprise at least one convolutional layer and / or at least one pooling layer. CNNs may reduce the dimensionality of a partial image and / or an image by applying a convolution, eg based on a convolutional layer, and / or by pooling. Applying a convolution may be suitable for selecting feature related to material information of a partial image.

[0058] In an embodiment, a model may be suitable for determining an output based on an input. In particular, model may be suitable for determining material data based on an image as input. A model may be a deterministic model, a data-driven model or a hybrid model. The deterministic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principles models. A deterministic model may comprise a set of equations that describe an interaction between the material and the patterned electromagnetic radiation thereby resulting in a condition measure, a vital sign measure or the like. A data-driven model may be a classification model. A hybrid model may be a classification model comprising at least one machine-learning architecture with deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. In an embodiment, the data-driven model may be a classification model. The classification model may comprise at least one machine-learning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbours, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The data-driven model may be parametrized according to a training data set. The data-driven model may be trained based on the training data set. Training the model may include parametrizing the model. The term training may also be denoted as learning. The term specifically may refer, without limitation, to a process of building the classification model, in particular determining and / or updating parameters of the classification model. Updating parameters of the classification model may also be referred to as retraining. Retraining may be included when referring to training herein. In an embodiment, the training data set may include at least one image and material information.

[0059] In an embodiment, extracting material data from the image with a data-driven model may comprise providing the image to a data-driven model. Additionally or alternatively, extracting material data from the image with a data-driven model may comprise may comprise generating an embedding associated with the image based on the data-driven model. An embedding may refer to a lower dimensional representation associated with the image such as a feature vector. Feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and / or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven model based on the embedding associated with the image. Additionally or alternatively, extracting material data from the image by providing the image to a data-driven model may comprise transforming the image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and / or material feature vector may be used for determining material data.

[0060] The determining of the object corresponds to a living organism may comprise comparing the extracted material data to desired material data. The object is determined to correspond to a living organism in case the material data matches the desired material data at least within tolerances. Otherwise, in case the material data does not match the desired material data, the object is determined to correspond to a non-living organism.

[0061] Determining if the extracted material data corresponds a desired material data may comprise determining a similarity of the extracted material data and the desired material data. Determining a similarity of the extracted material data and the desired material data may comprise comparing the extracted material data with the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin. The object is determined to correspond to a human in case the material data matches the material data of skin. Otherwise, in case the material data does not match the material data of skin, the object is determined to correspond to a non-living organism.

[0062] In an embodiment, step iii) may comprise generating at least one feature vector from the material data and matching the material feature vector with at least one associated reference template vector for material.

[0063] Determining if the object corresponds to a living organism based on the at least one pattern image may comprise determining at least one blood perfusion measure. Particularly thereby, it may be determined that the human is living.

[0064] The term “blood perfusion measure" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a blood volume flow through a given volume or mass of tissue. Typically, the blood perfusion measure may be given in units of ml / ml / s or ml / 100 g / min. The blood perfusion measure may represent a local blood flow through the at least one capillary network and one or more extracellular spaces in a body tissue.

[0065] Determining the at least one blood perfusion measure may comprise determining at least one speckle contrast of the pattern image. Alternatively or in addition, determining the at least one blood perfusion measure may comprise determining a blood perfusion measure based on the determined at least one speckle contrast. The term “speckle contrast " as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a degree of a variation in a speckle pattern generated by coherent light. The speckle pattern may be generated by the transmitter, particularly on the object. A speckle contrast may represent a measure for a mean contrast of an intensity distribution within an area of a speckle pattern. In particular, a speckle contrast K over an area of the speckle pattern may be expressed as a ratio of standard deviation o to the mean speckle intensity <l>, i.e.,

[0066] Speckle contrast may comprise a speckle contrast value. Speckle contrast values may be distributed between 0 and 1. The blood perfusion measure may be determined based on the speckle contrast. The blood perfusion measure may depend on the determined speckle contrast. If the speckle contrast changes, the blood perfusion measure derived from the speckle contrast may change accordingly. A blood perfusion measure may be a single number or value that may represent a likelihood that the object is a living subject. For monitoring of speckle contrast changes a plurality of pattern images generated at different points in time may be used.

[0067] For determining the speckle contrast, the complete pattern image may be used. Alternatively, for determining the speckle contrast, a section of the pattern image may be used. The section of the pattern image, preferably, represents a smaller area of the pattern image than an area of the complete pattern image.

[0068] In an embodiment, a data-driven model may be used for determining a blood perfusion measure. Data-driven model be parametrized and / or trained based on a training data set. The training data set may comprise a pattern image and a blood perfusion measure. In particular, the training data set comprises a plurality of historical pattern images and corresponding blood perfusion measure. The data-driven model may be parametrized and / or trained based on the training data set to output a blood perfusion measure based on receiving a pattern image. With respect to embodiments of the data-driven model, reference is made to the description of the data-driven model with respect to the extraction of material data, respectively.

[0069] In an embodiment, the determining if an object corresponds to a living organism based on the blood perfusion data may comprise determining if the blood perfusion measure corresponds to blood perfusion measure of a human being. Determining if the blood perfusion measure corresponds a human being may comprise comparing the blood perfusion measure to at least one pre-defined or pre-determined range of values of blood perfusion measure, e.g. stored in at least one database. In case the extracted blood perfusion measure is at least within tolerances within the re-defined or pre-determined range of values of blood perfusion measure, the object is determined to correspond to a living organism, otherwise not.

[0070] As outlined above, the extracting of the liveness data may comprise extracting at least one surface roughness measure. The surface roughness measure may be extracted from at least one speckle image of the object while the object is being illuminated. For example, the object may to illuminated by coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm. The term “coherent” electromagnetic radiation as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to light pattern and / or a plurality of light beams that have at least essentially a fixed phase relationship between electric field values at different locations and / or at different times. In particular, the coherent electromagnetic radiation may refer to electromagnetic radiation that is able to exhibit interference effects. The term “coherent” may also comprise partial coherence, i.e. a non-perfect correlation between phase values. The electromagnetic radiation may be completely coherent, wherein deviations of about ± 10% of phase relationship are possible. The coherent electromagnetic radiation is associated with a wavelength between 850 nm and 1400 nm. Preferably, the coherent electromagnetic radiation may be in the infrared range. Preferably, the coherent electromagnetic radiation may be associated with a wavelength between 880 nm and 1300 nm. In particular, the coherent electromagnetic radiation is associated with a wavelength between 900 nm and 1000 nm and / or the coherent electromagnetic radiation is associated with a wavelength between 1100 nm and 1200 nm. This may be advantageous since the sunlight has a bandgap in these regions. Followingly, the coherent electromagnetic radiation with the abovementioned wavelengths illuminated for generating the speckle image can be dif- ferentiated easier from incoming sun light. Followingly, the use of coherent electromagnetic radiation within the above-specified region may enable measurements of surface roughness even in the presence of sun light such as in the nature. Consequently, measurements of surface roughness can be easily and location-independently used. Overall, an improved signal-to-noise ratio can be achieved and the accuracy of evaluations of surface roughness can be increased.

[0071] The speckle image may comprise one or more light spots. A projection of patterned coherent electromagnetic radiation onto a regular surface may result in a light spot projected onto the regular surface independent of speckle. A projection of patterned coherent electromagnetic radiation onto a regular surface may result in a light spot projected onto the irregular surface comprising at least one speckle, preferably a plurality of speckles. The object may be associated with an at least partially irregular surface. Followingly, the speckle image may comprise a plurality of speckles. For example, if the patterned coherent electromagnetic radiation is projected at least partially on the skin of the user, a plurality of speckle is formed due to the interference of the coherent electromagnetic radiation. Followingly, a light spot may comprise zero, one or more speckle depending on the surface the patterned coherent electromagnetic radiation is projected on. Skin may have an irregular surface. Hence the projection of patterned coherent electromagnetic radiation may result in the formation of speckle within the one or more light spots. Projecting coherent electromagnetic radiation on an irregular surface results in the formation of speckle. Followingly, the light spot may comprise one or more speckle. A light spot may have a diameter between 0.5 mm and 5 cm, preferably 0.6 mm and 4 cm, more preferably, 0.7 mm and 3 cm, most preferably 0.4 and 2 cm.

[0072] The term “speckle image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an image showing a plurality of speckles. The speckle image may show a plurality of speckles. The speckle image may comprise an image showing the user, in particular at least one part of the face of the user, while the user is being illuminated with the coherent electromagnetic radiation, particularly on a respective area of interest comprised by the image. The speckle image may be generated while the user may be illuminated by coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm. The speckle image may show a speckle pattern. The speckle pattern may specify a distribution of the speckles. The speckle image may indicate the spatial extent of the speckles. The speckle image may be suitable for determining a surface roughness measure. The speckle image may be generated with at least one camera, e.g. of the detector of the TOF-system. For generating the speckle image, the user may be illuminated by the illumination source.

[0073] The term “speckle” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an optical phenomenon caused by interfering coherent electromagnetic radiation due to non-regular or irregular surfaces. Speckles may appear as contrast variations in an image such as a speckle image. The term “speckle pattern” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a distribution of the plurality of speckles. The distribution of the plurality of speckles may refer to a spatial distribution of at least one of the plurality of speckles and / or a spatial distribution of at least two of the plurality of speckles in relation to each other. The spatial distribution of the at least one of the plurality of speckles may refer to and / or specify a spatial extent of the at least one of the plurality of speckles. The spatial distribution of the at least two of the plurality of speckles may refer to and / or specify a spatial extent of the first speckle of the at least two speckles in relation to the second speckle of the at least two speckles and / or a distance between the first speckle of the at least two speckles and the second speckle of the at least two speckles.

[0074] As outlined above, the speckles may be caused by the irregularities of the surface, the speckles reflect the roughness of the surface. Followingly, determining the surface roughness measure based on the speckles in the speckle image utilizes the relation between the speckle distribution and the surface roughness. Thereby, a low-cost, efficient and readily available solution for surface roughness evaluation can be enabled.

[0075] The generating of the speckle image may be initiated by a user action or may automatically be initiated, e.g. once the presence of a user within a field of view and / or within a predetermined sector of the field of view of the camera is automatically detected. The term “field of view” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an angular extent of the observable world and / or at least one scene that may be captured or viewed by an optical system, such as the image generation unit. The field of view may, typically, be expressed in degrees and / or radians, and, exemplarily, may represent the total angle spanned by the image and / or viewable area.

[0076] For example, the camera may comprise the at least one image sensor and at least one further optical element. For example, the further optical element may be at least one lens. A lens may refer to an optical element suitable for influencing the expansion of the light beam associated with the coherent electromagnetic radiation. For example, the further optical element may be at least one polarizer. For example, the camera may comprise at least one image sensor, at least one lens and at least one polarizer. The polarizer may refer to an optical element suitable for selecting the electromagnetic radiation according to its polarization. In particular, the polarizer may be an optical element suitable for selecting the coherent electromagnetic radiation according to polarization its. Followingly, a part of the electromagnetic radiation, in particular coherent electromagnetic radiation may pass the polarizer while the rest of the electromagnetic radiation, in particular coherent electromagnetic radiation, may be turned away at least partially and / or may be absorbed at least partially. As the coherent electromagnetic radiation associated with a wavelength between 850 nm and 1400 nm penetrates the skin deeply, a part of the information received from the light reflected from the skin comprises information independent from the surface roughness which distracts the measurement of the surface roughness. To increase the sig- nal-to-noise ratio a polarizer can be used. The coherent electromagnetic radiation reflected from the surface of the object is usually polarized differently as the light reflected from deeper layers of the human skin. Followingly, the polarizer enables a selection of the desired signal from the undesired signal.

[0077] For example, a distance between the user and the camera used for generating the speckle image is between 10 cm and 1.5 m and / or wherein the distance between the user and an illumination source used for illuminating the user is between 10 cm and 1 .5 m. Preferably, the distance between the user and the camera may be between 20 cm and 1 .2 m. Preferably, the distance between the user and the illumination source may be between 20 cm and 1 .2 m. Adjusting the distance between object and camera ensure that a speckle image of sufficient quality is generated. Followingly, the above-specified distances enable a correct and reliable determination of the surface roughness. This is especially important in non-static contexts, where a user may operate the device by himself.

[0078] For example, the speckle image may show the user while being illuminated by coherent electromagnetic radiation and the surface roughness of the user’s skin may be determined. Preferably, the user may have generated the speckle image and / or initiated the generation of the speckle image. Preferably, the speckle image may be initiated by the user operating an application of a mobile electronic device. By doing so, the user can decide on his or her own when to determine the surface roughness of her or his skin. Followingly, a non-expert user is enabled to determine surface roughness and measurements can be carried out in more natural and less artificial contexts. Thereby, the surface roughness can be evaluated more realistically which in turn serves more realistic measure for the surface roughness. For example, the skin may have different surface roughness during the course of the day depending on the activity of the human. Doing sports may influence the surface roughness as well as creaming the skin. This influence can be verified with the herein described methods and systems.

[0079] For example, the speckle image may be associated with a resolution of less than 5 megapixel. Preferably, the speckle image may be associated with a resolution of less than 3 megapixel, more preferably less than 2.5 megapixel, most preferably less than 2 megapixel. Such speckles images can be generated with readily available, small and cheap smartphone cameras. Furthermore, the storage and processing capacities needed for evaluating the surface roughness measure are small. Thus, the low resolution of the speckle image used for evaluating the surface roughness enables the usage of mobile electronic devices for evaluating the surface roughness, in particular devices like smartphone or wearables since these devices have strictly limited size, memory and processing capacity.

[0080] The method may further comprise reducing the speckle image to a predefined size prior to determining the surface roughness measure. Reducing the speckle image to a predefined size may be based on applying one or more image augmentation techniques. Reducing the speckle image to a predefined sized may comprise selecting an area of the speckle image of the predefined size and cutting the speckle image to the area of the speckle image of the predefined size. The area of the speckle image of the predefined size may be associated with the living organism such as the human, in particular with the skin of the living organism such as the skin of the human. The part of the image other than the area of the speckle image of the predefined size may be associated with background and / or may be independent of the living organism such as a human. By doing so a reduced amount of data needs to be processed which decreases the time needed for determining the surface roughness or allows for less storage and processor to be needed. Furthermore, the part of the image useful for the analysis is selected. Hence, reducing the size may result in disregarding parts of the speckle image independent of the object or living organism such as a human. Followingly, the surface roughness measure can be determined easily and for the analysis disturbing parts not relating to the user are ignored.

[0081] For example, image augmentation techniques may comprise at least one of scaling, cutting, rotating, blurring, warping, shearing, resizing, folding, changing the contrast, changing the brightness, adding noise, multiply at least a part of the pixel values, drop out, adjusting colors, applying a convolution, embossing, sharpening, flipping, averaging pixel values or the like.

[0082] For example, the method may further comprise reducing the speckle image to a predefined size based on detecting the user in the speckle image. In particular, the speckle image may be reduced to a predefined size based on detecting the user in the speckle image prior to determining the surface roughness measure. In particular, reducing the speckle image to the predefined size based on detecting the user in the speckle image may comprise detecting the contour of the user, e.g. detecting the contour of a user’s face and reducing the speckle image to an area associated with the user, in particular with an area associated with the user’s face. Preferably the area associated with the user may be within the contour of the user, in particular the user and / or the contour of a user’s face.

[0083] The method may further comprise receiving at least one flood image. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by the user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user.

[0084] The flood image may show the contour of the user. The contour of the user may be detected based on the flood image. Preferably, the contour of the user may be detected by providing the flood image to an object detection data-driven model, in particular a user detection model, wherein object detection data-driven model may be parametrized and / or trained to receive the flood image and provide an indication on the contour of the user based on a training data set. The training data set may comprise flood images and indications on the contour of objects and / or humans. The indication of the contour may include a plurality of points indicating the location of a specific landmark associated with the user. For example, where the speckle image may be associated with a user’s face, the user’s face may be detected based on the contour, wherein the contour may indicate the landmarks of the face such as the nose point or the outer corner of the lips or eyebrows.

[0085] The term “surface roughness” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a property of a surface associated with the user. In particular, the surface roughness may characterize lateral and / or vertical extent of surface features. The surface roughness may be evaluated based on the surface roughness measure. The surface roughness measure may quantify the surface roughness.

[0086] The term “surface feature” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrarily shaped structure associated with the surface, in particular of the user. In particular, the surface feature may refer to a substructure of the surface associated with the user. A surface may comprise a plurality of surface features. For example, an uplift or a sink may be surface features. Preferably, a surface feature may refer to a part of the surface associated with an angle unequal to 90° against the surface normal.

[0087] The term “surface roughness measure” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a measure suitable for quantifying the surface roughness. Surface roughness measure may be related to the speckle pattern. For example, surface roughness measure may comprise at least one of a fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof. Preferably, the surface roughness measure may be suitable for describing the vertical and lateral surface features. Surface roughness measure may comprise a value associated with the surface roughness measure. Surface roughness measure may refer to a term of a quantity for measuring the surface roughness and / or to the values associated with the quantity for measuring the surface roughness. The determining of the surface roughness measure based on the speckle image may refer to determining the surface roughness measure based on a speckle pattern in the speckle image.

[0088] The surface roughness measure may be determined based on the speckle image by providing the speckle image to a model and receiving the surface roughness measure from the model. For example, the model may be suitable for determining an output based on an input. In particular, model may be suitable for determining a surface roughness measure based on the speckle image, preferably based on receiving speckle image. For example, the model may be or may comprise one or more of a physical model, a data- driven model or a hybrid model. A hybrid model may be a model comprising at least one data- driven model with physical or statistical adaptations and model parameters. Statistical or physical adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. For example, a data-driven model may represent a correlation between the surface roughness measure and the speckle image. The data- driven model may obtain the correlation between surface roughness measure and speckle image based on a training data set comprising a plurality of speckle images and a plurality of surface roughness measures. For example, the data-driven model may be parametrized based on a training data set to receive the speckle image and provide the surface roughness measure. The data-driven model may be trained based on a training data set. The training data set may comprise at least one speckle image and at least one corresponding surface roughness measure. The training data set may comprise a plurality of speckle image and a plurality of surface roughness measures. Training the model may comprise parametrizing the model. The data- driven model may be parametrized and / or trained to provide the surface roughness measure based on the speckle image, in particular receiving the speckle image. Determining the surface roughness measure based on the speckle image may comprise providing the speckle image to a data-driven model and receiving the surface roughness measure from the data-driven model. Providing the surface roughness measure based on the speckle image may comprise mapping the speckle image to the surface roughness measure. The data-driven model may be parametrized and / or trained to receive the speckle image. Data-driven model may receive the speckle image at an input layer. The term training may also be denoted as learning. The term specifically may refer, without limitation, to a process of building the data-driven model, in particular determining and / or updating parameters of the data-driven model. Updating parameters of the data-driven model may also be referred to as retraining. Retraining may be included when referring to training herein. During the training the data-driven model may adjust to achieve best fit with the training data, e.g. relating the at least on input value with best fit to the at least one desired output value. For example, if the neural network is a feedforward neural network such as a CNN, a backpropagation-algorithm may be applied for training the neural network. In case of a RNN, a gradient descent algorithm or a backpropagation-through-time algorithm may be employed for training purposes. Training a data-driven model may comprise or may refer without limitation to calibrating the model.

[0089] For example, the physical model may reflect physical phenomena in mathematical form, e.g., including first-principles models. A physical model may comprise a set of equations that describe an interaction between the object and the coherent electromagnetic radiation thereby resulting in a surface roughness measure. The physical model may be based on at least one of a fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof. In particular, the physical model may comprise one or more equations relating the speckle image and the surface roughness measure based on equations relating to the fractal dimension, speckle size, speckle contrast, speckle modulation, roughness exponent, standard deviation of the height associated with surface features, lateral correlation length, average mean height, root mean square height or a combination thereof.

[0090] For example, the fractal dimension may be determined based on the Fourier transform of the speckle image and / or the inverse of the Fourier transform of the speckle image. For example, the fractal dimension may be determined based on the slope of a linear function fitted to a double logarithmic plot of the power spectral density versus a frequency obtained by Fourier transform. The speckle size may refer to the spatial extent of one or more speckles. Where the speckle size may refer to the spatial extent of more than one speckle, the speckle size may be determined based on an average of more than one speckle sizes and / or a weighting of the more than one speckle sizes. The speckle contrast may refer to a measure for the standard deviation of at least a part of the speckle image in relation to the mean intensity of at least the part of the speckle image. The speckle modulation may refer to a measure for the intensity fluctuation associated with the speckles in at least a part of the speckle image. Roughness exponent, standard deviation of the height associated with surface features, lateral correlation length or a combination thereof may be determined based on the autocorrelation function associated with the double logarithmic plot of the power spectral density versus a frequency obtained by Fourier transform.

[0091] For example, determining the surface roughness measure based on the speckle image may comprise determining the surface roughness measure based on a speckle pattern. For example, determining the surface roughness based on the speckle pattern may comprise determining the surface roughness based on a distribution of a plurality of speckles in the speckle image. Determining the surface roughness measure based on the distribution of the plurality of speckles in the speckle image may refer to determining the distribution of the plurality of speckles in the speckle image. Determining the distribution of the speckles may comprise determining at least one of fractal dimension associated with the speckle image, speckle size associated with the speckle image, speckle contrast associated with the speckle image, speckle modulation associated with the speckle image, roughness exponent associated with the speckle image, standard deviation of the height associated with surface features associated with the speckle image, lateral correlation length associated with the speckle image, average mean height associated with the speckle image, root mean square height associated with the speckle image or a combination thereof.

[0092] Additionally or alternatively, determining the surface roughness measure may comprise determining at least one of fractal dimension associated with the speckle image, speckle size associated with the speckle image, speckle contrast associated with the speckle image, speckle modulation associated with the speckle image, roughness exponent associated with the speckle image, standard deviation of the height associated with surface features associated with the speckle image, lateral correlation length associated with the speckle image, average mean height associated with the speckle image, root mean square height associated with the speckle image or a combination thereof. For example, determining the surface roughness measure may be based on the distribution of the speckles in the speckle image. Determining the surface roughness measure based on the distribution of the speckles in the speckle image may comprise determining at least one of a size distribution of the speckles, a power spectral density associated with the speckle image, a fractal dimension associated with the speckle image, a speckle contrast, a speckle modulation or a combination thereof.

[0093] Additionally or alternatively, determining the surface roughness measure based on the distribution of the speckles in the speckle image may comprise providing the speckle image to a model, in particular a data-driven model, wherein the data-driven model may be parametrized and / or trained based on a training data set comprising one or more speckle image and one or more corresponding surface roughness measure.

[0094] For example, the surface roughness measure may be determined based on the speckle image by providing the speckle image to a model and receiving the surface roughness measure from the model. The model may be a data-driven model and may be parametrized and / or trained based on a training data set comprising a plurality of speckle images and corresponding surface roughness measures or indications of surface roughness measures. Additionally or alternatively, the model may be a physical model.

[0095] For example, the method may further comprise generating a partial speckle image. A partial speckle image may refer to a partial image generated based on the speckle image. The partial speckle image may be generated by applying one or more image augmentation techniques to the speckle image.

[0096] For example, the method may further comprise generating a first speckle image and a second speckle image. The speckle image may comprise the first speckle image and the second speckle image. The first speckle image may refer to a first part of the speckle image. The second speckle image may refer to a second part of the speckle image. Preferably, the first speckle image and the second speckle image may be different from each other. In particular, the first speckle image and the second speckle image may be non-overlapping. The first speckle image and the second speckle image may be generated by applying one or more image augmentation techniques to the speckle image. Determining the surface roughness measure based on the speckle image may comprise determining a first surface roughness measure based on the first speckle image and determining a second surface roughness measure based on the second speckle image. Providing the surface roughness measure may include providing the first surface roughness measure and the second surface roughness measure. In particular, the first surface roughness measure and the second surface roughness measure may be provided together. Preferably, the first surface roughness measure and the second surface roughness measure may be provided in a surface roughness measure map indicating the spatial distribution of surface roughness measures. For example, the surface roughness measure map may indicate the first surface roughness measure associated with a first area in the surface roughness measure map and the second surface roughness measure map may indicate the second surface roughness measure associated with a second area in the surface roughness measure map. In particular, the surface roughness measure map may be similar to a heat map, wherein the surface roughness measures may be plotted against the area associated with the respective surface roughness measures.

[0097] The surface roughness measure may be determined by using at least one processor.

[0098] For example, the method may comprise determining if the surface roughness measure corresponds to a surface roughness measure of a human being. For example, the method may comprise determining if the surface roughness measure corresponds to a surface roughness measure of the specific user. Determining if the surface roughness measure corresponds to a surface roughness measure of a human being and / or of the specific user may comprise comparing the surface roughness measure to at least one pre-defined or pre-determined range of values of surface roughness measure, e.g. stored in at least one database e.g. of the device or of a remote database such as of a cloud. In case the determined surface roughness measure is at least within tolerances within the re-defined or pre-determined range of values of surface roughness measure, the user is authenticated otherwise the authentication is unsuccessful. For example, the surface roughness measure may be a human skin roughness. In case the determined human skin roughness is within the range of 10 pm to 150 pm the object is considered as human being. However, other ranges are possible.

[0099] The method may be computer implemented. The term "computer implemented" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method involving at least one computer and / or at least one computer network. The computer and / or computer network may comprise at least one processor which is configured for performing at least one of the method steps of the method according to the present invention. Specifically, each of the method steps is performed by the computer and / or computer network. The method may be performed completely automatically, specifically without user interaction. For example, the illuminating and / or the generating of images may be triggered and / or executed by using at least one processor.

[0100] A single processing device may be configured to exclusively perform at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one of the embodiments of the method according to the present invention. Herein, the computer program as executed on the single processing device may comprise all instructions causing the computer to carry out the method. Alternatively, or in addition, at least one method step may be performed by using at least one remote device, especially selected from at least one of a server or a cloud server, particularly when the device and the remote device may be part of a computer network. In this case, the computer program may comprise at least one remote component to be executed by the at least one remote processing device to carry out the at least one method step. Further, the computer program may comprise at least one interface configured to forward to and / or receive data from the at least one remote component of the computer program.

[0101] In a further aspect, a system for determining if an object corresponds to a living organism is disclosed. The system comprises at least one time-of-flight system comprising at least one transmitter for generating a pattern of light beams for illuminating the object, wherein the transmitter comprises at least one array of light emitters, wherein the time-of-flight system further comprises at least one detector configured for generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams, at least one processor configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0102] The term “processor” as generally used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor, or computer processor may be configured for processing basic instructions that drive the computer or system. It may be a semi-conductor based processor, a quantum processor, or any other type of processor configures for processing instructions. As an example, the processor may be or may comprise a Central Processing Unit ("CPU"). The processor may be a (“GPU”) graphics processing unit, (“TPU”) tensor processing unit, ("CISC") Complex Instruction Set Computing microprocessor, Reduced Instruction Set Computing ("RISC") microprocessor, Very Long Instruction Word ("VLIW") microprocessor, or a processor implementing other instruction sets or processors implementing a combination of instruction sets. The processing means may also be one or more special-purpose processing devices such as an Application-Specific Integrated Circuit ("ASIC"), a Field Programmable Gate Array ("FPGA"), a Complex Programmable Logic Device ("CPLD"), a Digital Signal Processor ("DSP"), a network processor, or the like. The methods, systems and devices described herein may be implemented as software in a DSP, in a micro-controller, or in any other side-processor or as hardware circuit within an ASIC, CPLD, or FPGA. It is to be understood that the term processor may also refer to one or more processing devices, such as a distributed system of processing devices located across multiple computer systems (e.g., cloud computing), and is not limited to a single device unless otherwise specified. The processor may also be an interface to a remote computer system such as a cloud service. The processor may include or may be a secure enclave processor (SEP). An SEP may be a secure circuit configured for processing images. A "secure circuit" is a circuit that protects an isolated, internal resource from being directly accessed by an external circuit. The processor may be an image signal processor (ISP) and may include circuitry suitable for processing images, in particular images with personal and / or confidential information. The system may be configured for performing a method for determining if an object corresponds to a living organism according to the present invention, such as according to one or more of the embodiments given above or given in further detail below. For details, options and definitions, reference may be made to the method for determining if an object corresponds to a living organism discussed above and to the method and devices as described below.

[0103] In a further aspect, the present invention discloses a method for authenticating a user of a device, in particular to perform at least one operation on the device that requires authentication. The method steps may be performed in the given order or may be performed in a different order. Further, one or more additional method steps may be present which are not listed. Further, one, more than one or even all of the method steps may be performed repeatedly. For details, options and definitions, reference may be made to the method for determining if an object corresponds to a living organism and the system for determining if an object corresponds to a living organism as discussed above and to the methods and devices as described below.

[0104] The term “user” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a person intended to and / or using the device.

[0105] The device may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0106] The term “authenticating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to verifying an identity of a user. Specifically, the authentication may comprise distinguishing between the user from other humans or objects, in particular between an authorized access from a non-authorized access.

[0107] The authentication may comprise verifying identity of a respective user and / or assigning identity to a user. The authentication may comprise generating and / or providing identity information, e.g. to other devices or units such as to at least one authorization unit for authorization for providing access to the device. The identify information may be proofed by the authentication. For example, the identity information may be and / or may comprise at least one identity token. In case of successful authentication an image of a face recorded by an imaging detector such as of the TOF system or a further imaging detector may be verified to be an image of the user’s face and / or the identity of the user is verified. The authentication may be performed using at least one authentication process. The authentication process may comprise a plurality of steps such as at least one face detection, e.g. on at least one flood image as will be described in more detail below, and at least one identification step in which an identity is assigned to the detected face and / or at least one identity check and / or verifying an identity of the user is performed.

[0108] The authentication may be and / or may comprise a biometric authentication. The term "biometric authentication" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to authentication using at least one biometric identifier such as a distinctive, measurable characteristics used to label and describe individuals. The biometric identifier may be a physiological characteristics.

[0109] The method comprises receiving a request for accessing at least one resource associated with the device and executing at least one authentication process. The authentication process comprises a. determining a distance information of the user by using a time-of-flight system and determining if the user is within or outside of a working range of the authentication process by comparing the distance information to the working range; b. allowing the user to access the resource in case the user is determined to be within the working range and otherwise, in case the user is determined to be outside the working range, denying the user to access the resource.

[0110] The device may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0111] The term “access” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to entering and / or using the one or more functions associated with the device. The term ‘function associated with the device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary function such as access to at least one element and / or at least one resource of the device or associated with the device. The functions that require authentication of the user may be pre-defined. The one or more functions associated with the device may comprise unlocking the device, and / or access to an application, preferably associated with the device and / or access to a part of an application, preferably associated with the device. For example, the function may comprise access to a content of the device, e.g. as stored in a database of the device, and / or retrievable by the device. In an embodiment, allowing the user to access a resource may include allowing the user to perform at least one operation with a device and / or system. The resource may be a device, a system, a function of a device, a function of a system and / or an entity. Additionally and / or alternatively, allowing the user to access a resource may include allowing the user to access an entity. The entity may be physical entity and / or virtual entity. The virtual entity may be a database for example. The physical entity may be an area with restricted access. The area with restricted access may be one of the following: security areas, rooms, apartments, vehicles, parts of the before mentioned examples, or the like. The device may be locked and may only be unlocked by authorized user.

[0112] The device may further comprise at least one communication interface, such as a user interface, configured for receiving a request for accessing at least one resource associated with the device, in particular to perform at least one operation on the device that requires authentication. The term “request for accessing” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one act and / or instance of asking for access. The term “receiving a request” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of obtaining a request, e.g. from a data source and / or the user interface. The receiving may fully or partially take place automatically. The receiving of the request for accessing one or more functions associated with the device may be performed by using at least one communication interface. The receiving may comprise receiving at least one user input, e.g. via at least one user interface e.g. such as a display of the device, and / or a request from a remote device and / or cloud, e.g. via the communication of the device such as via the internet. For example, the request may be generated by or triggered by at least one user input, such as by inputting a security number or other unlocking action by the user, and / or may be send from a remote device and / or cloud such as via a connected account.

[0113] The authentication process may be performed using the at least one authentication unit configured for performing at least one authentication process of a user. The authentication unit may comprise at least one processor. The execution of the authentication process may be triggered and / or started by receiving the request.

[0114] The authentication process may comprise a plurality of steps.

[0115] For example, the authentication process may comprise performing at least one face detection step. The face detection step may comprise analyzing at least one image of the user, e.g. generated by the detector of the TOF system or a further camera. The image may be a flood image.

[0116] The authentication process further may comprise generating at least one flood image showing the user associated while the user is being illuminated by flood light and determining if the identity of the user corresponds to a verified identity based on the flood image. The authentication process may comprises allowing the user to access the resource in case the identity of the user corresponds to a verified identity and otherwise, in case the identity of the user does not correspond to a verified identity, denying the user to access the resource. The method may comprise: illuminating the user with flood light by using at least one the flood illumination source; capturing the at least one flood image by using at least one image generation unit such as the detector of the TOF system or a further camera.

[0117] The term “flood illumination source” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing substantially continuous spatial illumination. The term “flood light” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to substantially continuous spatial illumination, in particular diffuse and / or uniform illumination. The flood light has a wavelength in the infrared range, in particular in the near infrared range. The flood illumination source may comprise at least one LED or at least one least one VCSEL, preferably a plurality of VCSELs. The term “substantially continuous spatial illumination” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to uniform spatial illumination, wherein areas of non-uniform are possible. The area, e.g. covering a user, a portion of the user and / or a face of the user, illuminated from the flood illumination source, may be contiguous. Power may be spread over a whole field of illumination. In contrast, illumination provided by the light pattern may comprise at least two contiguous areas, in particular a plurality of contiguous areas, and / or power may be concentrated in small (compared to the whole field of illumination) areas of the field of illumination. The infrared flood illumination may be suitable for illuminating a contiguous area, in particular one contiguous area. The infrared pattern illumination may be suitable for illuminating at least two contiguous areas.

[0118] The term “flood image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an image generated by the image generation unit while illumination source is emitting infrared flood light, e.g. on an object and / or a user. The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging may be synchronized, e.g. by using at least one control unit.

[0119] The face detection step may comprise analyzing the flood image. For example, the authentication process may comprise performing at least one face detection using the flood image. The face detection may be performed locally on the device. Face identification, i.e. assigning an identity to the detected face, however, may be performed remotely, e.g. in the cloud, e.g. especially when identification needs to be done and not only verification. User templates can be stored at the remote device, e.g. in the cloud, and would not need to be stored locally. This can be an advantage in view of storage space and security.

[0120] The authentication process may comprise identifying the user based on the flood image. The term “identifying” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality.

[0121] The identifying may comprise assigning an identity to a detected face and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. For matching the flood image with a template, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered and / or evaluated. The template vector may be obtained from a template image.

[0122] The template image may be generated in an enrollment process. The term “enrollment process" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one step of registering, particularly to a service. In the enrollment process, the template image may be generated under secure conditions in a manner that it is guaranteed that the generated template image shows the user. The enrollment process may comprise at least one step of: capturing the template image; recording personal data and the like.

[0123] The face detection may comprise analyzing the flood image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network.

[0124] For example, the authentication may comprise identifying the user. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The identifying may comprise performing a face verification of the im- aged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template, e.g. a template image generated within an enrollment process. The identifying of the user may comprise determining if the imaged face is the face of the user, in particular if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device. Authentication may be successful if the flood image can be matched with an image template. Authentication may be unsuccessful if the flood image cannot be matched with an image template.

[0125] The term “memory" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one electronic storage space configured for storing data, instructions, and programs. The stored data, instruction and / or programs may be forwarded for processing to a processor. The memory may be or may comprise at least one of: a Random Access Memory; a Read-Only Memory; a Cache Memory; a Hard Disk Drive; a Solid State Drive; a Virtual Memory.

[0126] For determining if the identity of the user corresponds to a verified identity based on the flood image, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered. The template vector may be obtained from a template image. The template image may be generated in an enrollment process.

[0127] For example, the identifying of the user may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the device. Template may be an image of an authorized user. The template features and / or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit. The user may be successfully identified in case the distance is < the predefined limit at least within tolerances. The user may be declined and / or rejected otherwise.

[0128] The analyzing of the flood image may further comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the flood image and at least one offset; an inversion of flood image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transformation; applying a Hough-transformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image.

[0129] For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the flood image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprise at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks”, CoRR, abs / 1311.2901 , 2013, or C. Szegedy et al., “Going deeper with convolutions”, CoRR, abs / 1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011 , or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

[0130] The method may comprise determining a distance of the object from the detector of the TOF system by using the TOF system. The method may comprise determining if the distance of the object from the detector of the TOF system is within a working range and allowing the user to access a resource in response to determining that the distance is within working range. The present invention allows for considering the distance information obtained by the TOF system for the authentication process. The detector of the TOF system is suitable for determining a distance and generating an image for authentication. Thus, advantageously space in devices can be saved by using the TOF system for authentication.

[0131] Distances are important for face authentication as well since the face recognition models are associated with a working range. The working range may specify a distance range between the face of a user and the detector for generating an image of the user where the face recognition model works and / or is trained on the image of the user. In an embodiment, a working range may specify at least one upper and / or at least one lower boundary for a distance of an object from the detector and / or an illumination source. The working range may be associated with an authentication process. A working range may comprise at least one value. The value may be a numerical value, in particular a positive numerical value. An indication of a working range may be received, in particular prior to determining if the distance is within or outside of a working range of an authentication process. An indication of a working range may be suitable for determining if the distance is within or outside of a working range of an authentication process. An indication of a working range maybe suitable for comparing distance with a working range.

[0132] The distance information may be determined directly in the detector of the TOF system. The determining of the distance information may comprise considering calibration data, e.g. stored in a memory such as of the detector. The detector may comprise at least one communication interface configured for providing the image of the user to a further unit for further analysis and / or the distance information to the authentication unit for checking if the distance is within the working range or outside the working range.

[0133] The term "communication interface" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be lim-ited to a special or customized meaning. The term specifically may refer, without limitation, to an item or element forming a boundary configured for transferring information. In particular, the communication interface may be configured for transferring information from a computational device, e.g. a computer, such as to send or output information, e.g. onto an-other device. Additionally or alternatively, the communication interface may be configured for transferring information onto a computational device, e.g. onto a computer, such as to receive information. The communication interface may specifically provide means for transferring or exchanging information. In particular, the communication interface may provide a data transfer connection, e.g. Bluetooth, NFC, Ethernet, inductive coupling or the like. As an example, the communication interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port and a disk drive. The communication interface may be at least one web interface.

[0134] The authentication process further may comprise determining if the object corresponds to a living organism by performing a method for determining if an object corresponds to a living organism according to the present invention such as according to one or more of the embodiments given above or given in further detail below. The authentication process may comprise allowing the user to access the resource in case the pattern image of the user is determined to correspond to a living organism and otherwise, in case the pattern image of the user is determined not to correspond to a living organism, denying the user to access the resource.

[0135] The authentication unit may be further configured for considering additional security features extracted from the pattern image. In particular, the authentication unit may be further configured for extracting the liveness data and / or considering the extracted liveness data from the pattern image.

[0136] Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the extraction of the liveness data using the pattern image, particularly by running an appropriate computer program having a respective functionality. Particularly, the authentication unit may consider the liveness data as a parameter for validating the authentication process, the authentication process may be robust against being outwitted by using a recorded image of the user.

[0137] The authentication unit may be configured for outsourcing at least one step of the authentication process, such as the identifying of the user, and / or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and / or a cloud server. The device and the remote device may be part of a computer network, particularly the internet. Thereby, the device may be used as a field device that is used by the user for generating data required in the authentication process and / or its validation. The device may transmit the generated data and / or data associated to an intermediate step of the authentication process and / or its validation to the remote device. In such a scenario, the authentication unit may be and / or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and / or its validation may further be transmitted to the device. This data may be received by the connection interface comprised by the device. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

[0138] It is emphasized that data from the device may be transmitted to a specific remote device depending on at least one circumstance, such as a date, a day, a load of the specific remote device, and so on. The specific remote device may not be selected by the field device. Rather a further device may select to which specific remote device the data may be transmitted. The authentication process and and / or the generation of validation data may involve a use of several different entities of the remote device. At least one entity may generate intermediate data and transmit the intermediate data to at least one further entity.

[0139] The authentication unit may be further configured for determining the distance information using the TOF system at a plurality of positions of the user’s face and for determining a depth map. The determined depth map may be compared to a predetermined depth map of the user, e.g. determined during an enrollment process. The authentication unit may be configured for authenticating the user in case the determined depth map matches with the predetermined depth map of the user, in particular at least within tolerances. Otherwise, the user may be declined.

[0140] The allowing the user to access the resource may comprise authorization of the user. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of non-successful authentication. Thereby, the user may become aware of the result of the authentication. The authorization unit may be configured for allowing or declining the user to perform at least one operation on the device that requires authentication based on the material data and the identifying using the flood image. The authorization unit may be configured for allowing or declining the user to access one or more functions associated with the device depending on the authentication or denial. The allowing may comprise granting permission to access the one or more functions. The authorization unit may be configured for determining if the user correspond to an authorized user, wherein allowing or declining is further based on determining if the user corresponds to an authorized user. The term “authorization” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of assigning access rights to the user, in particular a selective permission or selective restriction of access to the device and / or at least one resource of the device. The authorization unit may be configured for access control. The term “authorization unit” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a unit such as a processor configured for authorization of a user. The authorization unit may comprise at least one processor or may be designed as software or application. The authorization unit and the authentication unit may be embodied integral, e.g. by using the same processor. The authorization unit may be configured for allowing the user to access the one or more functions, e.g. on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to access the one or more functions, e.g. on the device, in case of non-successful authentication.

[0141] The device, e.g. by using a user interface such as a display of the device, may be configured for displaying a result of the authentication and / or the authorization.

[0142] In a further aspect, a system for authenticating a user is disclosed. The system comprises a. at least one time-of-flight system; b. at least one processor; and c. at least one memory configured for storing instructions that, when executed by the processor cause the system to perform the method for authenticating a user of a device according to the present invention such as according to one or more of the embodiments given above or given in further detail below.

[0143] For details, options and definitions, reference may be made to the methods described herein and the system for determining if an object corresponds to a living organism as discussed above and to the methods and devices as described below.

[0144] The system may further be configured for determining if an object corresponds to a living organism. The time-of-flight system may comprise at least one transmitter for generating a pattern of light beams for illuminating the object. The transmitter may comprise at least one array of light emitters. The time-of-flight system further may comprise at least one detector configured for generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams. The processor may be configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0145] The system may be configured for performing a method for determining if an object corresponds to a living organism according to the present invention such as according to one or more of the embodiments given above or given in further detail below.

[0146] Further disclosed and proposed herein are computer programs including computer-executable instructions for performing one or both of the methods according to the present invention in one or more of the embodiments enclosed herein when the programs are executed on a computer or computer network. Specifically, the computer program may be stored on a computer-readable data carrier and / or on a computer-readable storage medium.

[0147] As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions. The computer-readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and / or a read-only memory (ROM).

[0148] Thus, specifically, one, more than one or even all of method steps i) to iii) and / or method steps a. to b. as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.

[0149] Further disclosed and proposed herein is a computer program product having program code means, in order to perform one or both of the methods according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and / or on a computer-readable storage medium.

[0150] Further disclosed and proposed herein is a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute one or both of the methods according to one or more of the embodiments disclosed herein.

[0151] Further disclosed and proposed herein is a computer program product with program code means stored on a machine-readable carrier, in order to perform one or both of the methods according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network. As used herein, a computer program product refers to the program as a tradable product. The product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and / or on a computer-readable storage medium. Specifically, the computer program product may be distributed over a data network. Finally, disclosed and proposed herein is a modulated data signal which contains instructions readable by a computer system or computer network, for performing one or both of the methods according to one or more of the embodiments disclosed herein.

[0152] Referring to the computer-implemented aspects of the invention, one or more of the method steps or even all of the method steps of the methods according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network. Thus, generally, any of the method steps including provision and / or manipulation of data may be performed by using a computer or computer network. Generally, these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and / or certain aspects of performing the actual measurements.

[0153] Specifically, further disclosed herein are: a computer or computer network comprising at least one processor, wherein the processor is adapted to perform one or both of the methods according to one of the embodiments described in this description, a computer loadable data structure that is adapted to perform one or both of the methods according to one of the embodiments described in this description while the data structure is being executed on a computer, a computer program, wherein the computer program is adapted to perform one or both of the methods according to one of the embodiments described in this description while the program is being executed on a computer, a computer program comprising program means for performing one or both of the methods according to one of the embodiments described in this description while the computer program is being executed on a computer or on a computer network, a computer program comprising program means according to the preceding embodiment, wherein the program means are stored on a storage medium readable to a computer, a storage medium, wherein a data structure is stored on the storage medium and wherein the data structure is adapted to perform one or both of the methods according to one of the embodiments described in this description after having been loaded into a main and / or working storage of a computer or of a computer network, and a computer program product having program code means, wherein the program code means can be stored or are stored on a storage medium, for performing one or both of the methods according to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.

[0154] In a further aspect, a use of a time-of-flight system for authenticating a user and / or determining if an object corresponds to a living organism is disclosed. The time-of-flight system may be comprised by a system for authenticating a user and / or a system for determining if an object corresponds to a living organism according to the present invention such as according to one or more of the embodiments given above or given in further detail below. In a further aspect, a use of an image generated by a detector of the time-of-flight system for authenticating a user and / or determining if an object corresponds to a living organism. The detector of the time-of-flight system may be comprised by a system for authenticating a user and / or a system for determining if an object corresponds to a living organism according to the present invention such as according to one or more of the embodiments given above or given in further detail below.

[0155] As used herein, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.

[0156] Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically are used only once when introducing the respective feature or element. In most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” are not repeated, nonwithstanding the fact that the respective feature or element may be present once or more than once.

[0157] Further, as used herein, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically" or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such way with other optional or non-optional features of the invention.

[0158] Summarizing and without excluding further possible embodiments, the following embodiments may be envisaged:

[0159] Embodiment 1 . A method for determining if an object corresponds to a living organism comprising i) illuminating the object with a pattern of light beams generated by at least one transmit- ter of a time-of-flight system, wherein the transmitter comprises at least one array of light emitters; ii) generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams by using at least one detector of the time-of-flight system; iii) extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0160] Embodiment 2. The method according to the preceding embodiment, wherein extracting liveness data comprises extracting material data and / or extracting blood perfusion data and / or extracting at least one surface roughness measure.

[0161] Embodiment 3. The method according to the preceding embodiment, wherein extracting of liveness data comprises providing the pattern image to a model and / or receiving material data and / or blood perfusion data or the surface roughness measure from the model, wherein the model is a data-driven model, wherein the model is configured, in particular parametrized and / or trained based on historical pattern images and corresponding liveness data.

[0162] Embodiment 4. The method according to any one of the two preceding embodiments, wherein the extracting of material data comprises generating one or more partial images from the pattern image.

[0163] Embodiment 5. The method according to any one of the three preceding embodiments, wherein the extracting of material data comprises generating a representation associated with the pattern image of the object such as at least one tensor representing the pattern image or a dimensionality-reduced representation of the pattern image.

[0164] Embodiment 6. The method according to any one of the preceding embodiments, wherein the light emitters comprise one or more of at least one laser source, at least one light emitting diode or at least one laser diode.

[0165] Embodiment 7. The method according to any one of the preceding embodiments, wherein the transmitter comprises at least one imaging and / or collimating optic comprising at least one optical element selected from the group consisting of: at least one refractive lens; at least one meta surface lens; and at least one diffractive optical element (DOE).

[0166] Embodiment 8. The method according to any one of the preceding embodiments, wherein the detector comprises at least one pixelated imaging element comprising a plurality of optical sensors, wherein the optical sensors are arranged in a two dimensional matrix. Embodiment 9. The method according to the preceding embodiment, wherein the optical sensors are configured for measuring an intensity of light from the object under illumination by the pattern of light beams.

[0167] Embodiment 10. A system for determining if an object corresponds to a living organism, wherein the system comprises

[0168] - at least one time-of-flight system comprising at least one transmitter for generating a pattern of light beams for illuminating the object, wherein the transmitter comprises at least one array of light emitters, wherein the time-of-flight system further comprises at least one detector configured for generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams,

[0169] - at least one processor configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0170] Embodiment 11 . The system according to the preceding embodiment, wherein the system is configured for performing a method for determining if an object corresponds to a living organism according to any one of the preceding embodiments referring to a method.

[0171] Embodiment 12. A computer program comprising instructions which, when the program is executed by the system according to any one of the preceding embodiments referring to a system, cause the system to perform the method according to any one of the preceding embodiments referring to a method.

[0172] Embodiment 13. A computer-readable storage medium comprising instructions which, when the instructions are executed by the system according to any one of the preceding embodiments referring to a system, cause the system to perform the method according to any one of the preceding embodiments referring to a method.

[0173] Embodiment 14. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of the preceding embodiments referring to a method.

[0174] Embodiment 15. A computer-implemented method for authenticating a user of a device, the method comprises receiving a request for accessing at least one resource associated with the device and executing at least one authentication process, wherein the authentication process comprises a. determining a distance information of the user by using a time-of-flight system and determining if the user is within or outside of a working range of the authentication process by comparing the distance information to the working range; b. allowing the user to access the resource in case the user is determined to be within the working range and otherwise, in case the user is determined to be outside the working range, denying the user to access the resource.

[0175] Embodiment 16. The method according to the preceding embodiment, wherein the authentication process further comprises determining if the object corresponds to a living organism by performing a method for determining if an object corresponds to a living organism according to any one of the preceding embodiments referring to a method for determining if an object corresponds to a living organism, wherein the authentication process comprises allowing the user to access the resource in case the pattern image of the user is determined to correspond to a living organism and otherwise, in case the pattern image of the user is determined not to correspond to a living organism, denying the user to access the resource.

[0176] Embodiment 17. The method according to any one of the two preceding embodiments, wherein the authentication process further comprises generating at least one flood image showing the user associated while the user is being illuminated by flood light and determining if the identity of the user corresponds to a verified identity based on the flood image, wherein the authentication process comprises allowing the user to access the resource in case the identity of the user corresponds to a verified identity and otherwise, in case the identity of the user does not correspond to a verified identity, denying the user to access the resource.

[0177] Embodiment 18. The method according to the preceding embodiment, wherein, for determining if the identity of the user corresponds to a verified identity based on the flood image, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector is considered, wherein the template vector is obtained from a template image, wherein the template image is generated in an enrollment process.

[0178] Embodiment 19. The method according to any one of the four preceding embodiments, wherein the device is selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0179] Embodiment 20. A system for authenticating a user comprising at least one time-of-flight system; at least one processor; and at least one memory configured for storing instructions that, when executed by the processor cause the system to perform the method for authenticating a user of a device according to any one of the preceding embodiments referring to a method for authenticating a user of a device.

[0180] Embodiment 21 . The system according to the preceding embodiment, wherein the system is further configured for determining if an object corresponds to a living organism, wherein the time-of-flight system comprises at least one transmitter for generating a pattern of light beams for illuminating the object, wherein the transmitter comprises at least one array of light emitters, wherein the time-of-flight system further comprises at least one detector configured for generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams, wherein the processor is configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0181] Embodiment 22. The system according to the preceding embodiment, wherein the system is configured for performing a method for determining if an object corresponds to a living organism according to any one of the preceding embodiments referring to a method for determining if an object corresponds to a living organism.

[0182] Embodiment 23. A computer program comprising instructions which, when the program is executed by the system for authenticating according to any one of the preceding embodiments referring to a system for authenticating, cause the system for authenticating to perform the method for authenticating a user according to any one of the preceding embodiments referring to a method for authenticating a user.

[0183] Embodiment 24. A computer-readable storage medium comprising instructions which, when the instructions are executed by the system for authenticating according to any one of the preceding embodiments referring to a system for authenticating, cause the system for authenticating to perform the method for authenticating a user according to any one of the preceding embodiments referring to a method for authenticating a user.

[0184] Embodiment 25. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method for authenticating a user according to any one of the preceding embodiments referring to a method for authenticating a user.

[0185] Embodiment 26. Use of a time-of-flight system for authenticating a user and / or determining if an object corresponds to a living organism. Embodiment 27. Use of an image generated by a detector of a time-of-flight system for authenticating a user and / or determining if an object corresponds to a living organism.

[0186] Short description of the Figures

[0187] Further optional features and embodiments will be disclosed in more detail in the subsequent description of embodiments, preferably in conjunction with the dependent claims. Therein, the respective optional features may be realized in an isolated fashion as well as in any arbitrary feasible combination, as the skilled person will realize. The scope of the invention is not restricted by the preferred embodiments. The embodiments are schematically depicted in the Figures. Therein, identical reference numbers in these Figures refer to identical or functionally comparable elements.

[0188] In the Figures:

[0189] Figure 1 shows an exemplary setup of a system for determining if an object corresponds to a living organism and of a system for authenticating a user;

[0190] Figure 2 shows a flow chart of an embodiment of a computer-implemented method for authenticating a user of a device; and

[0191] Figure 3 shows exemplarily extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0192] Detailed description of the embodiments

[0193] Figure 1 shows an exemplary setup of a system 110 for determining if an object corresponds to a living organism and of a system 112 for authenticating a user.

[0194] Determining if the user corresponds to a living organism may comprise distinguishing between non-living organisms, e.g. between skin and non-skin material such as silicon.

[0195] The system 110 comprises at least one time-of-flight system 114 comprising at least one transmitter 116 for generating a pattern of light beams for illuminating the object, wherein the transmitter 116 comprises at least one array of light emitters 118, wherein the time-of-flight system 114 further comprises at least one detector 120 configured for generating at least one pattern image 122 (exemplarily shown in Figure 3) of the object while the object is being illuminated by the pattern of light beams, at least one processor 124 configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0196] The system 112 for authenticating a user of a device comprises the at least one time-of-flight system 114; the at least one processor 124; and at least one memory 126 configured for storing instructions that, when executed by the processor 124 cause the system 112 to perform a method for authenticating a user of a device according to the present invention such as described in Figure 2.

[0197] The device may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0198] The systems 110 and / or 112 for authenticating a user may be comprised by the device of the user, e.g. may be an element of the device of the user, or may be the user’s device itself.

[0199] The elements of the system 110 and 112 are described in the following in connection with the description of an embodiment of a computer-implemented method for authenticating a user of a device as shown Figure 2.

[0200] As shown in Figure 2, the method comprises receiving a request for accessing at least one resource associated with the device and executing at least one authentication process.

[0201] The authentication process comprises

[0202] (reference number 128) determining a distance information of the user by using a time-of-flight system and determining if the user is within or outside of a working range of the authentication process by comparing the distance information to the working range;

[0203] (reference number 130) allowing the user to access the resource in case the user is determined to be within the working range and otherwise, in case the user is determined to be outside the working range, denying the user to access the resource.

[0204] The access may comprise entering and / or using the one or more functions associated with the device. The functions may be arbitrary functions such as access to at least one element and / or at least one resource of the device or associated with the device. The functions that require authentication of the user may be pre-defined. The one or more functions associated with the device may comprise unlocking the device, and / or access to an application, preferably associated with the device and / or access to a part of an application, preferably associated with the device. For example, the function may comprise access to a content of the device, e.g. as stored in a database of the device, and / or retrievable by the device. In an embodiment, allowing the user to access a resource may include allowing the user to perform at least one operation with a device and / or system. The resource may be a device, a system, a function of a device, a function of a system and / or an entity. Additionally and / or alternatively, allowing the user to access a resource may include allowing the user to access an entity. The entity may be physical entity and / or virtual entity. The virtual entity may be a database for example. The physical entity may be an area with restricted access. The area with restricted access may be one of the following: security areas, rooms, apartments, vehicles, parts of the before mentioned examples, or the like. The device may be locked and may only be unlocked by authorized user.

[0205] The device may comprise at least one communication interface, such as a user interface, configured for receiving a request for accessing at least one resource associated with the device, in particular to perform at least one operation on the device that requires authentication. The request for accessing may comprise at least one act and / or instance of asking for access. The receiving a request may comprise a process of obtaining a request, e.g. from a data source and / or the user interface. The receiving may fully or partially take place automatically. The receiving of the request for accessing one or more functions associated with the device may be performed by using at least one communication interface. The receiving may comprise receiving at least one user input, e.g. via at least one user interface e.g. such as a display of the device, and / or a request from a remote device and / or cloud, e.g. via the communication of the device such as via the internet. For example, the request may be generated by or triggered by at least one user input, such as by inputting a security number or other unlocking action by the user, and / or may be send from a remote device and / or cloud such as via a connected account.

[0206] The authentication process may be performed using the at least one authentication unit, e.g. comprising the processor 124, configured for performing at least one authentication process of a user. The execution of the authentication process may be triggered and / or started by receiving the request.

[0207] The authentication process may comprise a plurality of steps. For example, the authentication process may comprise performing at least one face detection step. The face detection step may comprise analyzing at least one image of the user, e.g. generated by the detector 120 of the TOF system 114 or a further camera.

[0208] The authentication process further may comprise generating at least one flood image showing the user associated while the user is being illuminated by flood light and determining if the identity of the user corresponds to a verified identity based on the flood image. The authentication process may comprises allowing the user to access the resource in case the identity of the user corresponds to a verified identity and otherwise, in case the identity of the user does not correspond to a verified identity, denying the user to access the resource.

[0209] The method may comprise: illuminating the user with flood light by using at least one the flood illumination source; capturing the at least one flood image by using at least one image generation unit such as the detector 120 of the TOF system 114 or a further camera.

[0210] The flood image may comprise an image showing a user, in particular the face of the user, while the user is being illuminated with the flood light. The flood image may be generated by imaging and / or recording light reflected by an object and / or user which is illuminated by the flood light. The flood image showing the user may comprise at least a portion of the flood light on at least a portion the user. For example, the illumination by the flood illumination source and the imaging may be synchronized, e.g. by using at least one control unit.

[0211] The face detection step may comprise analyzing the flood image. For example, the authentication process may comprise performing at least one face detection using the flood image. The face detection may be performed locally on the device. Face identification, i.e. assigning an identity to the detected face, however, may be performed remotely, e.g. in the cloud, e.g. especially when identification needs to be done and not only verification. User templates can be stored at the remote device, e.g. in the cloud, and would not need to be stored locally. This can be an advantage in view of storage space and security.

[0212] The authentication process may comprise identifying the user based on the flood image. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the identification of the user based on the flood image, particularly by running an appropriate computer program having a respective functionality. The identifying may comprise assigning an identity to a detected face and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. For matching the flood image with a template, a similarity between at least one image feature vector obtained from the flood image and at least one template feature vector may be considered and / or evaluated. The template vector may be obtained from a template image.

[0213] The template image may be generated in an enrollment process. The enrollment process may comprise at least one step of registering, particularly to a service. In the enrollment process, the template image may be generated under secure conditions in a manner that it is guaranteed that the generated template image shows the user. The enrollment process may comprise at least one step of: capturing the template image; recording personal data and the like.

[0214] The face detection may comprise analyzing the flood image. In particular, the analyzing of the flood image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network. With respect to image recognition reference is made to the description above.

[0215] For example, the authentication may comprise identifying the user. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the flood image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template, e.g. a template image generated within an enrollment process. The identifying of the user may comprise determining if the imaged face is the face of the user, in particular if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device. Authentication may be successful if the flood image can be matched with an image template. Authentication may be unsuccessful if the flood image cannot be matched with an image template.

[0216] As outlined above, the method proposes considering distance information obtained and / or provided by the TOF system 114. In step a. 128, the method may comprise determining a distance of the object from the detector 120 of the TOF system 114 by using the TOF system 114. The method may comprise determining if the distance of the object from the detector 120 is within a working range and allowing the user to access a resource in response to determining that the distance is within working range. The present invention allows for considering the distance information obtained by the TOF system for the authentication process. The detector 120 may be suitable for determining a distance and generating an image for authentication. Thus, advantageously space in devices can be saved by using the TOF system 114 for authentication.

[0217] Distances are important for face authentication as well since the face recognition models are associated with a working range. The working range may specify a distance range between the face of a user and the detector 120 for generating an image of the user where the face recognition model works and / or is trained on the image of the user. In an embodiment, a working range may specify at least one upper and / or at least one lower boundary for a distance of an object from the detector and / or an illumination source. The working range may be associated with an authentication process. A working range may comprise at least one value. The value may be a numerical value, in particular a positive numerical value. An indication of a working range may be received, in particular prior to determining if the distance is within or outside of a working range of an authentication process. An indication of a working range may be suitable for determining if the distance is within or outside of a working range of an authentication process. An indication of a working range maybe suitable for comparing distance with a working range.

[0218] The distance information may be determined directly in the detector 120 of the TOF system 114. The determining of the distance information may comprise considering calibration data, e.g. stored in a memory 126 such as of the detector 120. The detector 120 may comprise at least one communication interface configured for providing the image of the user to a further unit for further analysis and / or the distance information to the authentication unit for checking if the distance is within the working range or outside the working range.

[0219] The authentication process further may comprise determining if the object corresponds to a living organism by performing a method for determining if an object corresponds to a living organism according to the present invention.

[0220] The method for determining if an object corresponds to a living organism comprises i) (reference number 132) illuminating the object with a pattern of light beams generated by the transmitter 116 of the time-of-flight system 114, wherein the transmitter 116 comprises the array of light emitters 118; ii) (reference number 134) generating the at least one pattern image 122 of the object while the object is being illuminated by the pattern of light beams by using the detector 120 of the time-of-flight system 114; iii) (reference number 136) extracting liveness data from the pattern image 122 by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

[0221] The emitters may be configured for emitting light pulses. Each of the light emitters may be configured for emitting light beams one after the other, e.g. in time intervals, e.g. controlled by at least one control unit of the TOF-system 114.

[0222] The emitter may comprise one or more of at least one laser source, at least one light emitting diode or at least one laser diode. For example, the laser source may be at least one vertical cavity surface emitting laser (VCSEL).

[0223] For example, the transmitter 116 may be or may comprise at least one Radio Frequency (RF)- modulated light source. In this case, the light emitter may be and / or may comprise at least one light emitting diode or at least one laser diode. The transmitter 116 may be configured for modulating the light generated by the light emitter with an RF carrier. The transmitter may be configured for modulating the light with an RF carrier frequency from 100 kHz to 300 MHz, preferably from 80 MHz to 200 MHz. For example, the transmitter 116 may be configured for modulating the light with high speeds up to 100 MHz.

[0224] For example, the TOF system 114 may comprise at least one direct TOF imager. In this case, the light emitter may be and / or may comprise at least one laser source, e.g. at least one infrared laser source. This can allow making the illumination unobtrusive. A single pulse per frame from 10 to 100 Hz, preferably from 15 to 80 Hz, more preferably from 20 to 60 Hz may be used. For example, a single pulse per frame, e.g. 30 Hz, may be used. The transmitter 116 may comprise at least one imaging and / or collimating optic 138 comprising at least one optical element selected from the group consisting of: at least one refractive lens; at least one meta surface lens; and at least one diffractive optical element (DOE). The optical element is configured for collimating and / or replicating light.

[0225] The light beams illuminating the object may be reflected by the object and are imaged by the detector 120 of the TOF system 114. The detector 120 may be and / or may comprise at least one image sensor. The detector 120 comprises at least one pixelated imaging element comprising a plurality of optical sensors. The optical sensors may be arranged in a two dimensional matrix. For example, the detector 120 may comprise at least one CMOS sensor or at least one CCD chip.

[0226] Each of the pixels of the detector 120 may be configured for measuring the time the light has taken to travel from the transmitter to the object and back to detector 120. For example, in case of using the RF-modulated light source, the detector 120 of the TOF system may be configured for measuring a phase shift of the carrier. For example, in case of using a direct time-of-flight measurement principle, the detector may be configured for determining the time for a single pulse to leave the transmitter and reflect back to the detector. In particular, when running in the so-called trigger mode, the detector 120 can generate a 3D image comprising spatial and temporal data. The detector 120, e.g. each of the pixels, may be configured for measuring intensity of impinging light, in particular an amplitude of the modulated light on the detector.

[0227] The pattern image 122 may comprise an image showing the object, in particular at least parts of the face of the user, while the user is being illuminated with the pattern, particularly on a respective area of interest comprised by the image. Thus, the detector 120 may yield an intensity image of the object projected with the pattern. The pattern image may be generated by imaging and / or recording light reflected by an object, which is illuminated by the light pattern.

[0228] Thus, the present invention proposes using a TOF system 114 for generating an image, in particular a 2D image, e.g. a grayscale image, to be evaluated for extracting liveness data by beam profile analysis. This image can be processed in step iii), e.g. by using beam profile analysis, thereby obtaining material information about the object.

[0229] Step iii) 136, comprises extracting liveness data from the pattern image 122 by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data. Determining if the object corresponds to a living organism based on the pattern image may comprise extracting liveness data from the pattern image. Particularly thereby, it may be determined that the object is a human. In particular, extracting liveness data comprises extracting material data and / or extracting blood perfusion data.

[0230] For example, determining if the object corresponds to a living organism based on the at least one pattern image may comprise extracting material data from the pattern image 122. Particularly thereby, it may be determined that the user is a human. The material data may comprise an item of information on the type of material of the user detected in the pattern image 122. Extracting material data from the pattern image 122 may be or may comprise generating the material type and / or data derived from the material type. The material data may comprise an item of information on the type of material of the object detected in the pattern image 122.

[0231] The extracting of material data from the pattern image may comprise beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , WO 2020 / 187719 A1 , WO 2023 / 156469 A1 , WO 2023 / 156315 A1 , the full content of which is included by reference. Determining if the object corresponds to a living organism based the at least one pattern image may comprise determining if the extracted material data corresponds a desired material data. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. In the example, skin as desired material data may be compared with nonskin material or silicon as material data and the result may be declination since silicon or nonskin material may be different from skin. Comparing the material data with desired material data may comprise determining a similarity of the extracted material data and the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon.

[0232] For extracting the material data, the complete pattern image 122 may be used. Alternatively, partial images may be used. Extracting of material data may include generating one or more partial images from the pattern image 122. For example, different regions of the pattern image may be selected as partial images. The partial images may be different from each other. In particular, the partial images may be non-overlapping. Using partial images may allow having material data from different areas of the object (e.g. having different light conditions) and / or comparing the extracted material data and / or generating of a material map and / or reducing an uncertainty on the obtained material data.

[0233] In an embodiment, extracting material data from the pattern image may comprise generating the material type and / or data derived from the material type. Preferably, extracting material data may be based on the pattern image. Material data may be extracted by using at least one model. Extracting material data may comprise providing the pattern image 122 to at least one model and / or receiving material data from the model. Extracting material data may include providing the pattern image 122 to a model and / or receiving material data from the model. With respect to extraction of material data reference is made to the description above.

[0234] Additionally or alternatively, determining if the object corresponds to a living organism based on the at least one pattern image may comprise determining at least one blood perfusion measure. Particularly thereby, it may be determined that the human is living.

[0235] Determining the at least one blood perfusion measure may comprise determining at least one speckle contrast of the pattern image. Alternatively or in addition, determining the at least one blood perfusion measure may comprise determining a blood perfusion measure based on the determined at least one speckle contrast. The term “speckle contrast " as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a degree of a variation in a speckle pattern generated by coherent light. The speckle pattern may be generated by the transmitter, particularly on the object. A speckle contrast may represent a measure for a mean contrast of an intensity distribution within an area of a speckle pattern. In particular, a speckle contrast K over an area of the speckle pattern may be expressed as a ratio of standard deviation o to the mean speckle intensity <l>, i.e.,

[0236] Speckle contrast may comprise a speckle contrast value. Speckle contrast values may be distributed between 0 and 1. The blood perfusion measure may be determined based on the speckle contrast. The blood perfusion measure may depend on the determined speckle contrast. If the speckle contrast changes, the blood perfusion measure derived from the speckle contrast may change accordingly. A blood perfusion measure may be a single number or value that may represent a likelihood that the object is a living subject. For monitoring of speckle contrast changes a plurality of pattern images generated at different points in time may be used.

[0237] For determining the speckle contrast, the complete pattern image may be used. Alternatively, for determining the speckle contrast, a section of the pattern image may be used. The section of the pattern image, preferably, represents a smaller area of the pattern image than an area of the complete pattern image.

[0238] In an embodiment, a data-driven model may be used for determining a blood perfusion measure. Data-driven model be parametrized and / or trained based on a training data set. The training data set may comprise a pattern image and a blood perfusion measure. The data-driven model may be parametrized and / or trained based on the training data set to output a blood perfusion measure based on receiving a pattern image.

[0239] In an embodiment, the determining if an object corresponds to a living organism based on the blood perfusion data may comprise determining if the blood perfusion measure corresponds to blood perfusion measure of a human being. Determining if the blood perfusion measure corresponds a human being may comprise comparing the blood perfusion measure to at least one pre-defined or pre-determined range of values of blood perfusion measure, e.g. stored in at least one database. In case the extracted blood perfusion measure is at least within tolerances within the re-defined or pre-determined range of values of blood perfusion measure, the object is determined to correspond to a living organism, otherwise not.

[0240] Figure 3 shows from left to right an example of a pattern image 122, selected and extracted light spots 140 of the pattern image for beam profile analysis for extracting liveness data and (highly schematic) determining if the object corresponds to a living organism based on the liveness data 142.

[0241] The methods described above may be performed completely automatically, in particular without user interaction.

[0242] List of reference numbers system for determining if an object corresponds to a living organism system for authenticating a user time-of-flight system transmitter array of light emitters detector pattern image processor memory step a. step b. step i) step ii) step iii) imaging and / or collimating optic selected and extracted light spots determining if the object corresponds to a living organism

Claims

Claims1 . A method for determining if an object corresponds to a living organism comprising i) (132) illuminating the object with a pattern of light beams generated by at least one transmitter (116) of a time-of-flight system (114), wherein the transmitter (116) comprises at least one array of light emitters (118); ii) (134) generating at least one pattern image (122) of the object while the object is being illuminated by the pattern of light beams by using at least one detector (120) of the time-of-flight system (114); iii) (136) extracting liveness data from the pattern image (122) by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

2. The method according to the preceding claim, wherein liveness data is indicative of whether an object corresponds to a living organism, wherein extracting liveness data comprises extracting material data and / or extracting blood perfusion data and / or extracting at least one surface roughness measure.

3. The method according to the preceding claim, wherein extracting of liveness data comprises providing the pattern image (122) to a model and / or receiving material data and / or blood perfusion data or the surface roughness measure from the model, wherein the model is a data-driven model, wherein the model is configured, in particular parametrized and / or trained based on historical pattern images and corresponding liveness data.

4. The method according to any one of the preceding claims, wherein the light emitters comprise one or more of at least one laser source, at least one light emitting diode or at least one laser diode.

5. The method according to any one of the preceding claims, wherein the detector (120) of the time-of-flight system (114) is configured for detecting impinging light for time-of- flight analysis, wherein the detector (120) is configured for measuring the time the light has taken to travel from the transmitter (116) to the object and from the object to detector (120) and for detecting light for generating the pattern image (122).

6. A system (110) for determining if an object corresponds to a living organism, wherein the system (110) comprises- at least one time-of-flight system (114) comprising at least one transmitter (116) for generating a pattern of light beams for illuminating the object, wherein the transmitter(116) comprises at least one array of light emitters (118), wherein the time-of-flight system (114) further comprises at least one detector (120) configured for generating at least one pattern image (122) of the object while the object is being illuminated by the pattern of light beams,- at least one processor (124) configured for extracting liveness data from the pattern image (122) by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

7. A computer-implemented method for authenticating a user of a device, the method comprises receiving a request for accessing at least one resource associated with the device and executing at least one authentication process, wherein the authentication process comprises a. (128) determining a distance information of the user by using a time-of-flight system (114) and determining if the user is within or outside of a working range of the authentication process by comparing the distance information to the working range; b. (130) allowing the user to access the resource in case the user is determined to be within the working range and otherwise, in case the user is determined to be outside the working range, denying the user to access the resource.

8. The method according to the preceding claim, wherein the authentication process further comprises determining if the object corresponds to a living organism by performing a method for determining if an object corresponds to a living organism according to any one of the preceding claims referring to a method for determining if an object corresponds to a living organism, wherein the authentication process comprises allowing the user to access the resource in case the pattern image (122) of the user is determined to correspond to a living organism and otherwise, in case the pattern image of the user is determined not to correspond to a living organism, denying the user to access the resource.

9. The method according to any one of the two preceding claims, wherein the device is selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or, and / or a tablet computer, and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

10. A system (112) for authenticating a user comprising at least one time-of-flight system (114); at least one processor (124); andat least one memory (126) configured for storing instructions that, when executed by the processor (124) cause the system (112) to perform the method for authenticating a user of a device according to any one of the preceding claims referring to a method for authenticating a user of a device.11 . The system (112) according to the preceding claim, wherein the system (112) is further configured for determining if an object corresponds to a living organism, wherein the time-of-flight system (114) comprises at least one transmitter (116) for generating a pattern of light beams for illuminating the object, wherein the transmitter (116) comprises at least one array of light emitters (118), wherein the time-of-flight system (116) further comprises at least one detector (120) configured for generating at least one pattern image of the object while the object is being illuminated by the pattern of light beams, wherein the processor (124) is configured for extracting liveness data from the pattern image by using beam profile analysis and determining if the object corresponds to a living organism based on the liveness data.

12. A computer program comprising instructions which, when the program is executed by the system (110) for determining if an object corresponds to a living organism according to any one of the preceding claims referring to a system for determining if an object corresponds to a living organism, cause the system (110) to perform the method for determining if an object corresponds to a living organism according to any one of the preceding claims referring to a method for determining if an object corresponds to a living organism, and / or when the program is executed by the system (112) for authenticating according to any one of the preceding claims referring to a system for authenticating, cause the system (112) for authenticating to perform the method for authenticating a user according to any one of the preceding claims referring to a method for authenticating a user.

13. A computer-readable storage medium comprising instructions which, when the instructions are executed by the system (110) for determining if an object corresponds to a living organism according to any one of the preceding claims referring to a system for determining if an object corresponds to a living organism, cause the system (110) to perform the method for determining if an object corresponds to a living organism according to any one of the preceding claims referring to a method for determining if an object corresponds to a living organism, and / or when the instructions are executed by the system (112) for authenticating according to any one of the preceding claims referring to a system for authenticating, cause the system (112) for authenticating to perform the method for authenticating a user according to any one of the preceding claims referring to a method for authenticating a user.

14. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the methodfor determining if an object corresponds to a living organism according to any one of the preceding claims referring to a method for determining if an object corresponds to a living organism, and / or when executed by one or more processors, cause the one or more processors to per- form the method for authenticating a user according to any one of the preceding claims referring to a method for authenticating a user.

15. Use of a time-of-flight system (114) for authenticating a user and / or determining if an object corresponds to a living organism.

16. Use of an image generated by a detector (120) of a time-of-flight system (114) for authenticating a user and / or determining if an object corresponds to a living organism.

Images