Rolling shutter rgb-ir sensor

Abstract

The invention relates to a method for generating an IR image by a detector (110), wherein the detector (110) comprises a first plurality of IR pixels (112, 118, 236) and at least one second plurality of IR pixels (112, 118, 238), wherein the IR pixels (112, 118) comprised by the detector (110) are arranged in a plurality of rows, wherein the first plurality of IR pixels (112, 118, 236) comprises at least one first row of IR pixels (237) of the plurality of rows, wherein the at least one second plurality of IR pixels (112, 118, 238) comprises at least one second row of IR pixels (239) of the plurality of rows, wherein a read out of the pixels (112, 118, 122) for determining the pixel values is performed sequentially by reading out the at least one first row and then reading out the at least one second row, the method comprising: - illuminating an object by IR light; - receiving detection light from the object generated by the illuminated IR light at the first plurality of I R pixels (112, 118, 236); - determining the pixel values of the first plurality of IR pixels (112, 118, 236) by reading out the at least one first row of IR pixels (237), wherein illuminating the object is stopped before reading out the at least one first row; - further illuminating the object by IR light; - receiving detection light from the object generated by the illuminated IR light at the second plurality of IR pixels (112, 118, 238); - determining the pixel values of the second plurality of IR pixels (112, 118, 238) by reading out the at least one second row of IR pixels (239), wherein further illuminating is stopped before reading out the at least one second row; - providing the IR image based on the pixel values of the first plurality of IR pixels (112, 118, 236) and the pixel values of the at least one second plurality of IR pixels (112, 118, 238).

Description

[0001] Rolling Shutter RGB-IR Sensor

[0002] Description

[0003] Field of the invention

[0004] The invention relates to a detector, a method for generating an IR image and / or a RGB image, a method for authenticating a user, a device for imaging an object, a use of a detector, a use of a method for generating an IR image and / or a RGB image, a computer program, a computer- readable storage medium and a non-transient computer-readable.

[0005] The devices, methods and uses according to the present invention specifically may be employed for example in various areas of daily life, security technology, gaming, traffic technology, production technology, photography such as digital photography or video photography for arts, documentation or technical purposes, safety technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology or in the sciences. However, other applications are also possible.

[0006] Prior art

[0007] Typically a smartphone comprises at least two cameras, particularly a first camera for performing an authentication of a user of the smartphone, and at least one second camera for generating user images, also known as selfies.

[0008] The second camera may be sensitive to ambient light in the visible range, particularly when using Red, Green, and Blue (RGB) Bayer filter arrays on Complementary Metal-Oxide- Semiconductor (CMOS) sensors. In general, the second camera may have a resolution of at least 4 megapixels (MP). Such a resolution may be required for performing image processing techniques, such as artificial intelligence based image processing techniques, specifically computational photography, for enhancing the image quality. Such an image based processing techniques may be generate the bokeh effect, which mimics a depth-of-field behavior of professional camera. Other effects may comprise color optimization, re-sharpening, contrast adaptation, dynamic range adaptations and / or noise reduction, particularly when the image is taken at a low ambient light. To avoid disturbing near infrared components, a corresponding high-pass filter is usually built into the optics of the second camera.

[0009] The first camera may work actively, such as by using a light source, typically a laser and / or an LED generating light in the near infrared range. To prevent saturation of the sensitive elements of the first camera comprising ambient light and to be able to detect the active signal sufficiently well in bright environments, an NIR bandpass filter is, usually, built into the optics of the first camera. On the other, resolutions between 0.4 and 2.3 MP may already be sufficient, since the image data are processed do not require any subjective and / or aesthetic processing.

[0010] DE 102019219945 A1 relates to a method of operating a spectrometer device comprising: connecting the energy storage device to a charging device; determining a first temperature at the spectrometer device; determining a first reference spectrum on a reference material at the first temperature; determining a second temperature on the spectrometer device during or after charging; determining a second reference spectrum with the spectrometer device on the reference material at the second temperature; determining a second reference spectrum with the spectrometer device on the reference material at the second temperature; comparing the reference spectra; and performing compensation for the determined difference in a determined spectrum.

[0011] US 2017 / 0202493 A1 relates to a device and a method for noninvasively determining the hematocrit value of a subject. The device comprises a light source for emitting light onto a skin area of the subject, said light comprising first light at a first wavelength in a first wavelength range between 500 and 1000 nm and second light at a second wavelength in a second wavelength range between 1000 and 2000 nm, a reflection detector for detecting light reflected from said skin area of the subject in response to light illumination by said light source, a transmission detector for detecting light transmitted through said skin area of the subject in response to light illumination by said light source, a processing unit for deriving plethysmography, PPG, signals for said first and second wavelengths from the light detected by said reflection detector and said transmission detector, and an analysis unit for determining the hematocrit value of the subject from said PPG signals..

[0012] EP 3275 177 A1 relates to an imaging method, an image sensor and an imaging device. The image sensor includes: a filter array, a reading circuit, a controller, a converter and an image output interface, the controller is coupled to the reading circuit, the converter and the image output interface, wherein the filter array comprises a plurality of color filter array patterns, each color filter array pattern comprises multiple-color filters and at least one infrared (I R) filter; the multiple-color filters are configured to capture visible rays; the I R filters are configured to capture IR rays; the reading circuit is configured to read out IR signals from the IR filters only or read out IR signals from the I R filters and multiple-color signals from a part of the multiple-color filters under control of the controller; the converter is configured to convert the IR signals into IR digital signals under the control of the controller; the image output interface is configured to output the IR digital signals under the control of the controller.

[0013] US 2015 / 0347812 A1 relates to an electronic device and fingerprint recognition method. The electronic device includes a housing, a lens protection cover, a processing unit, an image capturing unit, a light-source sensor and a visible light source module. The lens protection cover is installed on the housing. The light source sensor is configured to detect external light which comes into the electronic device through the lens protection cover. The processing unit determines whether the detected external light is less than a threshold. The visible light source module is configured to generate visible light. When the detected external light is less than the threshold, the processing unit activates the visible light source module to generate the visible light, and activates the image capturing unit to capture a fingerprint image of a user's finger which is put on the lens protection cover for fingerprint recognition.

[0014] US 2018 / 0129866 A1 relates to an illuminator with a metalayer for redirecting light in a desired far-field radiation pattern by using subwavelength posts.

[0015] EP 2 315 063 A2 relates to a camera having an imaging unit, a sensor having a quantum efficiency of 60% or higher in a visible light range for detecting a focal point adjustment state in the imaging unit, and a control unit for outputting a control signal for adjusting the focal point on the imaging unit on the basis of the output from the sensor.

[0016] Problem addressed by the invention

[0017] It is therefore an object of the present invention to provide a detector, a method for generating an IR image and / or a RGB image, a method for authenticating a user, a device for imaging an object, a use of a detector, a use of a method for generating an IR image and / or a RGB image, a computer program, a computer-readable storage medium and a non-transient computer- readable facing the above-mentioned technical challenges.

[0018] Specifically, it is an object of the present invention to provide devices and methods, which provide a cost-effective and compact system design that requires the use of a single detector that is capable of generating an IR image and / or an RGB image.

[0019] Summary of the invention

[0020] This problem is solved by the invention with the features of the independent patent claims. Advantageous developments of the invention, which can be realized individually or in combination, are presented in the dependent claims and / or in the following specification and detailed embodiments.

[0021] In a first aspect of the present invention, a detector may be disclosed. The detector may be designated for detecting the object light from an object, such as diffusely reflected light. The verb “to detect” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of at least one of determining, measuring and monitoring at least one parameter, qualitatively and / or quantitatively, such as at least one of a physical parameter, a chemical parameter and a biological parameter. Consequently, the term “detector” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for detecting, i.e. for at least one of determining, measuring and monitoring, at least one parameter, qualitatively and / or quantitatively, such as at least one of a physical parameter, a chemical parameter and a biological parameter. The detector may be configured for generating at least one detector signal, more specifically at least one electrical detector signal, such as an analogue and / or a digital detector signal, the detector signal providing information on the at least one parameter measured by the detector. The detector signal may be used as a “raw” detector signal and / or may be processed or preprocessed before further used, e.g. by filtering and the like. Thus, the detector may comprise at least one processing device and / or at least one preprocessing device, such as at least one of an amplifier, an analogue / digital converter, an electrical filter and a Fourier transformation.

[0022] In the present case, the detector may be configured for detecting light propagating from an object to the detector. Thus, specifically, the detector may be or may comprise at least one optical detector. The optical detector may be configured for determining at least one optical parameter, such as an intensity and / or a power of light by which at least one sensitive area of the detector is irradiated.

[0023] The term “light” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to electromagnetic radiation in one or more of the infrared, the visible and the ultraviolet spectral range. Herein, the term “ultraviolet spectral range”, generally, refers to electromagnetic radiation having a wavelength of 1 nm to 380 nm, preferably of 100 nm to 380 nm. Further, in partial accordance with standard ISO-21348 in a valid version at the date of this document, the term “visible spectral range”, generally, refers to a spectral range of 380 nm to 760 nm. The term “infrared spectral range” (IR) generally refers to electromagnetic radiation of 760 nm to 1000 pm, wherein the range of 760 nm to 1.5 pm is usually denominated as “near infrared spectral range” (NIR) while the range from 1.5 p to 15 pm is denoted as “mid infrared spectral range” (MidlR) and the range from 15 pm to 1000 pm as “far infrared spectral range” (FIR).

[0024] The detector may comprise a plurality of pixels arranged in a plurality of rows and a plurality of columns for generating an IR image and a RGB image, the plurality of pixels may comprise:

[0025] - a plurality of IR pixels suitable for generating the IR image, wherein a number of IR pixels comprised by the detector is smaller than 3,000,000, wherein more than 60% of the plurality of pixels associated with at least one row of the plurality of rows are IR pixels, optionally wherein the at least one row may be the at least one first row of IR pixels and / or the at least one second row of IR pixels ;

[0026] - a plurality of RGB pixels suitable for generating the RGB image, wherein a number of RGB pixels comprised by the detector is larger than 2,500,000, wherein more than 60% of the pixels associated with at least one further row of the plurality of rows are RGB pixels, optionally wherein the at least one further row is the at least one first row of RGB pixels and / or the at least one second row of RGB pixels.

[0027] In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein. The term “associated with” may be or may comprise “related to”. The term “suitable for” may also be referred to as “configured for”. As already disclosed in the above, the detector is comprising a plurality of pixels arranged in a plurality of rows and a plurality of columns for generating an IR image and a RGB image.

[0028] The term “pixel” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the smallest individual area element of a detector sensitive to light that is designated for, thereby, generating a signal. The detector, specifically each pixel, thus, may be configured for generating at least one detector signal, more specifically at least one electrical detector signal, providing information on at least one optical parameter, such as the power and / or intensity of light by which the detector or the pixel of the detector is illuminated. Typically, a pixel may be selected from at least one of: a photodiode, a Charge-Coupled Device (CCD), a phototransistor, or a photoresistor. Preferably, a pixel may be a Complementary Metal-Oxide-Semiconductor (CMOS).

[0029] The terms “row” and “column” are broad terms and are to be given their ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The terms specifically may refer, without limitation, to, particularly straight, lines in which the pixels are arranged. Typically, a first line associated with a row may be orthogonal to a second line associated with a column, particularly with a maximum deviation therefrom of less than 50°, 40°, 30°, 20° or 10°. A row may refer to a vertical dimension of the detector. A column may refer to a horizontal direction of a detector.

[0030] The term “image” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a visual representation of an arbitrary item. An image may be formed by a plurality of pixels generating the image. The image pixels may represent the smallest unit of the image. The image pixels may be associated with the detector pixels, such as by considering at least one detector pixel when generating at least one associated image pixel. An “IR image” may be an image that is generated by considering infrared light. An “RGB image” may be an image that is generated by considering visible light.

[0031] As stated in the above, a plurality of IR pixels is suitable for generating the IR image, wherein a number of IR pixels comprised by the detector is smaller than 3,000,000, wherein more than 60% of the plurality of pixels associated with at least one row of the plurality of rows are IR pixels. More than 70%, more than 80%, or more than 90% or 100% of the plurality of pixels associated with the at least one row of the plurality of rows may be IR pixels. Each 3rd, 4th, or 5throw of the detector may be at least one row having more than 70%, more than 80%, or more than 90% or 100% of the plurality of pixels associated with IR pixels. Typically, the number of these at least one rows may be 160, 180, 200, 220, or 240.

[0032] The term “IR pixel” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary pixel sensitive to at least one wavelength of infrared light. The IR pixels may be sensitive to infrared light having a wavelength between 780 nm and 1500 nm. The IR pixels may be insensitive to visible light. Particularly therefore, the IR pixels may be associated with at least one wavelength-selective element, particularly comprised by the detector. As used herein, the term “wavelength-selective element” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary optical element which interacts with differing spectral portions of incident light in a different manner, e.g. by having at least one wavelengthdependent optical property, such as at least one wavelength-dependent optical property selected from the list consisting of a degree of reflection, a direction of reflection, a degree of refraction, a direction of refraction, an absorption, a transmission, an index of refraction. More specifically, the detector may comprise at least one filter element disposed in a beam path of the light from the object to the IR pixel, i.e. in the beam path of the object light, wherein the filter element, specifically may be configured such that of the IR pixel is exposed to IR light from the object. As an example, the filter element may be a bandpass filter. Further elements are feasible.

[0033] As further discussed in the above, a plurality of RGB pixels is suitable for generating the RGB image, wherein a number of RGB pixels comprised by the detector is larger than 2,500,000, wherein more than 60% of the pixels associated with at least one further row of the plurality of rows are RGB pixels. More than 70%, more than 80%, or more than 90% of the pixels associated with the at least one further row of the plurality of rows may be RGB pixels. Two, three or four connected rows of the detector may be at least one further row having more than 70%, more than 80%, or more than 90% of the plurality of pixels associated with RGB pixels. Typically, the number of these at least one further rows may be 880, 900, 920, 940, or 960.

[0034] The term “RGB pixel” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary pixel sensitive to at least one wavelength of visible light. The RGB pixel may be sensitive to visible light, particularly either red, green or blue light. The RGB pixel may be sensitive to a wavelength distribution having a Full Width at Half Maximum (FWHM) ranging from 600 nm to 780 nm (red pixel) and / or a wavelength distribution having a FWHM ranging from 500 nm to 580 nm (green pixel) and / or a wavelength distribution having a FWHM ranging from 400 nm and 520 nm (blue pixel). A deviation from at least one wavelength range may be compensated for, particularly by using software programming. The RGB pixels may be insensitive to IR light. Particularly therefore, the RGB pixels may be associated with at least one wavelength-selective element, particularly comprised by the detector. More specifically, the detector may comprise at least one filter element disposed in a beam path of the light from the object to the respective RGB pixel, i.e. in the beam path of the object light, wherein the filter element, specifically may be configured such that of the RGB pixel is exposed to RGB light, specifically red, green and / or blue visible light, from the object. As an example, the filter element may be a bandpass filter. Further elements are feasible The plurality of pixels arranged in the plurality of rows and the plurality of columns may be suitable for generating an IR image and / or a RGB image with an aspect ratio of 4:3. The term “aspect ratio” is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the ratio of a width of an image to a height, of an image. The aspect ratio may be denoted by using two numbers separated by a colon, such as 4:3, four-to-three. For an x:y aspect ratio, typically, the image may have a width of x arbitrary units and a height of y arbitrary units. For arranging the plurality of pixels in the plurality of rows and the plurality of columns suitable for generating an IR image and / or a RGB image with an aspect ratio of 4:3, the plurality of pixels may be arranged within and / or form a rectangular shape. Alternatively or in addition, the plurality of pixels arranged in the plurality of rows and the plurality of columns may be suitable for generating an IR image and / or a RGB image with a different aspect ratio of 1 .85: 1 , 2.39: 1 , 16:9 and / or 3:2.

[0035] The number of RGB pixel rows comprised by the detector may be larger than the number of IR pixel rows comprised by the detector, particularly by a factor of at least 2, 3, 4, or 5. The term “RGB pixel row” may refer to a row having more than 70%, more than 80%, or more than 90% of the pixels associated with RGB pixels. The term “IR pixel row” may refer to a row having more than 70%, more than 80%, or more than 90% of the pixels associated with IR pixels. The number of RGB pixels may be suitable for generating a resolution of the RGB image larger than 4 megapixels. The number of IR pixels may be suitable for generating a resolution of the IR image between 0.4 megapixel and 2 megapixels.

[0036] In a further aspect, a method for generating an IR image and / or a RGB image by a detector, wherein the detector comprises a first plurality of IR pixels and at least one second plurality of IR pixels, and / or wherein the detector comprises a first plurality of RGB pixels and / or a or at least one second plurality of RGB pixels, is disclosed. The method comprises the following steps that may be performed in the given order. However, a different order may also be possible. In particular, one, more than one or even all of the method steps may be performed once or repeatedly. Further, the method steps may be performed successively or, alternatively, one or more of the method steps may be performed in a timely overlapping fashion or even in a parallel fashion and / or in a combined fashion. The method may further comprise additional method steps that are not listed.

[0037] The first plurality of IR pixels may comprise at least one first row of IR pixels of the plurality of rows, wherein the at least one second plurality of IR pixels may comprises at least one second row of IR pixels 239 of the plurality of rows. The first plurality of RGB pixels may comprise at least one first row of RGB pixels of the plurality of rows, wherein the at least one second plurality of RGB pixels may comprises at least one second row of RGB pixels of the plurality of rows. A read out of the pixels for determining the IR and / or RGG pixel values may performed sequentially by reading out the at least one first row and then reading out the at least one second row.

[0038] The method is comprising the following steps: - illuminating an object by IR light and / or illuminating the object by visible light;

[0039] - receiving detection light from the object generated by the illuminated IR light at the first plurality of IR pixels and / or receiving detection light from the object generated by the visible light at the first plurality of RGB pixels;

[0040] - specifically in a first read-out sequence, determining the pixel values of the first plurality of IR pixels by reading out the at least one first row of IR pixels wherein illuminating the object by IR light is stopped before reading out the at least one first row and / or determining the pixel values of the first plurality of RGB pixels by reading out the at least one first row of RGB, optionally wherein illuminating the object RGB light is stopped before reading out the at least one first row when the object is illuminated by RGB light;

[0041] - further illuminating the object by IR light and / or visible light;

[0042] - receiving detection light from the object generated by the illuminated IR light at the second plurality of IR pixels and / or receiving detection light from the object generated by the visible light at the second plurality of RGB pixels;

[0043] - specifically in a second read-out sequence, determining the pixel values of the second plurality of IR pixels by reading out the at least one second row of IR pixels 239 wherein illuminating the object by IR light is stopped before reading out the at least one second row and / or determining the pixel values of the second plurality of RGB pixels by reading out the at least one second row of RGB, optionally wherein illuminating the object RGB light is stopped before reading out the at least one second row when the object is illuminated by RGB light;

[0044] - optionally, providing the RGB image and / or the IR image based on the pixel values of the first plurality of RGB pixels and / or IR pixels and the pixel values of the second plurality of RGB pixels and / or IR pixels.

[0045] In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein.

[0046] The term “illuminate”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to the process of exposing at least one element to light. For illuminating the object, particularly by illumination light, specifically the IR light and / or the RGB light, an illumination source may be used. The image illumination source may be a pattern illumination source and / or a light-emitting diode. The term “illumination source” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for generating or providing light in the sense of the above-mentioned definition. Light propagating from the object to the detector is denoted as “detection light”. The detection light may comprise at least one of illumination light reflected by the object, illumination light scattered by the object, illumination light transmitted by the object, luminescence light generated by the object, e.g. phosphorescence or fluorescence light generated by the object after optical, electrical or acoustic excitation of the object by the illumination light or the like. Thus, the detection light may directly or indirectly be generated through the illumination of the object by the illumination light, specifically the IR light and / or the RGB light.

[0047] The term “object” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary body, chosen from a living object and a non-living object. Thus, as an example, the at least one object may comprise one or more articles and / or one or more parts of an article, wherein the at least one article or the at least one part thereof may comprise at least one component which may provide a spectrum suitable for investigations. Additionally or alternatively, the object may be or may comprise one or more living beings and / or one or more parts thereof, such as one or more body parts of a human being, e.g. a user, and / or an animal. The object specifically may comprise at least one sample which may fully or partially be analyzed by spectroscopic methods. As an example, the object may be or may comprise at least one of: human or animal skin; edibles, such as fruits; plastics and textile. Typically, the object may be a user. The term “user” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a person intended to perform the method and / or an associated device running the method.

[0048] The term “receiving” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to exposing the pixel to light, particularly in a manner that the pixel accumulates a quantity, such as a charge, as a measure of the received light. The term “determining” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to reading out the pixel, particularly for generating the at least one detector signal. Particularly thereby, the accumulated quantity of the pixel may be forwarded to an authentication unit, such as by using a signal connection.

[0049] Providing the IR image may comprise combining the first plurality of IR pixels and the at least one second plurality of IR pixels in order to generate the IR image. The first plurality of IR pixels may relates to a first portion of the IR image and the second plurality of IR pixels may relate to second portion of the IR image. The first portion of the IR image may be different from the second portion of the IR image. This may also apply to the RGB image, accordingly.

[0050] The method further may comprise generating an RGB image. The detector may further comprise RGB pixels. Particularly for generating an RGB image, the method may comprise

[0051] - determining the pixel values of the RGB pixels by reading out the RGB pixels;

[0052] - providing at least one RGB image based on the pixel values of the RGB pixels.

[0053] The RGB pixels comprised by the detector may be arranged in a plurality of rows, wherein determining the pixel values of the RGB pixels comprises - determining the pixel values of the RGB pixels of any row of the plurality of rows of the RGB pixels in the same read out-sequence (244)in which the at least one first plurality of IR pixels is read out, and

[0054] - determining the pixel values of the RGB pixels of any row of the RGB pixels in the same read out-sequence (244)in which the at least one second plurality of IR pixels is read out, and wherein providing the at least one RGB image based on the pixel values of the RGB pixels comprises:

[0055] - providing a first RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence in which the at least one first plurality of IR pixels is read out; and

[0056] - providing a second RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence in which the at least one second plurality of IR pixels is read out.

[0057] The term “read out-sequence” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process during which the detector outputs detector signal, which correspond to the detected physical quantities. A read out-sequence may comprise the reading out of at least one pixel row or a plurality of pixel rows. Before the beginning of a read out-sequence illuminating the object may be stopped. During a read out-sequence only specific rows of the detector may be read-out. A read out- sequence may comprise a simultaneous read-out of the rows of the detector, e.g. global shutter, or sequential read-out of the rows of the detector, e.g. rolling shutter. The rows of the detector related to RGB-pixels and the rows of the detector related to IR-pixels may be read out in the same manner, such as both being read-out simultaneously or sequentially, or in different manners, e.g. the rows of the detector related to RGB-pixels may be read out simultaneously and the rows of the detector related to IR-pixels may be read out sequentially; or vice versa. A read-out sequence may include the read-out of the electrons induced by the photons, such as the accumulated charge of the pixels, the conversion if the electrons or the charge into a portion of the IR-image and / or the RGB-image and, if necessary, corrections, post-processing, composition of the portion of the IR-image and / or the RGB-image.

[0058] In contrast thereto, a “read out-cycle” may comprise at least one or a plurality of read out- sequences. A read out-cycle may be completed when any pixel of the detector is read-out in a manner that the IR-image and / or the RGB-image may be generated.

[0059] Determining the pixel values of the RGB pixels may be performed in a read out-sequence before a subsequent read out-sequence in which at least one of: the pixel values of the first plurality of IR pixels; the pixel values of the second plurality of IR pixels are determined.

[0060] Illuminating the object by IR light and / or further illuminating the object by IR light comprises illuminating the object by at least one of: illuminating the object by flood light by using a flood illumination source; illuminating the object by an infrared light pattern by using a pattern illumination source. When illuminating and further illuminating the object, the object may in both cases be illuminated by flood light. Alternatively, when illuminating and further illuminating the object, the object may in both cases be illuminated by an infrared light pattern. Further alternatively, when illuminating the object, the object may be illuminated by an infrared light pattern and when further illuminating the object, the object may be illuminated by flood light; or vice versa.

[0061] The term “flood illumination source” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing substantially continuous spatial illumination. The term “flood light” as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to substantially continuous spatial illumination, in particular diffuse and / or uniform illumination. The flood light has a wavelength in the infrared range, in particular in the near infrared range. The flood illumination source may comprise at least one LED or at least one least one VCSEL, preferably a plurality of VCSELs. The term “substantially continuous spatial illumination” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to uniform spatial illumination, wherein areas of non-uniform are possible. The area, e.g. covering a user, a portion of the user and / or a face of the user, illuminated from the flood illumination source, may be contiguous. Power may be spread over a whole field of illumination. In contrast, illumination provided by the light pattern may comprise at least two contiguous areas, in particular a plurality of contiguous areas, and / or power may be concentrated in small (compared to the whole field of illumination) areas of the field of illumination. The infrared flood illumination may be suitable for illuminating a contiguous area, in particular one contiguous area. The infrared pattern illumination may be suitable for illuminating at least two contiguous areas.

[0062] A first IR image may be generated while the first plurality of IR pixels and the at least one second plurality of IR pixels is illuminated by at least one of: flood light, pattern light and a second IR image may be generated while the first plurality of IR pixels and the at least one second plurality of IR pixels is illuminated by at least one of: flood light, pattern light.

[0063] The pixels, particularly the IR pixels and / or the RGB pixels, comprised by the detector may be receiving detection light at least partially simultaneously. The term “simultaneously” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to several events occurring at least partially at the same time. For receiving detection light simultaneously, the pixel, particularly the RGB pixels and / or the IR pixels, may receive detection light at least partially at the same time, particularly by starting to receive the respective light at the same time.

[0064] Before further illuminating the object, and particularly after receiving detection light from the object, the at least one a portion of the pixels comprised by the detector may be reset, particularly the second plurality of IR pixels and / or the second plurality of RGB pixels may be reset. The term “resetting” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process of restoring the pixel to its initial state. The reset may be performed, typically, after at least the portion of the pixels is read out and / or after at least a further portion of the pixels is read out. Typically a pixel accumulates a quantity, such as a charge, when receiving detection light, particularly during an exposure. Once the exposure is complete, the accumulated quantity may be read-out and, typically, may be converted into a detector signal. The reset process may involve clearing the accumulated quantity, particularly for clearing the pixels for the next exposure. Exemplarily a pixel may be reset by applying a reset voltage and / or signal to the detector, which may reset the accumulated quantity to a baseline and / or to a zero level.

[0065] The pixels, particularly of the IR pixels and / or of the RGB pixels, comprised by the detector may be arranged in a plurality of rows, wherein the read out of the pixels for determining the pixel values may be performed sequentially, particularly wherein the read out is performed sequentially by considering one row of the plurality of rows of the pixels after the other. For considering the one row of the plurality of rows of the pixels after the other, a first row of the plurality of rows may be read out, which is followed by a read out of a second row of the plurality of rows, which is followed by a read out of a third row of the plurality of rows, and so on.

[0066] The object may be illuminated before or until a row of the plurality of rows of the pixels, particularly a row of the IR pixels and / or a row of RGB pixels, for which the read out is performed first is receiving detection light, particularly illuminated after rows of the pixels for which the read out is performed are reset. The illumination of the object may be stopped before a read out of the respective plurality of pixels for determining the pixel values, particularly a read out of the first row of the plurality of pixels, more particularly of the IR pixels and / or the RGB pixels. A read out of the respective plurality of pixels for determining the pixel values, particularly a read out of the first row of the plurality of pixels, more particularly of the IR pixels and / or the RGB pixels may be considerably short with respect to receiving detection light from the object generated by the illuminated IR light and / or the visible light by the IR pixels and / or the RGB pixels.

[0067] The IR light may be an infrared light pattern generated by using at least one pattern illumination source. The term “pattern illumination source” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for generating or providing at least one light pattern, in particular at least one infrared light pattern. The term “light pattern” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary pattern comprising a plurality of light spots. The light spot may be at least partially spatially extended. At least one spot or any spot may have an arbitrary shape. In some cases a circular shape of at least one spot or any spot may be preferred.

[0068] The infrared light may be coherent. The infrared light pattern may be a coherent infrared light pattern. The pattern illumination source may be configured for emitting light at a single wavelength, e.g. in the near infrared region. In other embodiments, the pattern illumination source may be adapted to emit light with a plurality of wavelengths, e.g. for allowing additional measurements in other wavelengths channels.

[0069] The infrared light pattern may comprise at least one regular and / or constant and / or periodic pattern such as a triangular pattern, a rectangular pattern, a hexagonal pattern or a pattern comprising further convex tilings. For example, the infrared light pattern is a hexagonal pattern, preferably a hexagonal infrared light pattern, preferably a 2 / 5 hexagonal infrared light pattern. Using a periodical 2 / 5 hexagonal pattern can allow distinguishing between artefacts and usable signal.

[0070] The infrared light pattern may comprise at least one point pattern. The infrared light pattern has a low point density. The number of infrared light spots is below or equal 4000 spots. Alternatively or in addition, the infrared light pattern may comprise equal to or less than 3000 spots, preferably equal to or less than 2000 spots. The infrared light pattern may have a low point density, in particular in comparison with other structured light techniques having typically a point density of 10k - 30k in a field of view of 55x38°. Using such a low point density may allow compensating for the above-mentioned diffraction loss. By decreasing the number of spots projected onto an object and / or a user, a contrast in the IR image and / or the RGB image may be increased. Increasing the number of points would decrease the irradiance per point. The decreased number of spots may lead to an increase in irradiance of a spot and thus, to an increase in contrast in the IR image and / or the RGB image of the projection of the infrared light pattern. The infrared light pattern may have a periodic point pattern with a reduced number of spots, wherein each of the spots has a high irradiance. Such a light pattern can ensure improved authentication using illumination sources and detector behind a display. Moreover, the low number of spots can ensure complying with eye safety requirements and stability requirements. The allowed dose may be divided between the spots of the light pattern.

[0071] The pattern illumination source may comprise at least one pattern projector configured for generating the infrared light pattern. The pattern illumination source, e.g. the pattern projector, may comprise at least one emitter, in particular a plurality of emitters. The term “emitter” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary device configured for providing at least one light beam. The light beam may generate the infrared light pattern. The emitter may comprise at least one element selected from the group consisting of at least one laser source such as at least one semi-conductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one Indium Arsenide laser, at least one Gallium Arsenide laser, at least one transistor laser, at least 50 one diode pumped laser, at least one distributed feedback lasers, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL); at least one non-laser light source such as at least one LED or at least one light bulb. For example, the pattern projector comprises at least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs may be arranged in at least one array, e.g. comprising a matrix of VCSELs. The VCSELs may be arranged on the same substrate, or on different substrates. The term “vertical-cavity surface-emitting laser” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a semiconductor laser diode configured for laser beam emission perpendicular with respect to a top surface. Examples for VCSELs can be found e.g. in en.wikipedia.org / wikiA / erticalcavity_surface-emitting_laser. VCSELs are generally known to the skilled person such as from WO 2017 / 222618 A. Each of the VCSELs is configured for generating at least one light beam. The VCSEL or the plurality of VCSELs may be configured for generating the desired spot number equal or below or equal 4000 spots, preferably, equal or below 3000 spots, more preferably equal or below 2000 spots. The plurality of generated spots may be associated with the infrared light pattern. The VCSELs may be configured for emitting light beams at a wavelength range from 800 to 1000 nm. For example, the VCSELs may be configured for emitting light beams at 808 nm, 850 nm, 940 nm, and / or 980 nm. Preferably the VCSELs emit light at 940 nm, since terrestrial sun radiation has a local minimum in irradiance at this wavelength, e.g. as described in CIE 085-1989 „Solar spectral Irradiance”.

[0072] The pattern illumination source may comprise at least one optical element configured for increasing, e.g. duplicating, the number of spots, e.g. the spots generated by the pattern projector. The pattern illumination source, particularly the optical element, may comprises at least one diffractive optical element (DOE) and / or at least one metasurface element. The DOE and / or the metasurface element may be configured for generating multiple light beams from a single incoming light beam. For example, a VCSEL projecting up to 2000 spots and an optical element comprising a plurality of metasurface elements may be used to duplicate the number of spots. Further arrangements, particularly comprising a different number of projecting VCSEL and / or at least one different optical element configured for increasing the number of spots may be possible. Other multiplication factors are possible. For example, a VCSEL or a plurality of VCSELs may be used and the generated laser spots may be duplicated by using at least one DOE.

[0073] The pattern illumination source may comprise at least one transfer device. The term “transfer device”, also denoted as “transfer system”, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to one or more optical elements which are adapted to modify the light beam, particularly the light beam used for generating at least a portion of the infrared light pattern, such as by modifying one or more of a beam parameter of the light beam, a width of the light beam or a direction of the light beam. The transfer device may comprise at least one imaging optical device .The transfer device specifically may comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focus-tunable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system; at least one holographic optical element; at least one meta optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. Thus, the transfer device may comprise a multi-lens system having refractive properties.

[0074] As discussed above, the infrared light pattern may comprise a plurality of spots. The spots may define at least one Epipolar line, wherein the RGB pixels and / or the IR pixels are arranged in a plurality of rows, wherein an angle between the Epipolar line and the rows is smaller than 50°, 45°, 40°, or 30°. In this case the detector may be read-out sequentially, particularly in a manner that one or more first pixel rows is read-out, the detector is reset, the object is illuminated again and then one or more second pixel rows is read-out. The term “Epipolar line” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a line comprising one or more possible spot center positions on the detector, particularly as a result of the extrinsic configuration of at least one of: the detector; and the illumination source. The possible spot center positions may, alternatively or in addition, depend on at least one distance between the object and the detector and / or the illumination source.

[0075] As further discussed above, the pattern illumination source may comprise a plurality of vertical cavity surface-emitting lasers (VCSELs). Different portions of the plurality of VCSELs may be associated with different arrays. The different arrays may be illuminating the object simultaneously for determining the pixel values of the first plurality of IR pixels and the second plurality of IR pixels. Alternatively the different arrays may be illuminating the object subsequently in a manner that a first array of the different arrays may be illuminating the object for determining the pixel values of the first plurality of IR pixels, particularly for generating a portion of the IR image, particularly when a second array of the different arrays may be not illuminating the object, and the second array of the different arrays may be illuminating the object for determining the pixel values of the second plurality of IR pixels, particularly for generating a further portion of the IR image, particularly when the first array of the different arrays may be not illuminating the object.

[0076] The RGB light may be generated by using at least one light-emitting diode. The term “lightemitting diode” or briefly “LED”, as used herein, is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an optoelectronic semiconductor device capable of emitting light when an electrical current flows through the device. The optoelectronic semiconductor device may be configured for generating the light due to various physical processes, including one or more of spontaneous emission, induced emission, decay of metastable excited states and the like. Thus, as an example, the light-emitting diode, may comprise one or more of: a light-emitting diode based on spontaneous emission of light, in particular an organic light emitting diode, a light-emitting diode based on superluminescence (sLED), or a laser diode (LD) In the following, without narrowing the possible embodiments of the light-emitting diode to any of the before-mentioned physical principles or setups, the abbreviation “LED” will be used for any type of light-emitting diode. Specifically, the LED may comprise at least two layers of semiconductor material, wherein light may be generated at at least one interface between the at least two layers of semiconductor material, specifically due to a recombination of positive and negative electrical charges, e.g. due to electron-hole recombination. The at least two layers of semiconductor material may have differing electrical properties, such as at least one of the layers being an n-doped semiconductor material and at least one of the layers being a p-doped semiconductor material. Thus, as an example, the LED may comprise at least one pn-junction and / or at least one pin-set up. It shall be noted, however, that other device structures are feasible, too. The at least one semiconductor material may specifically be or may comprise at least one inorganic semiconducting material. It shall be noted, however, that organic semiconducting materials may be used additionally or alternatively.

[0077] The IR pixels may be suitable for generating the IR image and / or wherein the RGB pixels may be suitable for generating the RGB image. For providing the RGB image and / or the IR image, the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels may be considered, particularly wherein for considering the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels the respective first plurality and the respective second plurality may be combined. A portion of the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels may coincide, wherein the coinciding portion may be considered for determining at least one artifact in the IR image and / or the RGB image.

[0078] The term "artifact" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an visual distortion and / or anomaly that may occur in an image. The at least one artifact may be caused by at last one limitation of the imaging process, at last one algorithmic errors, at last one compression technique, and / or at last one hardware limitation. The artifact may be selected from at least one of: a compression artifacts; an aliasing artifact; a ghosting or motion artifact; a noise artifact; a geometry or mesh artifact; a reflection or refraction artifact; a moire pattern. There may be further artifact types. The method may be computer-implemented. The term "computer implemented " as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a method involving at least one computer and / or at least one computer network. The computer and / or computer network may comprise at least one processor which is con-figured for performing at least one of the method steps of the method according to the present invention. Specifically, each of the method steps is performed by the computer and / or computer network. The method may be performed completely automatically, specifically without user interaction.

[0079] In a further aspect, a method for authenticating a user of a device comprising a detector (110) is disclosed. The method comprises the following steps that may be performed in the given order. However, a different order may also be possible. In particular, one, more than one or even all of the method steps may be performed once or repeatedly. Further, the method steps may be performed successively or, alternatively, one or more of the method steps may be performed in a timely overlapping fashion or even in a parallel fashion and / or in a combined fashion. The method may further comprise additional method steps that are not listed. In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein.

[0080] The method comprises:

[0081] - generating an IR image by performing a method for generating an IR image by the detector;

[0082] - authenticating the user based on the IR image;

[0083] - allowing the user to access the device based on the authenticating the user.

[0084] The method may further comprise

[0085] - generating an RGB image, preferably by performing the method for generating an IR image by the detector;

[0086] - further authenticating the user based on the RGB image.

[0087] In a further aspect, a method for authenticating a user of a device comprising a detector may be disclosed. The method comprises the following steps that may be performed in the given order. However, a different order may also be possible. In particular, one, more than one or even all of the method steps may be performed once or repeatedly. Further, the method steps may be performed successively or, alternatively, one or more of the method steps may be performed in a timely overlapping fashion or even in a parallel fashion and / or in a combined fashion. The method may further comprise additional method steps that are not listed.

[0088] The method may comprise the following steps:

[0089] - illuminating the user by IR light ;

[0090] - generating an IR image, particularly of the user, by the detector while the user is being illuminated by IR light;

[0091] - illuminating the user by visible light; - generating a RGB image, particularly of the user, by the detector while the user is being illuminated by visible light;

[0092] - authenticating the user based on the IR image and / or the RGB image;

[0093] - allowing the user to access the device based on the authenticating the user. The detector may be a detector as discussed elsewhere herein. The IR image and / or a RGB image may be generated by performing a method for generating an IR image and / or a RGB image. The IR light may be an infrared light pattern, particularly generated by using at least one pattern illumination source; wherein the RGB light may be generated by using at least one lightemitting diode. The method may be computer-implemented. In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein.

[0094] The term “authenticating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to verifying an identity of a user. Specifically, the authentication may comprise distinguishing between the user from other humans or objects, in particular between authorized access from non-authorized accesses. The authentication may comprise verifying identity of a respective user and / or assigning identity to a user. The authentication may comprise generating and / or providing identity information, e.g. to other devices or units such as to at least one authorization unit for authorization for providing access to the device. The identify information may be confirmed by the authentication. For example, the identity information may be and / or may comprise at least one identity token. In case of successful authentication an image of a face recorded by the detector may be verified to be an image of the user’s face and / or the identity of the user is verified. The authenticating may be performed using at least one authentication process. The authentication process may comprise a plurality of steps such as at least one face detection on the IR image and / or the RGB image and at least one identification step in which an identity is assigned to the detected face and / or at least one identity check and / or verifying an identity of the user is performed.

[0095] The authentication may be performed by using at least one authentication unit, particularly comprised by the device. The term “authentication unit” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one unit configured for performing at least one authentication process of a user. The authentication unit may be or may comprise at least one processor. The processor may be an arbitrary logic circuitry configured for performing basic operations of a computer or system, and / or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processor may be configured for processing basic instructions that drive the computer or system. As an example, the processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a math co-processor or a numeric co-processor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an L1 and L2 cache memory. In particular, the processor may be a multi-core processor. Specifically, the processor may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processor may be or may comprise a microprocessor, thus specifically the processor’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processor may be or may comprise one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs) and / or one or more tensor processing unit (TPU) and / or one or more chip, such as a dedicated machine learning optimized chip, or the like. The processor specifically may be configured, such as by software programming, for performing one or more evaluation operations. At least one or any component of a computer program configured for performing the authentication process may be executed by the processing device. Alternatively or in addition, the authentication unit may be or may comprise a connection interface. The connection interface may be configured to transfer data from the device to a remote device; or vice versa. At least one or any component of a computer program configured for performing the authentication process may be executed by the remote device.

[0096] For example, the authentication unit may perform at least one face detection using the IR image and / or the RGB image. The face detection may be performed locally on the device. Face identification, i.e. assigning an identity to the detected face, however, may be performed remotely, e.g. in the cloud, e.g. especially when identification needs to be done and not only verification. User templates can be stored at the remote device, e.g. in the cloud, and would not need to be stored locally. This can be an advantage in view of storage space and security.

[0097] The authentication unit may be configured for identifying the user based on the IR image and / or the RGB image. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the identification of the user based on the IR image and / or the RGB image, particularly by running an appropriate computer program having a respective functionality. The term “identifying” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user.

[0098] The authentication process may comprise a plurality of steps. For example, the authentication process may comprise performing at least one face detection. The face detection step may comprise analyzing the IR image and / or the RGB image. In addition, for example, the authentication process may comprise identifying. The identifying may comprise assigning an identity to a detected face and / or at least one identity check and / or verifying an identity of the user. The identifying may comprise performing a face verification of the imaged face to be the user’s face. The identifying the user may comprise matching the IR image and / or the RGB image, e.g. showing a contour of parts of the user, in particular parts of the user’s face, with a template. The identifying of the user may comprise determining if the imaged face is the face of the user, in particular if the imaged face corresponds to at least one image of the user’s face stored in at least one memory, e.g. of the device. The analyzing of the IR image and / or the RGB image may comprise one or more of the following: a filtering; a selection of at least one region of interest; a formation of a difference image between the IR image and / or the RGB image and at least one offset; an inversion of the IR image and / or the RGB image; a background correction; a decomposition into color channels; a decomposition into hue; saturation; and brightness channels; a frequency decomposition; a singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transformation; applying a Radon-transformation; applying a Houghtransformation; applying a wavelet-transformation; a thresholding; creating a binary image. The region of interest may be determined manually by a user or may be determined automatically, such as by recognizing the user within the image. In particular, the analyzing of the IR image and / or the RGB image may comprise using at least one image recognition technique, in particular a face recognition technique. An image recognition technique comprises at least one process of identifying the user in an image. The image recognition may comprise using at least one technique selected from the technique consisting of: color-based image recognition, e.g. using features such as template matching; segmentation and / or blob analysis e.g. using size, or shape; machine learning and / or deep learning e.g. using at least one convolutional neural network.

[0099] The analyzing of the IR image and / or the RGB image may comprise determining a plurality of facial features. The analyzing may comprise comparing, in particular matching, the determined facial features with template features. The template features may be features extracted from at least one template. The template may be or may comprise at least one image generated in an enrollment process, e.g. when initializing the device. Template may be an image of an authorized user. The template features and / or the facial feature may comprise a vector. Matching of the features may comprise determining a distance between the vectors. The identifying of the user may comprise comparing the distance of the vectors to a least one predefined limit, wherein the user is successfully identified in case the distance is < the predefined limit at least within tolerances. The user declining and / or rejected otherwise.

[0100] For example, the image recognition may comprise using at least one model, in particular a trained model comprising at least one face recognition model. The analyzing of the IR image and / or the RGB image may be performed by using a face recognition system, such as FaceNet, e.g. as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. The trained model may comprises at least one convolutional neural network. For example, the convolutional neural network may be designed as described in M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks”, CoRR, abs / 1311 .2901 , 2013, or C. Szegedy et aL, “Going deeper with convolutions”, CoRR, abs / 1409.4842, 2014. For more details with respect to convolutional neural network for the face recognition system reference is made to Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832. As training data labelled image data from an image database may be used. Specifically, labeled faces may be used from one or more of G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments”, Technical Report 07-49, University of Massachusetts, Amherst, October 2007, the Youtube® Faces Database as described in L. Wolf, T. Hassner, and I. Maoz, “Face recognition in unconstrained videos with matched background similarity”, in IEEE Conf, on CVPR, 2011 , or Google® Facial Expression Comparison dataset. The training of the convolutional neural network may be performed as described in Florian Schroff, Dmitry Kalenichenko, James Philbin, “FaceNet: A Unified Embedding for Face Recognition and Clustering”, arXiv: 1503.03832.

[0101] The authentication unit may be further configured for extracting material data from the IR image and / or the RGB image. Particularly therefore, the authentication unit may forward data to a remote device. Alternatively or in addition, the authentication unit may perform the material determination based on the IR image and / or the RGB image, particularly by running an appropriate computer program having a respective functionality. Particularly by considering the material as a parameter for validating the authentication process, the authentication process may be robust against being outwitted by using a recorded image of the user.

[0102] The authentication unit may be configured for extracting the material data from the IR image and / or the RGB image by beam profile analysis of the light spots. With respect to beam profile analysis reference is made to WO 2018 / 091649 A1 , WO 2018 / 091638 A1 and WO 2018 / 091640 A1 , the full content of which is included by reference. * Beam profile analysis can allow for providing a reliable classification of scenes based on a few light spots. Each of the light spots of the IR image and / or the RGB image may comprise a beam profile. As used herein, the term “beam profile” may generally refer to at least one intensity distribution of the light spot on the optical sensor as a function of the pixel. The beam profile may be selected from the group consisting of a trapezoid beam profile; a triangle beam profile; a conical beam profile and a linear combination of Gaussian beam profiles.

[0103] The authentication unit may be configured for outsourcing at least one step of the authentication process, such as the identifying of the user, and / or at least one step of the validation of the authentication process, such as the consideration of the material data, to a remote device, specifically a server and / or a cloud server. The device and the remote device may be part of a computer network, particularly the internet. Thereby, the device may be used as a field device that is used by the user for generating data required in the authentication process and / or its validation. The device may transmit the generated data and / or data associated to an intermediate step of the authentication process and / or its validation to the remote device. In such a scenario, the authentication unit may be and / or may comprise a connection interface configured for transmitting information to the remote device. Data generated by the remote device used in the authentication process and / or its validation may further be transmitted to the device. This data may be received by the connection interface comprised by the device. The connection interface may specifically be configured for transmitting or exchanging information. In particular, the connection interface may provide a data transfer connection. As an example, the connection interface may be or may comprise at least one port comprising one or more of a network or internet port, a USB-port, and a disk drive.

[0104] It is emphasized that data from the device may be transmitted to a specific remote device depending on at least one circumstance, such as a date, a day, a load of the specific remote device, and so on. The specific remote device may not be selected by the field device. Rather a further device may select to which specific remote device the data may be transmitted. The authentication process and and / or the generation of validation data may involve a use of several different entities of the remote device. At least one entity may generate intermediate data and transmit the intermediate data to at least one further entity.

[0105] Extracting material data from the IR image and / or the RGB image may comprise generating the material type and / or data derived from the material type. Preferably, extracting material data may be based on the IR image and / or the RGB image. Material data may be extracted by using at least one model. Extracting material data may include providing the IR image and / or the RGB image to a model and / or receiving material data from the model. Providing the image to a model may comprise and may be followed by receiving the IR image and / or the RGB image at an input layer of the model or via a model loss function. The model may be a data-driven model. Data- driven model may comprise a convolutional neural network and / or an encoder decoder structure such as an autoencoder. Other examples for generating a representation may be FFT, wavelets, deep learning, like CNNs, energy models, normalizing flows, GANs, vision transformers, or transformers used for natural language processing, Autoregressive Image Modeling, Normalizing Flows, Deep Autoencoders, Deep Energy-Based Models. Supervised or unsupervised schemes may be applicable to generate a representation, also embedding in e.g. cosine or Euclidian metric in ML language. The data-driven model may be parametrized according to a training data set including at least one image and material data, preferably at least one IR image and / or RGB image and material data. In another embodiment, extracting material data may include providing the image to a model and / or receiving material data from the model. In another embodiment, the data-driven model may be trained according to a training data set including at least one image and material data. In another embodiment, the data-driven model may be parametrized according to a training data set including at least one image and material data. The data-driven model may be parametrized according to a training data set to receive the image and provide material data based on the received image. The data-driven model may be trained according to a training data set to receive the image and provide material data as output based on the received image. The training data set may comprise at least one image and material data, preferably material data associated with the at least one image. The image may comprise a representation of the image. The representation may be a lower dimensional representation of the image. The representation may comprise at least a part of the data or the information associated with the image. The representation of an image may comprise a feature vector. In an embodiment, determining a representation, in particular a lower-dimensional representation may be based on principal component analysis (PCA) mapping or radial basis function (RBF) mapping. Determining a representation may also be referred to as generating a representation. Generating a representation based on PCA mapping may include clustering based on features in the IR image and / or the RGB image. Additionally or alternatively, generating a representation may be based on neural network structures suitable for reducing dimensionality. Neural network structures suitable for reducing dimensionality may comprise encoder and / or decoder. In an example, neural network structure may be an autoencoder. In an example, neural network structure may comprise a convolutional neural network (CNN). The CNN may comprise at least one convolutional layer and / or at least one pooling layer. CNNs may reduce the dimensionality of a partial image and / or an image by applying a convolution, e.g. based on a convolutional layer, and / or by pooling. Applying a convolution may be suitable for selecting feature related to material information of the IR image and / or the RGB image.

[0106] A model may be suitable for determining an output based on an input. In particular, model may be suitable for determining material data based on an image as input. A model may be a deterministic model, a data-driven model or a hybrid model. The deterministic model, preferably, reflects physical phenomena in mathematical form, e.g., including first-principles models. A deterministic model may comprise a set of equations that describe an interaction between the material and the patterned electromagnetic radiation thereby resulting in a condition measure, a vital sign measure or the like. A data-driven model may be a classification model. A hybrid model may be a classification model comprising at least one machine-learning architecture with deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of the results since those provide a systematic relation between empiricism and theory. In an embodiment, the data-driven model may be a classification model. The classification model may comprise at least one machinelearning architecture and model parameters. For example, the machine-learning architecture may be or may comprise one or more of: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifiers, support vector machines, naive Bayes classifications, nearest neighbors, neural networks, convolutional neural networks, generative adversarial networks, support vector machines, or gradient boosting algorithms or the like. In the case of a neural network, the model can be a multi-scale neural network or a recurrent neural network (RNN) such as, but not limited to, a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. The data-driven model may be parametrized according to a training data set. The data-driven model may be trained based on the training data set. Training the model may include parametrizing the model. The term training may also be denoted as learning. The term specifically may refer, without limitation, to a process of building the classification model, in particular determining and / or updating parameters of the classification model. Updating parameters of the classification model may also be referred to as retraining. Retraining may be included when referring to training herein. In an embodiment, the training data set may include at least one image and material information.

[0107] Extracting material data from the image with a data-driven model may comprise providing the image to a data-driven model. Additionally or alternatively, extracting material data from the image with a data-driven model may comprise may comprise generating an embedding associated with the image based on the data-driven model. An embedding may refer to a lower dimensional representation associated with the image such as a feature vector. Feature vector may be suitable for suppressing the background while maintaining the material signature indicating the material data. In this context, background may refer to information independent of the material signature and / or the material data. Further, background may refer to information related to biometric features such as facial features. Material data may be determined with the data-driven model based on the embedding associated with the image. Additionally or alternatively, extracting material data from the image by providing the image to a data-driven model may comprise transforming the image into material data, in particular a material feature vector indicating the material data. Hence, material data may comprise further the material feature vector and / or material feature vector may be used for determining material data.

[0108] The authentication process may be validated based on the extracted material data. In an embodiment, the validating based on the extracted material data may comprise determining if the extracted material data corresponds a desired material data. Determining if extracted material data matches the desired material data may be referred to as validating. Allowing or declining the user and / or object to perform at least one operation on the device that requires authentication based on the material data may comprise validating the authentication or authentication process. Validating may be based on material data and / or image. Determining if the extracted material data corresponds a desired material data may comprise determining a similarity of the extracted material data and the desired material data. Determining a similarity of the extracted material data and the desired material data may comprise comparing the extracted material data with the desired material data. Desired material data may refer to predetermined material data. In an example, desired material data may be skin. It may be determined if material data may correspond to the desired material data. In the example, material data may be non-skin material or silicon. Determining if material data corresponds to a desired material data may comprise comparing material data with desired material data. A comparison of material data with desired material data may result in a allowing and / or declining the user and / or object to perform at least one operation that requires authentication. In the example, skin as desired material data may be compared with non-skin material or silicon as material data and the result may be declination since silicon or non-skin material may be different from skin.

[0109] The authentication process or its validation may include generating at least one feature vector from the material data and matching the material feature vector with associate reference template vector for material.

[0110] The authentication unit may be configured for authenticating the user in case the user can be identified and / or if the material data matches the desired material data. The device may comprise at least one authorization unit configured for allowing the user to perform at least one operation on the device, e.g. unlocking the device, in case of successful authentication of the user or declining the user to perform at least one operation on the device in case of nonsuccessful authentication. Thereby, the user may become aware of the result of the authentication.

[0111] In a further aspect, a device for imaging an object is disclosed. The device comprises a detector as disclosed elsewhere herein, wherein the detector is suitable for imaging the object. In the context of this given aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein. The term “device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary system operating on light and / or electrical currents. The device may be selected from the group consisting of: a television device; a game console; a personal computer; a mobile device, particularly a cell phone, and / or a smart phone, and / or , and / or a tablet computer, , and / or a laptop, and / or a tablet, and / or a virtual reality device, and / or a wearable, such as a smart watch; or another type of portable computer.

[0112] The device may be configured for generating an IR image and / or a RGB image by using the detector and performing a method for generating an IR image and / or a RGB image. The device may comprise an authentication unit for authenticating a user of the device, wherein the authentication unit may be configured for performing a method for authenticating a user.

[0113] In a further aspect, a use of a detector as disclosed elsewhere herein is disclosed. In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein. In a further aspect, a use of a method for generating an IR image and / or a RGB image for authenticating a user is disclosed. In the context of this aspect, reference may be made to any term, definition, embodiment disclosed anywhere else herein.

[0114] In a further aspect, a computer program is disclosed, which comprises instructions which, when the program is executed by the device, cause the device to perform the method according to any one of the preceding embodiments referring to a method. Specifically, the computer program may be stored on a computer-readable data carrier and / or on a computer-readable storage medium. The computer program may be executed on at least one processor comprised by the device. The computer program may generate input data by accessing and / or controlling at least one unit of the device, such as the pattern illumination source and / or the flood illumination source and / or the detector. The computer program may generate outcome data based on the input data, particularly by using the authentication unit.

[0115] As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions. The stored computerexecutable instruction may be associate with the computer program. The computer-readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and / or a read-only memory (ROM).

[0116] Thus, specifically, one, more than one or even all of method steps a. to e. as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.

[0117] Further disclosed and proposed herein is a computer program product having program code means, in order to perform the method according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and / or on a computer-readable storage medium.

[0118] Further disclosed and proposed herein is a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute the method according to one or more of the embodiments disclosed herein.

[0119] Further disclosed and proposed herein is a computer program product with program code means stored on a machine-readable carrier, in order to perform the method according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network. As used herein, a computer program product refers to the program as a tradable product. The product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and / or on a computer-readable storage medium. Specifically, the computer program product may be distributed over a data network.

[0120] Further disclosed and proposed herein is a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to one or more of the embodiments disclosed herein.

[0121] Finally, disclosed and proposed herein is a modulated data signal which contains instructions readable by a computer system or computer network, for performing the method according to one or more of the embodiments disclosed herein.

[0122] Referring to the computer-implemented aspects of the invention, one or more of the method steps or even all of the method steps of the method according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network. Thus, generally, any of the method steps including provision and / or manipulation of data may be per-formed by using a computer or computer network. Generally, these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and / or certain aspects of performing the actual measurements.

[0123] Specifically, further disclosed herein are: a computer or computer network comprising at least one processor, wherein the processor is adapted to perform the method according to one of the embodiments described in this description, a computer loadable data structure that is adapted to perform the method according to one of the embodiments described in this description while the data structure is being executed on a computer, a computer program, wherein the computer program is adapted to perform the method according to one of the embodiments described in this description while the program is being executed on a computer, a computer program comprising program means for performing the method according to one of the embodiments described in this description while the computer program is being executed on a computer or on a computer network, a computer program comprising program means according to the preceding embodiment, wherein the program means are stored on a storage medium readable to a computer, a storage medium, wherein a data structure is stored on the storage medium and wherein the data structure is adapted to perform the method according to one of the embodiments described in this description after having been loaded into a main and / or working storage of a computer or of a computer network, and a computer program product having program code means, wherein the program code means can be stored or are stored on a storage medium, for performing the method ac-cording to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.

[0124] As used herein, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.

[0125] Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically are used only once when introducing the respective feature or element. In most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” are not repeated, nonwithstanding the fact that the respective feature or element may be present once or more than once.

[0126] Further, as used herein, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically" or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such way with other optional or non-optional features of the invention. The present disclosure exhibits several advantages, discussed in the following. The disclosed detector and methods provide a cost-effective and compact system design that allows for the use of a single detector that is capable of generating an IR image and / or an RGB image.

[0127] Further a usage of only a minimum number of IR pixels for IR images is required. Particularly therefore, the number of RGB pixels is maximized by allowing for high resolution RGB images. The row-wise separation of IR pixels and RGB pixels allows for a dedicated and / or independent read out of the respective pixels.

[0128] The presented method may have the advantage that the IR-image image may be generated in a step-wise manner by switching the IR illumination on and off. In the case of a slow detector having a high resolution that is not read-out step-wise, long exposure times of the pixel rows are to be expected, in which ambient light makes a very large contribution and the illuminated IR light makes only a very small contribution. This may result in a low contrast, particularly in the case of pattern light is used. A step-wise generation of the image may allow the contribution of ambient light to be minimized accordingly, particularly in rows that are read-out last.

[0129] This may particularly be advantageous when a rolling shutter system is used. When using a rolling shutter system each row of IR pixels is read out one after the other. The presented method may result in a minimum length of exposure time for the individual row, particularly in rows that are read-out last, such that the contribution of ambient light may be minimized.

[0130] Overall, in the context of the present invention, the following embodiments are regarded as preferred:

[0131] Embodiment 1 . A detector comprising a plurality of pixels arranged in a plurality of rows and a plurality of columns for generating an IR image and a RGB image, the plurality of pixels comprising:

[0132] - a plurality of IR pixels suitable for generating the IR image, wherein a number of IR pixels comprised by the detector is smaller than 3,000,000, wherein more than 60% of the plurality of pixels associated with at least one row of the plurality of rows are IR pixels;

[0133] - a plurality of RGB pixels suitable for generating the RGB image, wherein a number of RGB pixels comprised by the detector is larger than 2,500,000, wherein more than 60% of the pixels associated with at least one further row of the plurality of rows are RGB pixels.

[0134] Embodiment 2. The detector according to the preceding Embodiment, wherein more than 70%, more than 80%, or more than 90% of the plurality of pixels associated with the at least one row of the plurality of rows are IR pixels.

[0135] Embodiment 3. The detector according to any one of the preceding Embodiments, wherein more than 70%, more than 80%, or more than 90% of the pixels associated with the at least one further row of the plurality of rows are RGB pixels. Embodiment 4. The detector according to any one of the preceding Embodiments, wherein the plurality of pixels arranged in the plurality of rows and the plurality of columns is suitable for generating an IR image and / or a RGB image with an aspect ratio of 4:3.

[0136] Embodiment 5. The detector according to any one of the preceding Embodiments, wherein the number of RGB pixel rows comprised by the detector is larger than the number of IR pixel rows comprised by the detector, particularly by a factor of at least 2, 3, 4, or 5.

[0137] Embodiment 6. The detector according to any one of the preceding Embodiments, wherein the number of RGB pixels is suitable for generating a resolution of the RGB image larger than 4 megapixels.

[0138] Embodiment 7. The detector according to any one of the preceding Embodiments, wherein the number of IR pixels is suitable for generating a resolution of the IR image between 0.4 megapixel and 2 megapixels.

[0139] Embodiment 8. A method for generating an IR image and / or a RGB image by a detector, wherein the detector comprises a first plurality of IR pixels and at least one second plurality of IR pixels, and / or wherein the detector comprises a first plurality of RGB pixels and / or a second plurality of RGB pixels, the method comprising:

[0140] - illuminating an object by IR light and / or illuminating the object by visible light;

[0141] - receiving detection light from the object generated by the illuminated IR light at the first plurality of IR pixels and / or receiving detection light from the object generated by the visible light at the first plurality of RGB pixels;

[0142] - determining the pixel values of the first plurality of IR pixels and / or determining the pixel values of the first plurality of RGB pixels;

[0143] - further illuminating the object by IR light and / or visible light;

[0144] - receiving detection light from the object generated by the illuminated IR light at the second plurality of IR pixels and / or receiving detection light from the object generated by the visible light at the second plurality of RGB pixels;

[0145] - determining the pixel values of the second plurality of IR pixels and / or determining the pixel values of the second plurality of RGB pixels;

[0146] - optionally, providing the RGB image and / or the IR image based on the pixel values of the first plurality of RGB pixels and / or IR pixels and the pixel values of the second plurality of RGB pixels and / or IR pixels.

[0147] Embodiment 9. The method according to the preceding method Embodiment, wherein providing the IR image comprises combining the first plurality of IR pixels and the at least one second plurality of IR pixels in order to generate the IR image.

[0148] Embodiment 10. The method according to any one of the preceding method Embodiments, wherein the method further comprises generating an RGB image, wherein the detector further comprises RGB pixels, wherein the method comprises determining the pixel values of the RGB pixels by reading out the RGB pixels; providing at least one RGB image based on the pixel values of the RGB pixels.

[0149] Embodiment 11 . The method according to the preceding method Embodiment, wherein the RGB pixels comprised by the detector are arranged in a plurality of rows, wherein determining the pixel values of the RGB pixels comprises

[0150] - determining the pixel values of the RGB pixels of any row of the plurality of rows of the RGB pixels in the same read out-sequence in which the at least one first plurality of IR pixels is read out, and

[0151] - determining the pixel values of the RGB pixels of any row of the RGB pixels in the same read out-sequence in which the at least one second plurality of IR pixels is read out, and wherein providing the at least one RGB image based on the pixel values of the RGB pixels comprises:

[0152] - providing a first RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence in which the at least one first plurality of IR pixels is read out; and

[0153] - providing a second RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence in which the at least one second plurality of IR pixels is read out.

[0154] Embodiment 12. The method according to Embodiment 3, wherein determining the pixel values of the RGB pixels is performed in a read out-sequence before a subsequent read out-sequence in which at least one of: the pixel values of the first plurality of IR pixels; the pixel values of the second plurality of IR pixels are determined.

[0155] Embodiment 13. The method according to any one of the preceding method Embodiments, wherein illuminating the object by IR light and / or further illuminating the object by IR light comprises illuminating the object by at least one of: illuminating the object by flood light by using a flood illumination source; illuminating the object by an infrared light pattern by using a pattern illumination source..

[0156] Embodiment 14. The method according to the preceding method Embodiment, wherein a first IR image is generated while the first plurality of IR pixels and the at least one second plurality of IR pixels is illuminated by at least one of: flood light, pattern light; and wherein a second IR image is generated while the first plurality of IR pixels and the at least one second plurality of IR pixels is illuminated by at least one of: flood light, pattern light.

[0157] Embodiment 15. The method according to the preceding method Embodiment, wherein the pixels, particularly the IR pixels and / or the RGB pixels, comprised by the detector receive detection light at least partially simultaneously.

[0158] Embodiment 16. The method according to any one of the preceding method Embodiments, wherein, before further illuminating the object, and particularly after receiving detection light from the object, the at least one a portion of the pixels comprised by the detector are reset, particularly the second plurality of IR pixels and / or the second plurality of RGB pixels are reset.

[0159] Embodiment 17. The method according to any one of the preceding method Embodiments, wherein the pixels, particularly of the IR pixels and / or of the RGB pixels, comprised by the detector are arranged in a plurality of rows, wherein the read out of the pixels for determining the pixel values is performed sequentially, particularly wherein the read out is performed sequentially by considering one row of the plurality of rows of the pixels after the other.

[0160] Embodiment 18. The method according to any one of the preceding method Embodiments, wherein the object is illuminated before or until a row of the plurality of rows of the pixels, particularly a row of the IR pixels and / or a row of RGB pixels, for which the read out is performed first is receiving detection light, particularly illuminated after rows of the pixels for which the read out is performed are reset.

[0161] Embodiment 19. The method according to any one of the preceding method Embodiments, wherein the illumination of the object is stopped before a read out of the respective plurality of pixels for determining the pixel values, particularly a read out of the first row of the plurality of pixels, more particularly of the IR pixels and / or the RGB pixels.

[0162] Embodiment 20. The method according to any one of the preceding method Embodiments, wherein a read out of the respective plurality of pixels for determining the pixel values, particularly a read out of the first row of the plurality of pixels, more particularly of the IR pixels and / or the RGB pixels is considerably short with respect to receiving detection light from the object generated by the illuminated IR light and / or the visible light by the IR pixels and / or the RGB pixels.

[0163] Embodiment 21 . The method according to any one of the preceding method Embodiments, wherein the IR light is an infrared light pattern generated by using at least one pattern illumination source.

[0164] Embodiment 22. The method according to the preceding method Embodiment, wherein the infrared light pattern comprises a plurality of spots, wherein the spots define at least one Epipolar line, wherein the RGB pixels and / or the IR pixels are arranged in a plurality of rows, wherein an angle between the Epipolar line and the rows is smaller than 50°, 45°, 40°, or 30°.

[0165] Embodiment 23. The method according to any one of the two preceding method Embodiments, wherein the pattern illumination source comprises a plurality of vertical cavity surface-emitting lasers (VCSELs), wherein different portions of the plurality of VCSELs are associated with different arrays, - wherein the different arrays are illuminating the object simultaneously for determining the pixel values of the first plurality of IR pixels and the second plurality of IR pixels; or

[0166] - wherein the different arrays are illuminating the object subsequently in a manner that o a first array of the different arrays is illuminating the object for determining the pixel values of the first plurality of IR pixels, particularly for generating a portion of the IR image, particularly when a second array of the different arrays is not illuminating the object, and o the second array of the different arrays is illuminating the object for determining the pixel values of the second plurality of IR pixels, particularly for generating a further portion of the IR image, particularly when the first array of the different arrays is not illuminating the object.

[0167] Embodiment 24. The method according to any one of the preceding method Embodiments, wherein the RGB light is generated by using at least one light-emitting diode.

[0168] Embodiment 25. The method according to any one of the preceding method Embodiments, wherein the IR pixels are suitable for generating the IR image and / or wherein the RGB pixels are suitable for generating the RGB image.

[0169] Embodiment 26. The method according to any one of the preceding method Embodiments, wherein, for providing the RGB image and / or the IR image, the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels is considered, particularly wherein for considering the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels the respective first plurality and the respective second plurality are combined.

[0170] Embodiment 27. The method according to any one of the preceding method Embodiments, wherein a portion of the first plurality of IR pixels and / or RGB pixels and the second plurality of IR pixels and / or RGB pixels coincides, wherein the coinciding portion is considered for determining at least one artifact in the IR image and / or the RGB image.

[0171] Embodiment 28. A method for authenticating a user of a device comprising a detector, the method comprising:

[0172] - generating an IR image by performing a method for generating an IR image by the detector according to any one of the preceding method Embodiments;

[0173] - authenticating the user based on the IR image;

[0174] - allowing the user to access the device based on the authenticating the user.

[0175] Embodiment 29. A method for authenticating a user of a device comprising a detector, the method comprising:

[0176] - illuminating the user by IR light ; - generating an IR image by the detector while the user is being illuminated by IR light;

[0177] - illuminating the user by visible light;

[0178] - generating a RGB image by the detector while the user is being illuminated by visible light;

[0179] - authenticating the user based on the IR image and / or the RGB image;

[0180] - allowing the user to access the device based on the authenticating the user.

[0181] Embodiment 30. The method according to any one of the two preceding method Embodiment, wherein the detector is a detector according to any one of the preceding detector Embodiments.

[0182] Embodiment 31 . The method according to any one of the three preceding method Embodiments, wherein the IR image and / or a RGB image is generated by performing a method according to any one of the preceding Embodiments referring to a method for generating an IR image and / or a RGB image.

[0183] Embodiment 32. The method according to any one of the four the preceding method Embodiments,

[0184] - wherein the IR light is an infrared light pattern, particularly generated by using at least one pattern illumination source;

[0185] - wherein the RGB light is generated by using at least one light-emitting diode.

[0186] Embodiment 33. The method according to any one of the five preceding method Embodiments, wherein the authentication is performed by using at least one authentication unit.

[0187] Embodiment 34. A device for imaging an object, wherein the device comprises a detector according to any one of the preceding Embodiments referring to a detector, wherein the detector is suitable for imaging the object.

[0188] Embodiment 35. The device according to the preceding Embodiment, wherein the device is configured for generating an IR image and / or a RGB image by using the detector and performing a method according to any one of the preceding Embodiments referring to a method for generating an IR image and / or a RGB image.

[0189] Embodiment 36. The device according to any one of the two preceding Embodiments, wherein the device comprises an authentication unit for authenticating a user of the device, wherein the authentication unit is configured for performing a method for authenticating a user according to any one of the preceding Embodiments referring to a method for authenticating a user is performed.

[0190] Embodiment 37. A use of a detector according to any one of the Embodiments referring to a detector for generating an IR image and / or a RGB image. Embodiment 38. A use of a method for generating an IR image and / or a RGB image according to any one of the Embodiments referring to method for generating an IR image and / or a RGB image for authenticating a user.

[0191] Embodiment 39. A computer program comprising instructions which, when the program is executed by the device according to any one of the preceding Embodiments referring to a device, cause the device to perform a method according to any one of the preceding Embodiments referring to a method.

[0192] Embodiment 40. A computer-readable storage medium comprising instructions which, when the instructions are executed by a device according to any one of the preceding Embodiments referring to a device, cause the device to perform a method according to any one of the preceding Embodiments referring to a method.

[0193] Embodiment 41 . A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of the preceding Embodiments referring to a method.

[0194] Embodiment 42. A device for imaging an object, wherein the device is configured for performing a method for generating an IR image by a detector according to any one of the claims referring to a method for generating an IR image by a detector.

[0195] Embodiment 43. A use of a detector in a method according to any one of the claims referring to a method for generating an IR image by a detector.

[0196] Embodiment 44. A use of a method for generating an IR image according to any one of the claims referring to a method for generating an IR image, wherein the method is used for authenticating a user.

[0197] Brief description of the figures

[0198] Further optional details and features of the invention are evident from the description of preferred exemplary embodiments which follows in conjunction with the dependent claims. In this context, the particular features may be implemented in an isolated fashion or in combination with other features. The invention is not restricted to the exemplary embodiments. The exemplary embodiments are shown schematically in the figures. Identical reference numerals in the individual figures refer to identical elements or elements with identical function, or elements which correspond to one another with regard to their functions.

[0199] Specifically, in the figures: Figures 1a and 1b show schematics of exemplary embodiments of detectors in accordance with the present invention ;

[0200] Figure 2 shows an exemplary embodiment of a method for generating an IR image and / or a RGB image by a detector;

[0201] Figure 3 shows an exemplary read-out cycle employed for generating the RGB image and / or the IR image;

[0202] Figure 4 shows an exemplary embodiment of a method for authenticating a user of a device comprising a detector;

[0203] Figure 5 shows an exemplary embodiment of a device for imaging an object;

[0204] Figure 6 shows an exemplary use case of the method for generating an IR image and / or a RGB image by a detector;

[0205] Figures 7a and 7b shows further exemplary use cases of the method for generating an IR image and / or a RGB image by a detector; and

[0206] Figure 8 shows a further exemplary embodiment of a method for authenticating a user of a device comprising a detector.

[0207] Detailed description of the embodiments:

[0208] In Figure 1 a, an exemplary embodiment of a detector 110 in accordance with the present invention is disclosed. The detector 110 is comprising a plurality of pixels 112 arranged in a plurality of rows 114 and a plurality of columns 116 for generating an IR image and a RGB image. The plurality of pixels 112 comprises:

[0209] - a plurality of IR pixels 112, 118 suitable for generating the IR image, wherein a number of IR pixels 112, 118 comprised by the detector 110 is smaller than 3,000,000 (not all of the pixels 112 are depicted in Figure 1a for reasons of clarity), wherein more than 60%, particularly any pixel 112, of the plurality of pixels 112 associated with at least one row 114 120 of the plurality of rows 114 are IR pixels 112, 118, optionally wherein the at least one row is the at least one first row of IR pixels 237 and / or the at least one second row of IR pixels 239;

[0210] - a plurality of RGB pixels 112, 122 suitable for generating the RGB image, wherein a number of RGB pixels 112, 122 comprised by the detector 110 is larger than 2,500,000 (not all of the pixels 112 are depicted in Figure 1a for reasons of clarity), wherein more than 60%, particularly any pixel 112, of the pixels 112 associated with at least one further row 114, 124 of the plurality of rows 114 are RGB pixels 112, 122, optionally wherein the at least one further row 114, 124 is the at least one first row of RGB 232 pixels and / or the at least one second row of RGB pixels 235. Alternatively, more than 70%, more than 80%, or more than 90% of the plurality of pixels 112 associated with the at least one row 114 of the plurality of rows 114 may be I R pixels 112, 118. Further alternatively, more than 70%, more than 80%, or more than 90% of the pixels 112 associated with the at least one further row 114 of the plurality of rows 114 may be RGB pixels 112, 122.

[0211] The plurality of pixels 112 arranged in the plurality of rows 114 and the plurality of columns 116 may be suitable for generating an IR image and / or a RGB image with an aspect ratio of 4:3. The number of RGB pixels 112, 122 may be suitable for generating a resolution of the RGB image larger than 4 megapixels. The number of I R pixels 112, 118 may be suitable for generating a resolution of the IR image between 0.4 megapixel and 2 megapixels.

[0212] In the exemplary embodiment illustrated in Figure 1 a, the number of RGB pixel rows 112, 122 comprised by the detector 110 is larger than the number of IR pixel rows 112, 118 comprised by the detector 110, particularly by a factor of at least 4. Alternatively, the number of RGB pixel rows 112, 122 comprised by the detector 110 may larger than the number of IR pixel rows 112, 118 comprised by the detector 110 by a factor of at least 2, 3, or 5.

[0213] In the illustrated embodiment of Figure 1a, each 5throw 114 of the detector 110 is at least one row 114 having more than 70%, more than 80%, or more than 90% of the plurality of pixels 112 associated with IR pixels 112, 118. Particularly thereby, a Bayer scheme comprising four RGB pixels, such as two green pixels, one red and one blue pixel, in a 2x2 arrangement, particularly a quadratic 2x2 arrangement, may not be interrupted. In the illustrated embodiment of Figure 1 b, each 4throw 114 of the detector 110 is be at least one row 114 having more than 70%, more than 80%, or more than 90% of the plurality of pixels 112 associated with IR pixels 112, 118. Particularly thereby, a Bayer scheme comprising four RGB pixels, such as two green pixels, one red and one blue pixel, in a 2x2 arrangement, particularly a quadratic 2x2 arrangement, may be interrupted. Alternatively, each 3rdrow 114 of the detector 110 may be at least one row 114 having more than 70%, more than 80%, or more than 90% of the plurality of pixels 112 associated with IR pixels 112, 118.

[0214] In Figure 2, an exemplary embodiment of a method 210 for generating an IR image and / or a RGB image by a detector 110, wherein the detector 110 comprises a first plurality of IR pixels 112, 118, 236 and at least one second plurality of I R pixels 112, 118, 238, and / or wherein the detector 110 comprises a first plurality of RGB pixels 112, 122 and / or a second plurality of RGB pixels 112, 122, 234 is illustrated.

[0215] The first plurality of IR pixels 236 may comprise at least one first row of IR pixels 237 of the plurality of rows, wherein the at least one second plurality of IR pixels 238 may comprises at least one second row of IR pixels 239 of the plurality of rows. The first plurality of RGB pixels 232 may comprise at least one first row of RGB pixels 233 of the plurality of rows, wherein the at least one second plurality of RGB pixels 234 may comprises at least one second row of RGB pixels 235 of the plurality of rows. A read out of the pixels for determining the IR pixel values and / or RGB pixel values may performed sequentially by reading out the at least one first row and then reading out the at least one second row. The sequential read-out may be performed in sequential read out- sequence 244.

[0216] The method 210 is comprising the following steps:

[0217] - in a step 212, illuminating an object by IR light and / or illuminating the object by visible light;

[0218] - in a step 214, receiving detection light from the object generated by the illuminated IR light at the first plurality of IR pixels 112, 118, 236 and / or receiving detection light from the object generated by the visible light at the first plurality of RGB pixels 112, 122, particularly when step 212 is performed;

[0219] - in a step 216, determining the pixel 112 values of the first plurality of IR pixels 112, 118, 236 by reading out the at least one first row of IR pixels 237 wherein illuminating the object by IR light is stopped before reading out the at least one first row and / or determining the pixel 112 values of the first plurality of RGB pixels 112, 122, 232 by reading out the at least one first row of RGB, optionally wherein illuminating the object RGB light is stopped before reading out the at least one first row when the object is illuminated by RGB light;

[0220] - in a step 218, further illuminating the object by IR light and / or visible light;

[0221] - in a step 220, receiving detection light from the object generated by the illuminated IR light at the second plurality of IR pixels 112, 118, 238 and / or receiving detection light from the object generated by the visible light at the second plurality of RGB pixels 112, 122, 234, particularly when step 218 is performed;

[0222] - in a step 222, determining the pixel 112 values of the second plurality of IR pixels 112, 118, 238 by reading out the at least one second row of IR pixels 239 wherein illuminating the object by IR light is stopped before reading out the at least one second row and / or determining the pixel values of the second plurality of RGB pixels 112, 122, 234 by reading out the at least one second row of RGB, optionally wherein illuminating the object RGB light is stopped before reading out the at least one second row when the object is illuminated by RGB light;

[0223] - in a step 224, optionally, providing the RGB image and / or the IR image based on the pixel values of the first plurality of IR pixels 112, 118, 236 and / or RGB pixels 112,

[0224] 122 and the pixel values of the second plurality of I R pixels 112, 118, 238 and / or RGB pixels 112, 122.

[0225] The method 210 may be computer-implemented. Providing the IR image may comprise combining the first plurality of IR pixels 236 and the at least one second plurality of IR pixels 238 in order to generate the IR image. The first plurality of IR pixels 236 may relate to a first portion of the IR image and the second plurality of IR pixels 238 may relate to second portion of the IR image. The first portion of the IR image may be different from the second portion of the IR image. This may also apply to the RGB image, accordingly.

[0226] As already indicated, the method 210 further may comprise generating an RGB image. The detector may further comprise RGB pixels 112, 122. Particularly for generating an RGB image, the method may comprise - determining the pixel values of the RGB pixels 112, 122 by reading out the RGB pixels 112, 122;

[0227] - providing at least one RGB image based on the pixel values of the RGB pixels 112, 122.

[0228] In Figure 6 an exemplary use case of the method 210 is illustrated. On the horizontal axis 228, the time is depicted and, on the vertical axis 230, pixels rows are depicted. As already indicated, the RGB pixels comprised by the detector may be arranged in a plurality of rows, such as at least one first row of RGB pixels 233 and at least one second row of RGB pixels 235, wherein determining the pixel values of the RGB pixels comprises

[0229] - determining the pixel values of the RGB pixels of any row of the plurality of rows of the RGB pixels 232, 234 in the same read out-sequence 244 in which the at least one first plurality of IR pixels 236 is read out, and

[0230] - determining the pixel values of the RGB pixels of any row of the RGB pixels 232, 234 in the same read out-sequence 244 in which the at least one second plurality of IR pixels 238 is read out, and wherein providing the at least one RGB image based on the pixel values of the RGB pixels comprises:

[0231] - providing a first RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence 244 in which the at least one first plurality of I R pixels 236 is read out; and

[0232] - providing a second RGB image based on the pixel values of the RGB pixels determined in the same read out-sequence 244 in which the at least one second plurality of IR pixels 238 is read out.

[0233] As may be derived from Figure 7a, determining the pixel values of the RGB pixels 112, 122 may be performed in a read out-sequence 244 before a subsequent read out-sequence 244 in which the pixel values of the first plurality of IR pixels 236 and the pixel values of the second plurality of IR pixels 238 are determined, specifically in the same read out-sequence 244. On the horizontal axis 240, the time is depicted and, on the vertical axis 242, pixels rows are depicted. The method may be repeated such that a first RGB image is generated, a first IR image is generated and then a second RGB image is generated, a second IR image is generated, and so on.

[0234] As may be derived from Figure 7b, determining the pixel values of the RGB pixels 112, 122 may be performed in a read out-sequence 244 before a subsequent read out-sequence 244 in which at least one of: the pixel values of the first plurality of IR pixels 236; the pixel values of the second plurality of IR pixels 238 are determined, specifically in two subsequent read out-sequence 244. On the horizontal axis 241 , the time is depicted and, on the vertical axis 243, pixels rows are depicted. The method may be repeated such that a first RGB image is generated, a first IR image is generated and then a second RGB image is generated, a second IR image is generated, and so on.

[0235] Particularly in such scenario, illuminating the object by IR light, specifically in a read out-sequence 244, and / or further illuminating the object by IR light, specifically in a subsequent read out- sequence 244, comprises illuminating the object by at least one of: illuminating the object by flood light by using a flood illumination source; illuminating the object by an infrared light pattern by using a pattern illumination source. When illuminating and further illuminating the object, the object may in both cases be illuminated by flood light. Alternatively, when illuminating and further illuminating the object, the object may in both cases be illuminated by an infrared light pattern. Further alternatively, when illuminating the object, the object may be illuminated by an infrared light pattern and when further illuminating the object, the object may be illuminated by flood light; or vice versa.

[0236] A first IR image may be generated while the first plurality of IR pixels 236 and the at least one second plurality of IR pixels 238 is illuminated by at least one of: flood light, pattern light and a second IR image may be generated while the first plurality of IR pixels 236 and the at least one second plurality of IR pixels 238 is illuminated by at least one of: flood light, pattern light.

[0237] The pixels 112, particularly the IR pixels 112, 118 and / or the RGB pixels 112, 122, comprised by the detector 110 may receive detection light, particularly in the step 214 and / or step 220, at least partially simultaneously. In an optional step 226, before further illuminating the object in step 218, and particularly after receiving detection light from the object, such as in step 214, the at least one a portion of the pixels 112 comprised by the 110 detector may be reset, particularly the second plurality of IR pixels 112, 118, 238 and / or the second plurality of RGB pixels 112, 122, 234 may be reset.

[0238] An exemplary read-out cycle 310 employed for generating the RGB image and / or the IR image is depicted in Figure 3. On a vertical axis 312 the plurality of rows 114 of the detector 110 are shown. On a horizontal axis 314 the time is shown shown. The illumination of the object, particularly during the step 212 and / or step 218, is illustrated in Figure 3 by the rectangles 212, 218 indicated by the dotted lines. The time of exposure of the respective rows to the before the read-out is indicated by the rectangles 326 indicated by the dash-dotted lines.

[0239] In a first time frame 316, detection light from the object may be received, such as by performing step 214, particularly when step 212 is being performed. In a second time frame 318, the pixel values of the first plurality of IR pixels 112, 118, 236 may be determined and / or the pixel values of the first plurality of RGB pixels 112, 122, 232 may be determined, such as by performing step 216. In a third time frame 320, the second plurality of IR pixels 112, 118, 238 and / or the second plurality of RGB pixels 112, 122, 234 may be reset, such as by performing step 226. In a fourth time frame 322, further detection light from the object may be received, such as by performing step 220, particularly when step 218 is being performed. In a fifth time frame 324, the pixel values of the second plurality of IR pixels 112, 118, 238 may be determined and / or the pixel values of the second plurality of RGB pixels 112, 122, 234 may be determined, such as by performing step 222.

[0240] As may be derived from Figure 3, the pixels 112, particularly of the RGB pixels 112, 122 and / or of the IR pixels 112, 118, comprised by the detector 110 may be arranged in a plurality of rows 114, wherein the read out of the pixels 112, particularly in step 216 and / or step 222, for determining the pixel values may be performed sequentially, particularly wherein the read out is performed sequentially by considering one row 114 of the plurality of rows 114 of the pixels 112 after the other.

[0241] The object may be illuminated, particularly in step 212 and / or step 218, before or until a row 114 of the plurality of rows 114 of the pixels 112, particularly a row 114 of the I R pixels 112, 118 and / or a row 114 of RGB pixels 112, 122, for which the read out is performed first, is receiving detection light, particularly illuminated after rows 114 of the pixels 112 for which the read out is performed are reset.

[0242] The illumination of the object, particularly in step 212 and / or step 218, may be stopped before a read out, particularly in step 216 and / or step 222, of the respective plurality of pixels 112 for determining the pixel values, particularly a read out of the first row 114 of the plurality of pixels 112, more particularly of the IR pixels 112, 118 and / or the RGB pixels 112, 122.

[0243] A read out of the respective plurality of pixels 112 for determining the pixel values, particularly a read out of the first row 114 of the plurality of pixels 112, more particularly of the IR pixels 112, 118 and / or the RGB pixels 112, 122 may be considerably short with respect to receiving detection light from the object generated by the illuminated IR light and / or the visible light by the IR pixels 112, 118 and / or the RGB pixels 112, 122.

[0244] The IR pixels 112, 118 may be suitable for generating the IR image and / or wherein the RGB pixels 112, 122 may be suitable for generating the RGB image. For providing the RGB image and / or the IR image, the first plurality of IR pixels 112, 118, 236 and / or RGB pixels 112, 122 and the second plurality of IR pixels 112, 118, 238 and / or RGB pixels 112, 122 may be considered, particularly wherein for considering the first plurality of IR pixels 112, 118, 236 and / or RGB pixels 112, 122 and the second plurality of IR pixels 112, 118, 238 and / or RGB pixels 112, 122 the respective first plurality and the respective second plurality may be combined. A portion of the first plurality of IR pixels 112, 118, 236 and / or RGB pixels 112, 122 and the second plurality of IR pixels 112, 118, 238 and / or RGB pixels 112, 122 may coincide, wherein the coinciding portion may be considered for determining at least one artifact in the IR image and / or the RGB image.

[0245] In Figure 8, an exemplary embodiment of a method 810 for authenticating a user of a device 510 comprising a detector 110 is disclosed. The method comprises the following steps:

[0246] - in a step 812, generating an IR image by performing a method for generating an IR image by the detector 110 as elsewhere described herein;

[0247] - in a step 814, authenticating the user based on the IR image;

[0248] - in a step 816, allowing the user to access the device 510 based on the authenticating the user.

[0249] In Figure 4, an exemplary embodiment of a method 410 for authenticating a user of a device 510 comprising a detector 110 is disclosed. The method is comprising the following steps:

[0250] - in a step 412, illuminating the user by IR light ; in a step 414, generating an IR image by the detector 110 while the user is being illuminated by IR light, particularly in step 412; in a step 416, illuminating the user by visible light; in a step 418, generating a RGB image by the detector 110 while the user is being illuminated by visible light, particularly in a step 416; in a step 420, authenticating the user based on the IR image and / or the RGB image; in a step 422, allowing the user to access the device 510 based on the authenticating the user.

[0251] The method 410 may be computer-implemented.

[0252] In Figure 5, an exemplary embodiment of a device 510 for imaging an object is disclosed. The device 510 comprises a detector 110 as disclosed elsewhere herein, wherein the detector 110 is suitable for imaging the object. The device 510 may be configured for generating an IR image and / or a RGB image by using the detector 110 and performing a method 210 for generating an IR image and / or a RGB image. The device 510 may be configured for performing a method 410 for authenticating a user. The authentication of a user of the device 510 may be performed by using at least one authentication unit 512, particularly by using at least one authentication unit 512 configured for performing the method 410 for authenticating a user. Particularly therefore, the authentication unit 512 may receive and / or consider at least one detector signal of the detector 110, particularly received from a signal connection 514 between the authentication unit 512 and the detector 110, such as an electric wire.

[0253] The IR light may be an infrared light pattern generated by using at least one pattern illumination source 516, particularly as comprised by the device 510. The RGB light may be generated by using at least one light-emitting diode 518, particularly as comprised by the device 510.

[0254] The infrared light pattern may comprise a plurality of spots 520, wherein the spots define at least one Epipolar line 522, wherein the RGB pixels 112, 122 and / or the IR pixels 112, 118 are arranged in a plurality of rows 114, wherein an angle 524 between the Epipolar line 522 and the rows 114 is smaller than 50°, 45°, 40°, or 30°, which is illustrated in Figure 5 in a highly schematic manner.

[0255] The pattern illumination source 516 may comprise a plurality of vertical cavity surface-emitting lasers (VCSELs), wherein different portions of the plurality of VCSELs may be associated with different arrays,

[0256] - wherein the different arrays may be illuminating the object simultaneously for determining the pixel values of the first plurality of I R pixels 112, 118, 236 and the second plurality of I R pixels 112, 118, 238; or

[0257] - wherein the different arrays may be illuminating the object subsequently in a manner that o a first array of the different arrays may be illuminating the object for determining the pixel values of the first plurality of I R pixels 112, 118, 236 particularly for generating a portion of the IR image, particularly when a second array of the different arrays may be not illuminating the object, and the second array of the different arrays may be illuminating the object for determining the pixel values of the second plurality of I R pixels 112, 118, 238, particularly for generating a further portion of the IR image, particularly when the first array of the different arrays may be not illuminating the object.

[0258] List of reference numbers

[0259] 110 detector

[0260] 112 pixel

[0261] 114 row

[0262] 116 column

[0263] 118 IR pixel

[0264] 120 row comprising a majority of IR pixels

[0265] 122 RGB pixel

[0266] 124 further row comprising a majority of RGB pixels

[0267] 210 method for generating an IR image and / or a RGB image

[0268] 212 illuminating an object

[0269] 214 receiving detection light

[0270] 216 determining pixel values

[0271] 218 further illuminating an object

[0272] 220 receiving detection light

[0273] 222 determining pixel values

[0274] 224 providing the RGB image and / or the IR image

[0275] 226 resetting pixel

[0276] 228 horizontal axis

[0277] 230 vertical axis

[0278] 232 first plurality of RGB pixels

[0279] 233 at least one first row of RGB pixels

[0280] 234 second plurality of RGB pixels

[0281] 235 at least one second row of RGB pixels

[0282] 236 first plurality of I R pixels

[0283] 237 at least one first row of I R pixels

[0284] 238 second plurality of IR pixels

[0285] 239 at least one second row of IR pixel

[0286] 240 horizontal axis

[0287] 241 horizontal axis

[0288] 242 vertical axis

[0289] 243 vertical axis

[0290] 244 read out-sequence

[0291] 310 read-out cycle

[0292] 312 vertical axis

[0293] 314 horizontal axis

[0294] 316 first time frame

[0295] 318 second time frame

[0296] 320 third time frame

[0297] 322 fourth time frame

[0298] 324 fifth time frame

[0299] 410 method for authenticating a user

[0300] 412 illuminating the user by IR light generating an IR image illuminating the user by visible light generating a RGB image authenticating the user allowing the user to access the device device authentication unit signal connection pattern illumination source light-emitting diode spots

[0301] Epipolar line angle

Claims

Claims1 . A method for generating an IR image by a detector (110), wherein the detector (110) comprises a first plurality of IR pixels (112, 118, 236) and at least one second plurality of I R pixels (112, 118, 238), wherein the I R pixels (112, 118) comprised by the detector (110) are arranged in a plurality of rows, wherein the first plurality of IR pixels (112, 118, 236) comprises at least one first row of IR pixels (237) of the plurality of rows, wherein the at least one second plurality of IR pixels (112, 118) comprises at least one second row of IR pixels (239) of the plurality of rows, wherein a read out of the pixels (112, 118, 122) for determining the pixel values is performed sequentially by reading out the at least one first row and then reading out the at least one second row, the method comprising:- illuminating an object by IR light;- receiving detection light from the object generated by the illuminated IR light at the first plurality of I R pixels (112, 118, 236);- determining the pixel values of the first plurality of IR pixels (112, 118, 236) by reading out the at least one first row of IR pixels (237), wherein illuminating the object is stopped before reading out the at least one first row of IR pixels (237);- further illuminating the object by IR light;- receiving detection light from the object generated by the illuminated IR light at the second plurality of IR pixels (112, 118, 238);- determining the pixel values of the second plurality of IR pixels (112, 118, 238) by reading out the at least one second row of IR pixels (239), wherein further illuminating is stopped before reading out the at least one second row of IR pixels (239);- providing the IR image based on the pixel values of the first plurality of IR pixels (112, 118, 236) and the pixel values of the at least one second plurality of IR pixels (112, 118, 238).

2. The method according to the preceding method claim, wherein providing the IR image comprises combining the first plurality of IR pixels (112, 118, 236) and the at least one second plurality of IR pixels (112, 118) in order to generate the IR image.

3. The method according to any one of the preceding method claims, wherein the method further comprises generating an RGB image, wherein the detector (110) further comprises RGB pixels (112, 122), wherein the method comprises- determining the pixel values of the RGB pixels (112, 122) by reading out the RGB pixels (112, 122);- providing at least one RGB image based on the pixel values of the RGB pixels (112, 122).

4. The method according to the preceding method claim, wherein the RGB pixels (112, 122) comprised by the detector (110) are arranged in a plurality of rows, wherein determining the pixel values of the RGB pixels (112, 122) comprises- determining the pixel values of the RGB pixels (112, 122) of any row of the plurality of rows of the RGB pixels (112, 122) in the same read out-sequence (244) in which the at least one first plurality of IR pixels (112, 118, 236) is read out, and- determining the pixel values of the RGB pixels (112, 122) of any row of the RGB pixels (112, 122) in the same read out-sequence (244) in which the at least one second plurality of I R pixels (112, 118, 238) is read out, and wherein providing the at least one RGB image based on the pixel values of the RGB pixels (112, 122) comprises:- providing a first RGB image based on the pixel values of the RGB pixels (112, 122) determined in the same read out-sequence (244) in which the at least one first plurality of IR pixels (112, 118, 236) is read out; and- providing a second RGB image based on the pixel values of the RGB pixels (112, 122) determined in the same read out-sequence (244)in which the at least one second plurality of I R pixels (112, 118, 238) is read out.

5. The method according to claim 3, wherein determining the pixel values of the RGB pixels (112, 122) is performed in a read out-sequence (244) before a subsequent read out-sequence (244) in which at least one of: the pixel values of the first plurality of IR pixels (112, 118, 236); the pixel values of the second plurality of I R pixels (112, 118, 238) are determined.

6. The method according to any one of the preceding method claims, wherein illuminating the object by IR light and / or further illuminating the object by IR light comprises illuminating the object by at least one of: illuminating the object by flood light by using a flood illumination source; illuminating the object by an infrared light pattern by using a pattern illumination source..

7. The method according to the preceding method claim, wherein a first IR image is generated while the first plurality of IR pixels (112, 118, 236) and the at least one second plurality of IR pixels (112, 118, 238) is illuminated by at least one of: flood light, pattern light; and wherein a second IR image is generated while the first plurality of IR pixels (112, 118, 236) and the at least one second plurality of I R pixels (112, 118, 238) is illuminated by at least one of: flood light, pattern light.

8. The method according to any one of the preceding method claims,- wherein the object is illuminated before or until a row of the plurality of rows (114) of the pixels (112) is read-out for determining the pixel values of the first plurality of IR pixels (112, 118, 236), and- wherein before further illuminating the object the second plurality of IR pixels (112, 118, 238) are reset.

9. The method according to any one of the preceding method claims, wherein the IR light is an infrared light pattern generated by using at least one pattern illumination source (516), wherein the infrared light pattern comprises a plurality of spots (520), wherein thespots (520) define at least one Epipolar line (522), wherein the IR pixels (112, 118) are arranged in a plurality of rows (114), wherein an angle (524) between the Epipolar line (522) and the rows (114) is smaller than 50°, 45°, 40°, or 30°.

10. The method according to any one of the preceding method claims, wherein the IR light is an infrared light pattern generated by using at least one pattern illumination source (516), wherein the pattern illumination source (516) comprises a plurality of vertical cavity surface-emitting lasers, VCSELs, wherein different portions of the plurality of VCSELs are associated with different arrays,- wherein the different arrays are illuminating the object simultaneously for determining the pixel values of the first plurality of I R pixels (112, 118, 236) and the second plurality of IR pixels (112, 118, 238); or- wherein the different arrays are illuminating the object subsequently in a manner that o a first array of the different arrays is illuminating the object for determining the pixel values of the first plurality of IR pixels (112, 118, 236), and o the second array of the different arrays is illuminating the object for determining the pixel values of the second plurality of IR pixels (112, 118, 238).11 . The method according to any one of the nine preceding method claims, wherein the detector (110) comprises a plurality of pixels (112) arranged in a plurality of rows (114) and a plurality of columns (116) for generating the IR image and the RGB image, the plurality of pixels (112) comprising:- a plurality of IR pixels (112, 118) configured for generating the IR image, wherein a number of IR pixels (112, 118) comprised by the detector (110) is smaller than 3,000,000, wherein more than 60% of the plurality of pixels (112) associated with at least one row (120) of the plurality of rows (114) are IR pixels (112, 118), optionally wherein said at least one row (120) is the at least one first row of IR pixels (237) and / or the at least one second row of IR pixels (239);- a plurality of RGB pixels (112, 122) configured for generating the RGB image, wherein a number of RGB pixels (112, 122) comprised by the detector (110) is larger than 2,500,000, wherein more than 60% of the pixels (112) associated with at least one further row (124) of the plurality of rows (114) are RGB pixels (112, 122).

12. The method according to the preceding method claim, wherein more than 70%, more than 80%, more than 90% of the plurality of pixels (112) associated with the at least one row (120) of the plurality of rows (114) are IR pixels (112, 118).

13. The method according to any one of the two preceding method claims, wherein more than 70%, more than 80%, more than 90% of the pixels (112) associated with the at least one further row (124) of the plurality of rows (114) are RGB pixels (112, 122).

14. A method for authenticating a user of a device (510) comprising a detector (110), the method comprising:- generating an IR image by performing a method for generating an IR image by the detector (110) according to any one of the preceding method claims;- authenticating the user based on the IR image;- allowing the user to access the device (510) based on the authenticating the user.

15. A device (510) for imaging an object, wherein the device (510) is configured for performing a method for generating an IR image by a detector (110) according to any one of the claims referring to a method for generating an IR image by a detector (110).

16. A use of a detector (110) in a method according to any one of the claims referring to a method for generating an IR image by a detector (110).

17. A use of a method for generating an IR image according to any one of the claims referring to a method for generating an IR image, wherein the method is used for authenticating a user.

18. A computer program comprising instructions which, when the program is executed by the device (510) according to any one of the preceding claims referring to a device (510), cause the device (510) to perform a method according to any one of the preceding claims referring to a method.

19. A computer-readable storage medium comprising instructions which, when the instructions are executed by a device (510) according to any one of the preceding claims referring to a device (510), cause the device (510) to perform a method according to any one of the preceding claims referring to a method.

20. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of the preceding claims referring to a method.

Images