OLED projector

Abstract

A method for authenticating the user of device (110) is proposed. This method is: a. A projector (112) projects multiple light beams onto the user through a display (114), the multiple light beams including a first light beam and a second light beam, and the display (114) directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, step (130) b. A step (132) of generating a pattern image showing the projection of multiple light beams onto the user, c. A step (134) of extracting biological data from the pattern image, d. A step (136) that allows the user to perform an operation on a device (110) that requires authentication based on the biometric data, Includes.

Description

[Technical Field]

[0001] The present invention relates to a method for authenticating a device user and a device for authenticating a user. The present invention further relates to a computer program, a computer-readable storage medium, and a non-temporary computer-readable medium. The devices, methods, and uses according to the present invention can be specifically employed in various fields, such as daily life, security technology, gaming, transportation technology, production technology, photography such as digital photography or videography for art, documentary or technical purposes, safety technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology, or science. However, other applications are also possible. [Background technology]

[0002] Authentication systems available on mobile devices such as smartphones and tablets include a camera. These mobile devices typically feature a front display, such as an organic light-emitting diode (OLED) area and / or a quantum dot light-emitting diode (QLED) area. The camera may be positioned behind this front display. Furthermore, such authentication devices may also have one or more light-emitting diodes and / or lasers, or other light projectors, positioned behind the display. Typically, the light projector projects a pattern, such as a dot pattern, onto a target (e.g., a face), the camera captures the projected image for the user, and a processor determines the material information. If the material is classified as skin, it is identified as a human; otherwise, it is identified as a spoofed target. The pattern generated by the light projector can be designed for 3D algorithms; that is, the pattern can be designed to easily solve the so-called correspondence problem. In smartphone applications, the resulting 3D depth map can be used for further facial recognition.

[0003] However, when the projector and camera are placed behind a display such as an OLED display, the performance of the authentication system deteriorates. Even transparent OLEDs have low transmittance, and strong diffraction can distort the optical image. As a result, the light beam from the projector may be split into multiple light beams. The light beam loses power due to diffraction, and the output of the light beam projected to the user is significantly reduced. For example, under certain conditions, with an 80% diffraction loss and an OLED transmittance of 10%, only about 2% of the light beam remains. This performance degradation can be compensated for by increasing the projector output, but this is undesirable considering the battery performance of smartphones. Furthermore, light beams from the side order of diffraction may be diffused into the scene in undesirable ways. This makes it even more difficult to find the projected spot. [Overview of the project] [Problems that the invention aims to solve]

[0004] Therefore, an object of the present invention is to provide a device and method that addresses the aforementioned technical problems of known devices and methods. Specifically, an object of the present invention is to provide a device and method that improves the signal-to-noise ratio in pattern images, thereby enabling accurate and reliable secure authentication even behind low-transmittance OLEDs. [Means for solving the problem]

[0005] This problem is solved by a method for authenticating a device user and a device for authenticating a user, having the features of the independent claim. Advantageous embodiments that can be realized in one or any combination are described in the dependent claims and throughout the specification.

[0006] A first aspect of the present invention discloses a method for authenticating a device user. This method involves the following steps: a. A projector projects multiple light beams onto a user through a display, particularly a device display, wherein the multiple light beams include a first light beam and a second light beam, and the first light beam and the second light beam are directed to illuminate an area of ​​the user that at least partially overlaps with the display. b. A step of generating a pattern image showing the projection of the multiple light beams onto the user, c. A step of extracting biological data from the pattern image, d. The step of allowing the user to perform operations on a device that requires authentication based on the biometric data, Includes.

[0007] The above method steps may be performed in a given order or in a different order. Furthermore, there may be one or more additional method steps that are not listed. Furthermore, one, more than one, or even all of the method steps may be repeated.

[0008] This method may be carried out by a computer. As used herein, the term “carried out by a computer” is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, this term may refer to a method comprising at least one computer and / or at least one computer network. The computer and / or computer network may comprise at least one processor configured to perform at least one of the steps of the method according to the present invention. Specifically, each step of the method is performed by the computer and / or computer network. This method may be performed entirely automatically, specifically without user intervention.

[0009] The devices may be selected from the group consisting of: television devices; game consoles; personal computers; mobile devices, in particular mobile phones, and / or smartphones, and / or tablet computers, and / or laptops, and / or tablets, and / or virtual reality devices, and / or wearables such as smartwatches; or other types of portable computers. The term “user” as used herein is a broad term and should be given the usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to any person who intends to use the device and / or a person who uses the device.

[0010] As used herein, the term “authenticate” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to verifying a user’s identity, without limitation. Specifically, authentication may include distinguishing a user from other people or objects, in particular distinguishing between authorized and unauthorized access. Authentication may include verifying the identity of each user and / or assigning an identity to a user. Authentication may include generating and / or providing identity information to other devices or units, such as at least one authorization unit, in order to provide access to a device. Identity information may be proven by authentication. For example, identity information may be at least one identity token and / or may include at least one identity token. If authentication is successful, the facial image recorded by at least one image generation unit can be verified to be the user's facial image, and / or the user's identity can be verified. Authentication can be performed using at least one authentication process. The authentication process may include several steps, such as at least one face detection on at least one flood image, and at least one identification step in which an identity is assigned to the detected face, and / or verification of the user's identity is performed, as will be described in more detail below.

[0011] As used herein, the term “light” is a broad term and should be given its usual and conventional meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to electromagnetic radiation in one or more of the infrared, visible, and ultraviolet spectral ranges. Here, the term “ultraviolet spectral range” generally refers to electromagnetic radiation having wavelengths from 1 nm to 380 nm, preferably from 100 nm to 380 nm. Furthermore, in part in accordance with the version of the standard ISO-21348 effective as of the date of this document, the term “visible spectral range” generally refers to the spectral range from 380 nm to 760 nm. The term "infrared spectral range" (IR) generally refers to electromagnetic radiation from 760 nm to 1000 μm, with the range from 760 nm to 1.5 μm usually denoted as the "near-infrared spectral range" (NIR), the range from 1.5 μm to 15 μm as the "mid-infrared spectral range" (MidIR), and the range from 15 μm to 1000 μm as the "far-infrared spectral range" (FIR). Preferably, the light used for typical purposes of the present invention is light in the infrared (IR) spectral range, more preferably near-infrared (NIR) and / or mid-infrared spectral range (MidIR) light. Specifically, the projector projects at least one infrared light pattern, and the projected light beam has wavelengths in the infrared spectral range, preferably 800 nm to 1300 nm, more preferably 900 nm to 1000 nm, and most preferably 1100 nm to 1200 nm.

[0012] As used herein, the term “project” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to, without limitation, providing at least one light beam, in particular a light pattern, onto at least one surface. As used herein, the term “projector” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to, without limitation, an optical device configured to project at least one light beam onto a surface. A projector may be configured to generate and / or provide at least one light pattern, in particular at least one infrared light pattern.

[0013] As used herein, the term “light pattern” is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to at least one arbitrary pattern comprising multiple light spots. The light spots may be at least partially spatially extended. At least one or any of the spots may have any shape. In some cases, at least one or any of the spots may be circular in shape. The spots may be arranged in consideration of the structure of a display. Typically, the arrangement of the OLED pixel structure of a display may be considered.

[0014] The light pattern may be an infrared light pattern. As used herein, the term “infrared light pattern” is a broad term and should be given a common and conventional meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to a light pattern containing spots in the infrared spectral range. The infrared light pattern may also be a near-infrared light pattern.

[0015] The light projected by the projector may be coherent. The light pattern may be coherent, particularly an infrared light pattern. The projector may be configured to emit light of a single wavelength, for example, a single wavelength in the near-infrared region. In other embodiments, the projector may be adapted to emit light at multiple wavelengths, for example, to enable additional measurements in other wavelength channels.

[0016] The light pattern may include regular and / or constant and / or periodic patterns such as triangular patterns, rectangular patterns, hexagonal patterns, or patterns including convex tiling. For example, the light pattern is a hexagonal pattern, preferably a hexagonal light pattern, and preferably a 2 / 5 hexagonal infrared light pattern. Using a periodic 2 / 5 hexagonal pattern makes it possible to distinguish between noise (artefact) and usable signal.

[0017] The light pattern may include at least one dot pattern. The light pattern has a low dot density. For example, the number of light beams projected onto the user is less than 5000, preferably less than 3000, more preferably less than 2000, even more preferably less than 1500, and most preferably less than 1000. The light pattern may have a low dot density compared to other structured light techniques, which typically have a dot density of 10k to 30k, especially in a 55° × 38° field of view. Using such a low dot density can compensate for the aforementioned diffraction loss. The contrast of the pattern image can be increased by reducing the number of spots projected onto the object and / or the user. Increasing the number of dots reduces the illuminance per dot. A decrease in the number of dots leads to an increase in the illuminance of the dots, resulting in improved contrast of the pattern image of the infrared light pattern projection. The light pattern may have a periodic dot pattern with a reduced number of dots, where each dot has high irradiance. Such a light pattern can ensure improved authentication using an illumination source and image generation unit on the back of the display. Furthermore, a smaller number of spots ensures compliance with eye safety and stability requirements. The allowed dose can be divided among the spots in the light pattern.

[0018] At least one light spot may be associated with a beam divergence angle of 0.2° to 0.5°, preferably 0.1° to 0.3°. The term “beam divergence angle” as used herein is a broad term and should be given its usual and conventional meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may, without limitation, refer to at least one measurement of the increase in at least one diameter equivalent value, such as at least one diameter and / or radius, with respect to the distance from the optical aperture from which the beam is emitted. This measurement may be an angle or an angle equivalent value. In the context of the present invention, typically, the beam divergence angle is usually 1 / e 2 It can be determined by that.

[0019] A projector may include at least one emitter, and more particularly, multiple emitters. As used herein, the term “emitter” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to at least one arbitrary device configured to provide at least one light beam, without limitation. The emitter may be selected from the group consisting of: at least one laser source such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separated-containment heterostructure laser, at least one quantum cascade laser, at least one dispersion Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode-excited laser, at least one dispersion feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical-cavity surface-emitting laser (VCSEL); at least one non-laser source such as at least one LED or at least one light bulb; and at least one edge-emitting laser.

[0020] For example, the projector includes at least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs can be arranged in at least one array (e.g., including a matrix of VCSELs). The VCSELs can be arranged on the same substrate or on different substrates. As used herein, the term "vertical cavity surface emitting laser" is a broad term and should be given the meaning that is ordinary and customary to those of skill in the art and should not be limited to a special or customized meaning. This term can specifically refer, without limitation, to a semiconductor laser diode configured to emit a laser beam perpendicular to an upper surface. Examples of VCSELs are described, for example, at en.wikipedia.org / wiki / Vertical-cavity_surface-emitting_laser. VCSELs are generally known to those of skill in the art, such as from WO2017 / 222618A. Each of the VCSELs is configured to generate at least one light beam. The VCSEL or plurality of VCSELs can be configured to generate a desired number of spots. The VCSEL can be configured to emit a light beam in a wavelength range from 800 to 1000 nm. For example, the VCSEL can be configured to emit a light beam at 808 nm, 850 nm, 940 nm, and / or 980 nm. The VCSEL preferably emits light at 940 nm, because, as described, for example, in CIE085-1989 "Solar spectral Irradiance", the solar irradiance on the earth has a minimum irradiance at this wavelength.

[0021] The display may be configured to correct spots as the spots generated by the projector traverse the display, for example by increasing the number of spots. For example, the display can function as a diffractive optical element (DOE). This saves resources because the display functions as a DOE, eliminating the need for additional DOEs. Furthermore, since the light beam is amplified by the display, fewer emitters, such as VCSEL cavities, are required. Additionally or alternatively, the projector may include at least one optical element selected from the group consisting of: at least one lens; at least one microlens array (MLA); at least one diffractive optical element (DOE); and at least one metasurface element, configured to correct spots, for example by increasing the number of spots. The DOE and / or metasurface element may be configured to generate multiple light beams from a single incident light beam. For example, an optical element including a VCSEL that projects up to 2000 spots and multiple metasurface elements can be used to duplicate the number of spots. Further configurations are possible, in particular further configurations including different numbers of projecting VCSELs and / or at least one different optical element configured to increase the number of spots. Other multiplication factors are also possible. For example, a VCSEL or multiple VCSELs may be used, and the generated laser spot may be replicated by using at least one DOE.

[0022] The projector includes at least one transfer device. The term “transfer device,” also referred to as “transfer system,” as used herein is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may refer to one or more optical elements adapted to modify a light beam, used to generate at least a portion of a light beam, particularly an infrared light pattern, such as by changing one or more of the beam parameters of the light beam, the width of the light beam, or the direction of the light beam. The transfer device may have at least one imaging optical device. The transfer device may specifically include at least one selected from the group consisting of: at least one lens, e.g., at least one lens selected from the group consisting of at least one adjustable lens, at least one aspherical lens, at least one spherical lens, and at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or beam splitting mirror; at least one multi-lens system; at least one holographic optical element; and at least one meta-optical element. Specifically, the transfer device comprises at least one refractive optical lens stack. The transfer device may also comprise a multi-lens system having refractive properties.

[0023] This method can further include emitting flood light by at least one flood illumination source and generating at least one flood image while the flood illumination source emits the flood light. As used herein, the term "flood illumination source" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. Specifically, this term can refer to at least one arbitrary device configured to provide substantially continuous spatial illumination without limitation. As used herein, the term "flood light" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. Specifically, this term can refer to substantially continuous spatial illumination, particularly diffused and / or uniform illumination without limitation. The flood light has wavelengths in the infrared range, particularly in the near-infrared range. The flood illumination source can include at least one LED or at least one VCSEL, preferably a plurality of VCSELs. The plurality of VCSELs can overlap in a uniform area. As used herein, the term "substantially continuous spatial illumination" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. Specifically, this term can refer to uniform spatial illumination where non-uniform areas may exist without limitation. The area irradiated by the flood illumination source (e.g., an area covering a user, a part of the user, and / or the user's face) can be continuous. The output can be diffused over the entire illumination field. In contrast, illumination by an optical pattern can include at least two continuous areas, particularly a plurality of continuous areas, and / or the output can be concentrated in a small area within the illumination field (compared to the entire illumination field). Infrared flood illumination is suitable for irradiating a continuous area, particularly one continuous area. Infrared pattern illumination is suitable for irradiating at least two continuous areas.

[0024] A flood irradiation source can irradiate a measurement area (e.g., the user, a part of the user, and / or the user's face) with a substantially constant irradiation intensity. The term “constant” as used herein is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may, without limitation, refer to the temporal aspect of the exposure time. Flood light may vary over time and / or be substantially constant over time. The term “substantially constant” as used herein is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may, without limitation, refer to perfectly constant irradiation and embodiments in which deviation from constant irradiation is possible within ±10% (≤±10%), preferably ±5% (≤±5%), and more preferably ±2% (≤±2%).

[0025] The emission of flood light and the illumination of the light pattern may be performed consecutively or at least partially overlapping. For example, the flood light and the light pattern may be emitted simultaneously. For example, one of the flood light or the light pattern may be emitted at a lower intensity than the other.

[0026] The projector and flood source may include at least one VCSEL, preferably a plurality of VCSELs. The projector may comprise a plurality of first VCSELs mounted on a first platform. The flood source may comprise a plurality of second VCSELs mounted on a second platform. The second platform may be located next to the first platform. The projector may include a heat sink. A first increment, including the first platform, may be mounted above the heat sink. A second increment, including the second platform, may be mounted above the heat sink. The second increment may differ from the first increment. Thus, the first platform may be further away from the optical elements configured to increase (e.g., duplicate) the number of spots. The second platform may be closer to the optical elements. The beams emitted from the second VCSELs may be focal-shifted, thereby forming overlapping spots. This results in substantially continuous illumination, i.e., flood illumination.

[0027] As used herein, the term “display” is a broad term and should be given the usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term can refer to any shape of device configured to display information items. Information items may be any information, such as at least one image, at least one chart, at least one histogram, at least one graphic, text, number, at least one symbol, or operation menu. A display may be at least one display panel or may include at least one display panel. A display may have any shape, for example, rectangular. A display may be the front display of a device. A display may include at least one of a display panel, in particular having a plurality of pixels and / or a plurality of transistors, or glass, in particular cover glass, in particular glass configured to cover a display panel.

[0028] The display, specifically the display panel, may be, or comprise, at least one organic light-emitting diode (OLED) display and / or at least one quantum dot light-emitting diode (QLED). As used herein, the term “organic light-emitting diode (OLED)” is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to a light-emitting diode (LED) which is a film of an organic compound in which the light-emitting electroluminescent layer is configured to emit light in response to an electric current. OLED displays may be configured to emit visible light. As used herein, the term “quantum dot light-emitting diode (QLED)” is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to a display technology that utilizes semiconductor particles called quantum dots to generate color on a display. These quantum dots emit light of several different colors depending on their size when excited by light. By using a combination of red, green, and / or blue quantum dots, QLED displays can display a wide range of colors with high brightness and color accuracy.

[0029] The display may be at least partially transparent. The display may be at least partially transparent in at least one continuous area covering the projector, flood source, and / or image generation unit. The transmittance of the display may be 20% or less, preferably 15% or less, and more preferably 10% or less. The intensity of the light beam after projection through the display may be 10% or less (≤10%) of the intensity of the light beam at the time of emission. The display may have a transmittance of 10%. Preferably, the transmittance is less than 8%, more preferably less than 6%, even more preferably less than 5%, even more preferably less than 4%, even more preferably less than 3%, and most preferably less than 2.5%.

[0030] The display has at least one continuous area: - A light pattern incident on a continuous area while being projected from a projector traverses the display; - Flood light incident on a continuous area while being illuminated from a flood source traverses the display; - User light, generated by a light pattern and / or flood light shining on the user, incident on a continuous area, traverses the display to collide with the image generation unit. It may be at least partially transparent so as to satisfy at least one of the following conditions.

[0031] As used herein, the term “at least partially transparent” is a broad term and should be given a common and conventional meaning to those skilled in the art, and should not be limited to a special or customized meaning. Specifically, the term may refer, without limitation, to the property of a display to transmit at least partially light in a particular wavelength range (e.g., the infrared spectral region, particularly the near-infrared spectral region). For example, a display may be semi-transparent in the near-infrared region. For example, a display may have a transmittance of 20% to 50% in the near-infrared region. A display may have different transmittances for different wavelength ranges. The present invention may propose a device comprising an image generation unit and a projector that can be positioned behind the display of the device. The transparent area of ​​the display allows for the operation of the image generation unit and projector behind the display.

[0032] The display can be at least partially transparent as described above. A partially transparent continuous area of ​​the display can be associated with a first pixel density value (pixels per inch (PPI)), and further areas of the display can be associated with a second pixel density value. The first pixel density value may be lower than the second pixel density value. The transmittance of light passing through a continuous area may be higher than the transmittance of light passing through another area. The first pixel density value may be 450 PPI or less, preferably 300 to 440 PPI, more preferably 350 to 450 PPI. The first pixel density value may be constant with a maximum deviation of 20%, preferably 10%, across the entire continuous area. The second pixel density value may be 400 to 500 PPI, preferably 450 to 500 PPI.

[0033] A continuous area of ​​at least partially transparent parts of a display may include a first area and a second area. The first area may be associated with a first number of transistors configured to control at least one pixel, and the second area may be associated with a second number of transistors configured to control at least one pixel, the first number of transistors may be smaller than the second number of transistors. The first number of transistors and / or the second number of transistors refer to or may be the density of transistors.

[0034] As used herein, the term “pixel” is a broad term and should be given its usual and conventional meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to an image unit representing an addressable element, in particular the smallest image unit. A collection of pixels may represent a display. Pixels can be manipulated by changing their color, brightness, contrast, etc. In particular for manipulating pixels, pixels may be driven by at least one transistor, for example, a transistor that controls the current required to drive the pixels. Typically, thin-film transistors may be used to drive pixels. TFTs may be preferably used in flat-panel displays.

[0035] The method involves directing a first light beam and a second light beam to illuminate an area of ​​the user that at least partially overlaps with the display. The term “light beam” as used herein is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to a quantity of light, specifically a quantity of light traveling in essentially the same direction, including the possibility that the light beams have a diffusion angle or broadening angle. The light beams may specifically be Gaussian rays. However, other embodiments are possible. The terms “first” and “second” are used purely as nominal terms and do not provide any information, in particular, regarding their order, and / or, for example, whether other light beams exist. The term “partially overlapping” as used herein is a broad term and should be given in a common and conventional sense to those skilled in the art, and should not be limited to any special or customized meaning. This term can specifically refer, without limitation, to at least two light spots projected onto the user's surface and illuminating the same area (e.g., completely identical areas, or at least partially identical areas). For example, the overlap is at least 10%, preferably at least 20%, and more preferably at least 50%. Higher overlaps (e.g., 100%) are also possible.

[0036] For example, a first light spot (e.g., a light spot with a first radius r1) can illuminate a first area of ​​the user, and a second light spot with a second radius r2 (r1 ≤ r2) can illuminate a second area of ​​the user. For example, the first and second areas may be the same or different. For example, the first radius may be smaller than the second radius. For example, the first and second areas may be offset from each other. Any combination of these examples is possible.

[0037] Because at least two light spots partially overlap, the intensity of the overlapping area increases. This enables accurate, reliable, and secure authentication even behind displays with low light transmittance.

[0038] Specifically, a display can have diffraction properties. A display can function as a grating with a periodic structure that diffracts incident light from a projector into multiple beams traveling in different directions, i.e., at different diffraction angles. The diffraction angle may depend on the structure of the display and the wavelength of the incident light. For example, the diffraction of a display is:

number

[0039] The projector's radiation angle may be selected according to the following rules:

number

number

[0040] The projector's radiation angle can be selected by taking into account possible deviations from an ideal periodic pixel grid, particularly by using the rules described above. The relationship between the pixel arrangements of the display, defined by the distance d between at least two pixels in at least one dimension, can have a deviation of up to 50%, preferably up to 40%, and more preferably up to 25%. Thus, the projector's radiation angle θ proj,n This can be selected considering a tolerance of 50% or less (≤50%), preferably 40% or less (≤40%), and more preferably 25% or less (≤25%).

[0041] This rule makes it possible to match the minor diffraction order with the projector's new or original (zero-order) light beam. For example, primary and tertiary spots fall together, and secondary spots fall together. This can lead to the convergence of light spots. This can reduce the projector's radiant energy loss. Another advantage is that artifacts can be eliminated or utilized by adjusting the alignment of the projector and display so that n-th order spots converge in a single spatial position. This enables accurate, reliable, and secure authentication even behind displays with low transmittance.

[0042] Projectors can feature emitters with inherently narrow emission profiles. For example, a combination of a VCSEL and optical elements such as an MLA, DOE, metasurface, or lens can be used. Such a configuration can have a narrower emission profile than commercially available LEDs.

[0043] As described above, the method includes generating a pattern image showing the projection of multiple light beams onto a user. The generation of the pattern image may be performed by using at least one image generation unit. The term “image generation unit” as used herein is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may refer to at least one unit of a device configured to generate at least one image, without limitation. Images may be generated via hardware and / or software interfaces, which can be considered image generation units. The terms “image generation” or “imaging” as used herein are a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may refer to capturing and / or generating and / or determining and / or recording at least one image by using an image generation unit, without limitation. Image generation may include imaging and / or recording an image. Image generation may include capturing multiple images, such as a single image and / or a sequence of images. To generate an image via a hardware and / or software interface, capturing and / or generating and / or determining and / or recording an image may be triggered and / or initiated by the hardware and / or software interface. For example, image generation may include the continuous recording of a sequence of images, such as a video or movie. Image generation may be initiated by user operation, or it may be initiated automatically, for example, when the presence of at least one object or user is automatically detected within the field of view of the image generation unit and / or within a predetermined sector of the field of view. As used herein, the term “field of view” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning.This term can specifically refer, without limitation, to the angular range of the observable world and / or at least one scene that can be captured or observed by an optical system such as an image-generating unit. Typically, the field of view may be expressed in degrees and / or radians, and exemplarily, it may represent the total angle extended by the image and / or observable area.

[0044] The image generation unit may comprise at least one optical sensor, in particular at least one pixelated optical sensor. The image generation unit may comprise at least one CMOS sensor or at least one CCD chip. For example, the image generation unit may comprise at least one CMOS sensor that may be sensitive to the infrared spectral range. As used herein, the term “image” is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may, without limitation, refer to data recorded by using an optical sensor (e.g., multiple electronic readings from a CMOS chip or CCD chip). The image may include raw image data or it may be a pre-processed image. For example, pre-processing may include applying at least one filter to the raw image data and / or at least one background correction and / or at least one background removal.

[0045] For example, an image generation unit may include one or more of the following: at least one monochrome camera (e.g., including monochrome pixels), at least one color (e.g., RGB) camera (e.g., including color pixels), and at least one IR camera. The cameras may be CMOS cameras. The camera may include at least one monochrome camera chip (e.g., a CMOS chip). The camera may include at least one color camera chip (e.g., an RGB CMOS chip). The camera may include at least one IR camera chip (e.g., an IR CMOS chip). For example, the camera may include monochrome (e.g., black and white) pixels and color pixels. Color pixels and monochrome pixels can be combined within the camera. The camera may generally include a one-dimensional or two-dimensional array of image sensors such as pixels.

[0046] As described above, the image generation unit may be at least one camera. For example, the camera may be the device's built-in camera and / or an external camera. As described above, the device's built-in camera and / or external camera may be accessed via a hardware and / or software interface used as the image generation unit. If the device is a smartphone or includes a smartphone, the image generation unit may be the smartphone's front camera and / or back camera, such as a selfie camera.

[0047] The image generation unit may have a field of view ranging from 10°×10° to 75°×75°, preferably 55°×65°. The image generation unit may have a resolution of less than 2MP, preferably in the range of 0.3MP to 1.5MP.

[0048] The image generation unit may include further elements such as one or more optical elements (e.g., one or more lenses). For example, the light sensor may be a fixed-focus camera having at least one lens fixedly tuned to the camera. Alternatively, the camera may have one or more variable lenses that are automatically or manually adjustable. The camera may have at least one light filter, for example, at least one bandpass filter. The bandpass filter may be matched to the spectrum of a light emitter. However, other cameras are also possible.

[0049] As used herein, the term “pattern image” is a broad term and should be given a common and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may, without limitation, refer to an image produced by an image generation unit while, for example, an object and / or a user is illuminated with a light pattern. A pattern image may include an image showing at least a portion of a user, particularly the user’s face, while the user is illuminated with the light pattern, particularly each area of ​​interest included in the image. A pattern image may be produced by imaging and / or recording light reflected by an object and / or a user illuminated by the light pattern. A pattern image showing a user may include at least a portion of the light pattern illuminated on at least a portion of the user. For example, projection by a projector and imaging by using an image generation unit may be synchronized, for example, by using at least one control unit of the device.

[0050] As described above, the method may include the emission of flood light from at least one flood source and the generation of at least one flood image while the flood source is emitting the flood light. The term “flood image” as used herein is a broad term and should be given a common and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may, without limitation, refer to an image generated by an image generation unit while the source is emitting infrared flood light, for example, onto an object and / or a user. The flood image may include an image showing the user, particularly the user’s face, while the user is being illuminated by the flood light. The flood image may be generated by imaging and / or recording the light reflected by the object and / or user illuminated by the flood light. A flood image showing a user may include at least a portion of the flood light on at least a portion of the user. For example, illumination by the flood source and imaging by using an image generation unit can be synchronized, for example, by using at least one control unit of the device.

[0051] The image generation unit may be configured to image and / or record pattern images and flood images simultaneously or at different timings. The image generation unit may be configured to image and / or record pattern images and flood images in at least partially overlapping measurement areas or areas corresponding to measurement areas.

[0052] The device may be configured to authenticate the user of a device in order to perform at least one operation on a device requiring authentication. The device may include at least one authentication unit configured to perform at least one user authentication process, particularly using flood images and pattern images. The authentication unit may be configured to use a facial recognition process that operates on flood images, pattern images, and / or extracted biometric data, particularly extracted biometric data derived from pattern images.

[0053] As used herein, the term “authentication unit” is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer to at least one unit configured to perform at least one user authentication process. An authentication unit may be at least one processor, or may include at least one processor, and / or may be designed as software or an application.

[0054] The term “processor” as used herein is a broad term and should be given its usual and idiomatic meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to any logic circuit configured to perform the basic operations of a computer or system, and / or, generally, a device configured to perform calculations or logical operations. In particular, a processor may be configured to process basic instructions that drive a computer or system. As an example, a processor may comprise at least one arithmetic logic unit (ALU), at least one floating-point unit (FPU), such as a mathematical coprocessor or a numerical coprocessor, a number of registers, specifically registers configured to supply operands to the ALU and store the results of calculations, and memory such as L1 cache memory and L2 cache memory. In particular, a processor may be a multi-core processor. Specifically, a processor may be, or comprise, a central processing unit (CPU). Additionally or alternatively, the processor may be a microprocessor, or comprise a microprocessor, and therefore specifically, the elements of the processor may be contained on a single integrated circuit (IC) chip. Additionally or alternatively, the processor may be one or more chips, such as one or more application-specific integrated circuits (ASICs) and / or one or more field-programmable gate arrays (FPGAs) and / or one or more tensor processing units (TPUs) and / or dedicated machine learning optimization chips. Specifically, the processor may be configured to perform one or more evaluation operations, for example, by software programming. At least one or any component of a computer program configured to perform the authentication process may be executed by the processing device. Alternatively or additionally, the authentication unit may be a connection interface, or comprise a connection interface. The connection interface may be configured to transfer data from one device to a remote device, or vice versa.At least one or any component of a computer program configured to perform the authentication process may be executed by a remote device.

[0055] For example, an authentication unit can perform at least one face detection using a flood image. Face detection may be performed locally on the device. However, face identification, i.e., assigning an ID (identity) to the detected face, may be performed remotely, such as in the cloud, especially if identification, not just verification, is required. User templates can be stored on a remote device such as the cloud, eliminating the need for local storage. This can be advantageous in terms of storage capacity and security.

[0056] The authentication unit may be configured to identify a user based on a flood image. Therefore, in particular, the authentication unit may transfer data to a remote device. Alternatively or additionally, the authentication unit may perform user identification based on a flood image, in particular by running an appropriate computer program having the respective function. The term “identifying” as used herein is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized meaning. Identifying may include assigning an identity to a detected face, and / or performing at least one identity check, and / or verifying the user’s identity.

[0057] The authentication process may involve multiple steps.

[0058] For example, the authentication process may include performing at least one face detection step. The face detection step may include analyzing the flood image. Furthermore, for example, the authentication process may include identification. Identification may include assigning an identity to the detected face and / or performing at least one identity verification and / or verifying the user's identity. Identification may include performing face verification to determine that the imaged face is the user's face. User identification may include matching the template with the flood image, for example, showing the contours of part of the user, particularly the contours of part of the face. User identification may include determining whether the imaged face is the user's face, and in particular whether the imaged face corresponds to an image of the user's face stored, for example, in at least one memory of the device. If the flood image does not match the image template, authentication may fail.

[0059] The analysis of a flood image may include one or more of the following: filtering; selection of at least one region of interest; formation of a difference image between the flood image and at least one offset; inversion of the flood image; background correction; decomposition into color channels; decomposition into hue; saturation; luminance channels; frequency decomposition; singular value decomposition; application of a Canney edge detector; application of the Laplace operator of a Gaussian filter; application of a difference Gaussian filter; application of the Sobel operator; application of the Laplace operator; application of the Schall operator; application of the Prewitt operator; application of the Roberts operator; application of the Kirsch operator; application of a high-pass filter; application of a low-pass filter; application of a Fourier transform; application of the Radon transform; application of the Huff transform; application of a wavelet transform; thresholding; and generation of a binary image. The region of interest may be determined manually by the user or automatically, such as by recognizing a user in the image. In particular, the analysis of a flood image may include the use of at least one image recognition technique, in particular face recognition technique. The image recognition technique includes at least one process of identifying a user in the image. Image recognition may include the use of at least one technique selected from the following: color-based image recognition using features such as template matching; segmentation and / or blob analysis using size or shape; and machine learning and / or deep learning using at least one convolutional neural network.

[0060] Analysis of flood images may include determining multiple facial features. Analysis may include comparing, in particular matching, the determined facial features with template features. Template features may be features extracted from at least one template. A template may be, or include, at least one image generated during the registration process (e.g., during device initialization). A template may be an image of an authorized user. Template features and / or facial features may include vectors. Feature matching may include determining the distance between vectors. User identification may include comparing the distance between vectors to at least one predefined threshold value, and user identification is successful if the distance is at least within an acceptable range and less than or equal to the predefined threshold value. Otherwise, the user is declining and / or rejected.

[0061] For example, image recognition may include using a trained model that includes at least one model, particularly at least one face recognition model. Flood image analysis can be performed by using a face recognition system such as FaceNet, described, for example, Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," arXiv:1503.03832. The trained model may include at least one convolutional neural network. For example, a convolutional neural network may be designed as described in MD Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," CoRR, abs / 1311.2901, 2013, or C. Szegedy et al., "Going deeper with convolutions," CoRR, abs / 1409.4842, 2014. For more information on convolutional neural networks for face recognition systems, see Florian Schroff, Dmitry Kalenichenko, and James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," arXiv:1503.03832. Labeled image data from an image database can be used as training data.Specifically, labeled faces can be obtained from one or more of the following: the YouTube® Faces database, as described in "Labeled faces in the wild: A database for studying face recognition in unconstrained environments" by GB Huang, M. Ramesh, T. Berg, and E. Learned-Miller, Technical Report 07-49, University of Massachusetts, Amherst, October 2007; or in "Face recognition in unconstrained videos with matched background similarity" by Wolf, T. Hassner, and I. Maoz at the IEEE Conf. at CVPR in 2011; or the Google® Facial Expression Comparison dataset. Training of a convolutional neural network can be carried out as described in Florian Schroff, Dmitry Kalenichenko, and James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," arXiv:1503.03832.

[0062] As described above, this method includes extracting biometric data (liveness data) from pattern images, for example, by using an authentication unit.

[0063] Therefore, in particular, the authentication unit can transfer data to a remote device. Alternatively or additionally, the authentication unit can extract biometric data based on pattern images, in particular by running appropriate computer programs having their respective functions. In particular, by considering biometric data as a parameter for validating the authentication process, the authentication process can be robust against deception by using recorded images of the user.

[0064] The authentication unit may be configured to outsource at least one step of the authentication process, such as user identification, and / or at least one step of the verification process, such as consideration of material data, to a remote device, specifically a server and / or a cloud server. The device and the remote device may be part of a computer network, particularly the internet. This allows the device to be used as a field device used by the user to generate the data necessary for the authentication process and / or its verification. The device may transmit the generated data and / or data related to intermediate steps of the authentication process and / or its verification to the remote device. In such a scenario, the authentication unit may be, and / or have, a connection interface configured to transmit information to the remote device. The data generated by the remote device used for the authentication process and / or its verification may be further transmitted to the device. This data may be received by the connection interface on which the device is provided. The connection interface may be configured in particular for the transmission or exchange of information. In particular, the connection interface can provide a data transfer connection. As an example, the connection interface may be, or include, at least one port, including one or more of a network port or internet port, a USB port, or a disk drive.

[0065] It should be emphasized that data from a device may be sent to a specific remote device depending on at least one circumstance (such as date, day of the week, or load on a particular remote device). A specific remote device may not be selected by the field device. Rather, a further device may select a specific remote device to which data can be sent. The authentication process and / or the generation of validity verification data may involve the use of multiple different entities of the remote device. At least one entity may generate intermediate data, which can then be sent to at least one further entity.

[0066] As used herein, the term “biometric data” is a broad term and should be given its usual and conventional meaning to those skilled in the art, and should not be limited to any special or customized meaning. Specifically, the term may refer, without limitation, to data that enables the distinction between living human beings, particularly users, and non-living materials such as paper or 3D face masks. Biometric data may include blood perfusion data and / or material data. Extraction of biometric data may include extraction of material data and / or blood perfusion data. Biometric data may include information about the material on the user's surface onto which the spot is projected. Biometric data may include information about at least one vital sign.

[0067] This method may include, for example, extracting material data from a pattern image by beam profile analysis of light spots, using an authentication unit. For beam profile analysis, see WO2018 / 091649A1, WO2018 / 091638A1, and WO2018 / 091640A1, the full contents of which are included by reference. Beam profile analysis enables reliable scene classification based on a small number of light spots. Each light spot in a pattern image may include a beam profile. In this specification, "beam profile" generally refers to at least one intensity distribution of a light spot in an optical sensor as a function of pixels. The beam profile may be selected from the group consisting of trapezoidal beam profiles; triangular beam profiles; conical beam profiles; and linear combinations of Gaussian beam profiles.

[0068] Extracting material data from a pattern image may include generating material types and / or generating data derived from material types. Preferably, the extraction of material data may be based on a pattern image. Material data may be extracted by using at least one model. Extracting material data may include providing a pattern image to a model and / or receiving material data from a model. Providing a pattern image to a model may include receiving a pattern image in the model's input layer or receiving a pattern image via the model's loss function, and these may follow.

[0069] The model may be a data-driven model. A data-driven model may include encoder-decoder structures such as convolutional neural networks and / or autoencoders. Other examples of models that generate representations may include FFT, wavelets, deep learning such as CNNs, energy models, normalization flows, GANs, vision transformers, or transformers used in natural language processing, autoregressive image modeling, normalization flows, deep autoencoders, and deep energy-based models. Supervised or unsupervised schemes are applicable to generate representations, and embedding into cosine or Euclidean metrics in ML languages ​​is also possible, for example. A data-driven model may be parameterized based on a training dataset comprising at least one image data and material data, preferably at least one pattern image data and material data. In another embodiment, extracting material data may include providing pattern images to the model and / or receiving material data from the model. In another embodiment, a data-driven model may be trained according to a training dataset comprising at least one image and material data. In another embodiment, a data-driven model may be parameterized according to a training dataset comprising at least one image and material data. A data-driven model can be parameterized according to a training dataset, receive images, and provide material data based on the received images. The training dataset may include at least one image and material data, preferably material data associated with at least one image.

[0070] An image is either a representation of an image or may contain a representation of an image. The representation may be a low-dimensional representation of an image. The representation may contain at least some of the data or information associated with the image. The representation of an image may contain feature vectors. In one embodiment, determining a representation, in particular a low-dimensional representation, may be based on principal component analysis (PCA) mapping or radial basis function (RBF) mapping. Determining a representation may also be called generating a representation. Generating a representation based on PCA mapping may involve clustering based on features of a pattern image and / or sub-images. Additionally or alternatively, generating a representation may be based on a neural network structure suitable for dimensionality reduction. A neural network structure suitable for dimensionality reduction may include an encoder and / or decoder. In one example, the neural network structure may be an autoencoder. In one example, the neural network structure may include a convolutional neural network (CNN). A CNN may include at least one convolutional layer and / or at least one pooling layer. A CNN can reduce the dimensionality of a sub-image and / or pattern image, for example, by applying convolutions based on the convolutional layer and / or by pooling. The application of convolution may be suitable for selecting features related to material information in pattern images.

[0071] The model may be suitable for determining output based on input. In particular, the model may be suitable for determining material data based on an image as input. The model may be a deterministic model, a data-driven model, or a hybrid model. A deterministic model may include, for example, a first-principles model, preferably reflecting a physical phenomenon in mathematical form. A deterministic model may include a set of equations that describe the interaction between a material and patterned electromagnetic radiation, thereby yielding state measurements, vital sign measurements, etc. A data-driven model may be a classification model. A hybrid model may be a classification model that includes at least one machine learning architecture with deterministic or statistical adaptations and model parameters. Statistical or deterministic adaptations may be introduced to improve the quality of results, as they provide a systematic relationship between empirical rules and theory. In one embodiment, the data-driven model may be a classification model. The classification model may include at least one machine learning architecture and model parameters. For example, a machine learning architecture may be one or more of the following, or include them: linear regression, logistic regression, random forest, piecewise linear, nonlinear classifier, support vector machine, naive Bayes classification, nearest neighbor classification, neural network, convolutional neural network, generative adversarial network, support vector machine, or gradient boosting algorithm. In the case of a neural network, the model may be a multiscale neural network, or a recurrent neural network (RNN), such as a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network, but is not limited to these. A data-driven model may be parameterized based on a training dataset. A data-driven model may be trained based on a training dataset. Training a model may include parameterizing the model. The term training may also be written as learning. Specifically, the term may refer to the process of building a classification model, in particular the process of determining and / or updating the parameters of the classification model, but is not limited to these.Updating the parameters of a classification model is sometimes referred to as retraining. When training is mentioned herein, it may include retraining. In one embodiment, the training dataset may include at least one image and material information.

[0072] Extracting material data from an image using a data-driven model may include providing the image to the data-driven model. Additionally or alternatively, extracting material data from an image using a data-driven model may include generating image-associated embeddings based on the data-driven model. Embeddings may refer to low-dimensional representations associated with the image, such as feature vectors. Feature vectors may be suitable for suppressing background while preserving material signatures that indicate material data. In this context, background may refer to information independent of material signatures and / or material data. Furthermore, background may refer to information related to biometric features, such as facial features. Material data can be determined using a data-driven model based on image-associated embeddings. Additionally or alternatively, extracting material data from an image by providing the image to a data-driven model may include transforming the image into material data, particularly material feature vectors that indicate material data. Thus, material data may further include material feature vectors, and / or material feature vectors may be used to determine material data.

[0073] The authentication process may be validated based on extracted material data. In one embodiment, validating based on extracted material data may include determining whether the extracted material data corresponds to desired material data. Determining whether the extracted material data matches the desired material data may be referred to as validation. Allowing or denying at least one operation on a device requiring authentication based on the user and / or material data may include validating the authentication or authentication process. Validation may be performed based on material data and / or images. Determining whether the extracted material data corresponds to desired material data may include determining the similarity between the extracted material data and the desired material data. Determining the similarity between the extracted material data and the desired material data may include comparing the extracted material data with the desired material data. The desired material data may refer to predetermined material data. As an example, the desired material data may be skin. It can be determined whether the material data corresponds to the desired material data. In this example, the material data may be a non-skin material or silicone. Determining whether the material data corresponds to the desired material data may include comparing the material data with the desired material data. Comparing material data with desired material data may result in permitting and / or denying the user and / or object from performing at least one operation that requires authentication. In this example, skin as the desired material data may be compared with non-skin material or silicone as material data, and since silicone or non-skin material is different from skin, the result may be denial.

[0074] The authentication process or its validity verification may include generating at least one feature vector from material data and matching the material feature vector with a reference template vector associated with the material.

[0075] In addition to, or instead of using, material data, the method may include the extraction of blood perfusion data. For example, the light beam projected by the projector may be coherent, patterned infrared irradiation. Extracting blood perfusion data may include determining the speckle contrast of the pattern image and determining blood perfusion measurements based on the determined speckle contrast. The speckle contrast can represent an index of the mean contrast of the intensity distribution within an area of ​​the speckle pattern. The speckle contrast K across an area of ​​the speckle pattern is given by the standard deviation σ and the mean speckle intensity. It can be expressed as a ratio of, that is,

number

[0076] Speckle contrast can include speckle contrast values. Speckle contrast values ​​can range from 0 to 1. Blood flow perfusion measurements can be determined based on speckle contrast.

[0077] Blood flow perfusion measurements may depend on the determined speckle contrast. If the speckle contrast changes, the blood flow perfusion measurements derived from the speckle contrast may change accordingly. Blood flow perfusion measurements may be a single numerical value or value that represents the probability that an object is living.

[0078] For example, the entire pattern image can be used to determine the speckle contrast. Alternatively, only a portion of the pattern image can be used to determine the speckle contrast. Preferably, the portion of the pattern image represents an area smaller than the area of ​​the entire pattern image. The portion of the pattern image can be obtained by cropping the pattern image.

[0079] In one embodiment, a data-driven model may be used to determine blood perfusion measurements. The data-driven model may be parameterized and / or trained based on a training dataset. The training dataset may include pattern images and blood perfusion measurements. The data-driven model may be parameterized and / or trained based on the training dataset to output blood perfusion measurements based on the reception of pattern images.

[0080] The authentication process may be validated based on blood perfusion measurements. In one embodiment, validation based on blood perfusion measurements may include determining whether the blood perfusion measurements correspond to human blood perfusion measurements. Determining whether the blood perfusion measurements correspond to humans may be referred to as validation. Allowing or denying a user and / or object to perform at least one operation on a device requiring authentication based on blood perfusion measurements may include validation of authentication or the authentication process. Validation may be based on blood perfusion measurements. Determining whether the blood perfusion measurements correspond to humans may include comparing the blood perfusion measurements with a range of at least one predefined or predetermined blood perfusion measurement values ​​stored, for example, in at least one database. If the extracted blood perfusion measurement values ​​are at least within the tolerance range of the predefined or predetermined blood perfusion measurement values, the authentication is considered validated; otherwise, it is invalid. If authentication is validated, the method may allow the user to perform at least one operation requiring authentication. On the other hand, if authentication is not validated, the method may deny the user to perform at least one operation requiring authentication.

[0081] The method may include, for example, using at least one authorization unit to allow a user to perform an operation on a device that requires authentication based on biometric data. In particular, the method may include at least one authorization step, for example, using at least one authorization unit. The term “authorization step” as used herein is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may refer to, without limitation, a step of assigning a user access rights, in particular to a device and / or to at least one resource of the device, selective authorization or selective restriction of access. The authorization unit may be configured for access control. The term “authorization unit” as used herein is a broad term and should be given in a common and idiomatic sense to those skilled in the art, and should not be limited to any special or customized sense. Specifically, the term may refer to, without limitation, a unit such as a processor configured to authorize a user. The authorization unit may include at least one processor, or may be designed as software or an application. The authorization unit and the authentication unit may be embodied as a single unit, for example, by using the same processor. The authorization unit may be configured to allow the user to perform at least one action on the device (e.g., unlock the device) if authentication is successful, and to deny the user at least one action if authentication fails. This allows the user to be aware of the authentication result. This method may include displaying the authentication result on a display.

[0082] At least one operation on a device requiring authentication may be access to the device (e.g., unlocking the device) and / or access to an application (preferably one associated with the device) and / or access to a portion of the application (preferably one associated with the device). In one embodiment, allowing a user to access a resource may include allowing the user to perform at least one operation on the device and / or system. The resource may be a device, a system, a function of a device, a function of a system and / or an entity. Additionally and / or alternatively, allowing a user to access a resource may include allowing the user to access an entity. The entity may be a physical entity and / or a virtual entity. A virtual entity may be, for example, a database. A physical entity may be an area with restricted access. An area with restricted access may be one of the following: a security area, a room, an apartment, a vehicle, or some of the examples above. The device and / or system may be locked. The device and / or system may only be unlocked by an authorized user.

[0083] All method steps described can be performed using a device. Thus, a single processing device may be configured to exclusively execute at least one computer program, in particular at least one line of computer program code configured to execute at least one algorithm, as used in at least one embodiment of the method according to the present invention. Here, the computer program executed on the single processing device may include all instructions that cause the computer to execute the method. Alternatively, or additionally, at least one method step may be performed by using at least one remote device, in particular at least one selected from at least one server or cloud server, when the device and remote device are part of a computer network. In this case, the computer program may include at least one remote component executed by at least one remote processing device to execute at least one method step. The remote component may have the function of performing user identification and / or material data extraction. Furthermore, the computer program may include at least one interface configured to transfer and / or receive data to at least one remote component of the computer program.

[0084] In a further embodiment, a device for authenticating a user is disclosed.

[0085] This device: - In particular, the device projects multiple light beams onto the user through at least one display, the multiple light beams including a first light beam and a second light beam, and the display is configured to direct the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, with at least one projector, - At least one image generation unit configured to generate a pattern image showing the projection of multiple light beams onto the user; - At least one processor configured to extract biometric data from a pattern image and allow the user to perform operations on a device that requires authentication based on the biometric data, It is equipped with.

[0086] For details, options, and definitions of this device, please refer to the methods described above. Specifically, the device may be configured to perform methods according to the present invention, as described above or below in further detail. Therefore, please refer to further aspects of this disclosure.

[0087] In further embodiments, a computer program is disclosed which, when executed by the device, causes the device to perform the method according to any of the above-described embodiments of the Method. Specifically, the computer program may be stored in a computer-readable data carrier and / or computer-readable storage medium. The computer program may be executed on at least one processor provided in the device. The computer program may generate input data by accessing and / or controlling at least one unit of the device, such as a projector and / or a flood source and / or an image generation unit. Based on the input data, the computer program may generate result data, in particular using an authentication unit.

[0088] As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” may specifically refer to non-temporary data storage means such as hardware storage media on which computer-executable instructions are stored. Stored computer-executable instructions may be associated with computer programs. Specifically, a computer-readable data carrier or storage medium may be, or include, storage media such as random access memory (RAM) and / or read-only memory (ROM).

[0089] Therefore, specifically, one, more, or all of the methods described above, steps a. through e., can be carried out by using a computer or a computer network, preferably by using a computer program.

[0090] Further disclosed and proposed herein are computer program products having program code means for carrying out the methods according to one or more embodiments described herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and / or computer-readable storage medium.

[0091] Further disclosed and proposed herein are data carriers having stored data structures that, after being loaded into a computer or computer network, for example, the working memory or main memory of a computer or computer network, can then perform methods according to one or more embodiments disclosed herein.

[0092] Further disclosed and proposed herein are computer program products having program code means stored in a machine-readable carrier for carrying out methods according to one or more embodiments disclosed herein when the program is executed on a computer or computer network. In this specification, a computer program product means a program as a tradable product. The product can generally exist in any form, such as in paper form or on a computer-readable data carrier and / or computer-readable storage medium. Specifically, computer program products can be distributed via data networks.

[0093] Further disclosed and proposed herein is a non-temporary computer-readable medium containing instructions that, when executed by one or more processors, cause one or more processors to carry out a method according to one or more embodiments disclosed herein.

[0094] Finally, disclosed and proposed herein are modulated data signals containing instructions readable by a computer system or computer network for carrying out methods according to one or more embodiments disclosed herein.

[0095] Referring to the computer implementation aspects of the present invention, one or more method steps, or even all of the method steps, of the methods according to one or more embodiments disclosed herein can be performed using a computer or computer network. Therefore, generally, any method step involving data provision and / or manipulation can be performed using a computer or computer network. Generally, these method steps may include any method steps except those requiring manual intervention, such as specific aspects of sample provision and / or the performance of actual measurements.

[0096] Specifically, further disclosed herein are: - A computer or computer network comprising at least one processor, wherein the processor is adapted to carry out a method according to one of the embodiments described herein, - A computer-loadable data structure adapted to carry out the method according to one of the embodiments described herein while the data structure is being executed on a computer, - A computer program, wherein the computer program is adapted to carry out a method according to one of the embodiments described herein while the program is being executed on a computer, - A computer program that includes programming means for carrying out a method according to one of the embodiments described herein while the computer program is running on a computer or computer network, - A computer program comprising a programming means according to a prior embodiment, wherein the programming means is stored in a computer-readable storage medium, and the computer program is... - A storage medium wherein a data structure is stored in the storage medium and is adapted to carry out a method according to one of the embodiments described herein after the data structure has been loaded into the main storage and / or working storage of a computer or computer network, - A computer program product having program code means, wherein the program code means can be stored in a storage medium or is stored in a storage medium in order to carry out a method according to one of the embodiments described herein when the program code means is executed on a computer or computer network, That is the case.

[0097] The terms “have,” “equip,” and “include,” as used herein, or any grammatical variations thereof, are used in a non-exclusive manner. Thus, these terms can refer to both situations in which the entity described in this context has no further features in addition to the features introduced by these terms, and situations in which one or more further features exist. For example, the expressions “A has B,” “A equips B,” and “A includes B” can refer to both situations in which A has no other elements besides B (i.e., A is composed solely of B), and situations in which entity A has one or more further elements besides B, such as elements C, C and D, or further elements.

[0098] Furthermore, it should be noted that the terms “at least one,” “one or more,” or similar expressions indicating that a feature or element may exist more than once, are typically used only once when introducing each feature or element. In most cases, when referring to each feature or element, the expressions “at least one” or “one or more” are not repeated, regardless of the fact that each feature or element may exist once or more times.

[0099] Furthermore, the terms “preferably,” “more preferably,” “specifically,” “more specifically,” “particularly,” “even more specifically,” or similar terms used herein are used in combination with any feature without limiting the possibility of alternatives. Thus, the features introduced by these terms are any features and are not intended to limit the scope of the claims in any sense. The present invention can also be carried out by using alternative features, as will be recognized by those skilled in the art. Similarly, features introduced by “in one embodiment of the present invention” or similar expressions are intended to be any features, without limiting alternative embodiments of the present invention, without limiting the scope of the present invention, and without limiting the possibility of combining such features with any or non-any other features of the present invention.

[0100] In summary, without prejudice to further possible embodiments, the following embodiments may be conceivable: Embodiment 1. A method for authenticating a device user, wherein the method is: a. A projector projects multiple light beams onto a user through a display, the multiple light beams including a first light beam and a second light beam, and the display directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user; b. A step of generating a pattern image showing the projection of multiple light beams onto the user, c. A step of extracting biological data from the pattern image, d. The step of allowing a user to perform an operation on a device that requires authentication based on the biometric data, Methods that include...

[0101] Embodiment 2. The method according to the prior embodiment, wherein the radiation angle of the projector is selected considering the diffraction characteristics of the display.

[0102] Embodiment 3. The method according to the preceding embodiment, wherein the radiation angle of the projector is selected such that the light beam corresponding to the minor diffraction order from the display matches another minor order or zero order.

[0103] Embodiment 4. The radiation angle of the projector is selected according to the following rules:

number

[0104] Embodiment 5. The method according to any one of the preceding embodiments, wherein the radiation angle of the projector is selected considering a tolerance of 50% or less (≤50%), preferably 40% or less (≤40%), and more preferably 25% or less (≤25%).

[0105] Embodiment 6. The method according to any one of the preceding embodiments, wherein the number of light beams projected onto the user is less than 5000, preferably less than 3000, more preferably less than 2000, even more preferably less than 1500, and most preferably less than 1000.

[0106] Embodiment 7. The method according to any one of the preceding embodiments, wherein the projector projects at least one infrared light pattern, and the projected light beam has wavelengths in the infrared spectral range, preferably 800 nm to 1300 nm, more preferably 900 nm to 1000 nm, and most preferably 1100 nm to 1200 nm.

[0107] Embodiment 8. The projector comprises: at least one laser source such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one isolated-containment heterostructure laser, at least one quantum cascade laser, at least one dispersion Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode-excited laser, at least one dispersion feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface-emitting laser (VCSEL); at least one non-laser source such as at least one LED or at least one light bulb; and at least one emitter selected from the group consisting of at least one edge-emitting laser, according to any one of the preceding embodiments.

[0108] Embodiment 9. The method according to any one of the preceding embodiments, wherein the display is configured to correct the light spot generated by the projector as it traverses the display, and / or the projector includes at least one optical element selected from the group consisting of: at least one lens; at least one microlens array (MLA); at least one diffractive optical element (DOE); and at least one metasurface element.

[0109] Embodiment 10. The method according to any one of the prior embodiments, comprising emitting flood light from at least one flood source and generating at least one flood image while the flood source is emitting the flood light.

[0110] Embodiment 11. The method according to any one of the prior embodiments, wherein the display is at least one organic light-emitting diode (OLED) display and / or at least one quantum dot light-emitting diode (QLED) display, or includes them.

[0111] Embodiment 12. The method according to any one of the preceding embodiments, wherein the display is at least partially transparent in at least one continuous area covering the projector and / or image generation unit, and the display has a transmittance of 20% or less, preferably 15% or less, more preferably 10% or less.

[0112] Embodiment 13. The method according to any one of the preceding embodiments, wherein the intensity of the light beam after projection through the display corresponds to 10% or less (≤10%) of the intensity related to the intensity of the light beam at emission, and the display has a transmission characteristic of 10%, preferably less than 8%, more preferably less than 6%, even more preferably less than 5%, even more preferably less than 4%, even more preferably less than 3%, and most preferably less than 2.5%. Embodiment 14. The method according to any one of the two preceding embodiments, wherein a partially transparent continuous area is associated with a first pixel density value (pixels per inch (PPI)), and further areas of the display are associated with a second pixel density value, the display having a first pixel density value of 450 PPI or less, preferably 300 to 440 PPI, more preferably 350 to 450 PPI, the first pixel density value is constant across the continuous area with a maximum deviation of 20%, preferably 10%, and the second pixel density value is 400 to 500 PPI, preferably 450 to 500 PPI.

[0113] Embodiment 15. The method according to any one of the prior embodiments, wherein the biological data includes blood perfusion data and / or material data.

[0114] Embodiment 16. The method according to any one of the prior embodiments, wherein the extraction of the biological data includes the extraction of material data and / or the extraction of blood perfusion data.

[0115] Embodiment 17. The method of the preceding embodiment, wherein material data extraction includes providing the pattern image to a model and / or receiving material data from the model.

[0116] Embodiment 18. The method according to any one of the two preceding embodiments, wherein the extraction of blood perfusion data comprises determining the speckle contrast of the pattern image and determining a blood flow measurement based on the determined speckle contrast, the speckle contrast representing a measurement of the average contrast of the intensity distribution within an area of ​​the speckle pattern.

[0117] Embodiment 19. The method according to any one of the preceding embodiments, wherein the device is selected from the group consisting of: television devices; game consoles; personal computers; mobile devices, particularly mobile phones, and / or smartphones, and / or tablet computers, and / or laptop computers, and / or tablets, and / or virtual reality devices, and / or wearables such as smartwatches; or other types of portable computers.

[0118] Embodiment 20. The method according to any one of the prior embodiments, wherein the method is implemented on a computer.

[0119] Embodiment 21. A device for authenticating a user, wherein the device is: - At least one projector configured to project multiple light beams onto a user through at least one display, wherein the multiple light beams include a first light beam and a second light beam, and the display directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, - At least one image generation unit configured to generate a pattern image showing the projection of multiple light beams onto the user; - At least one processor configured to extract biometric data from the pattern image and to allow the user to perform operations on a device that requires authentication based on the biometric data, A device equipped with the following features.

[0120] Embodiment 22. The device according to the preceding embodiment, wherein the device includes a display configured to direct a light beam to illuminate at least partially overlapping areas of the user.

[0121] Embodiment 23. The device described in one of the preceding embodiments that reference the device, wherein the device is configured to perform the method described in one of the preceding embodiments that reference the method.

[0122] Embodiment 24. A computer program including instructions that, when executed by a device described in any one of the prior embodiments that reference the Method, causes the device to perform the method described in any one of the prior embodiments that reference the Method.

[0123] Embodiment 25. A computer-readable storage medium containing instructions, which, when the instructions are performed by a device described in any one of the preceding embodiments that reference the device, causes the device to perform a method described in any one of the preceding embodiments that reference the method.

[0124] Embodiment 26. A non-temporary computer-readable medium containing instructions that, when executed by one or more processors, causes the one or more processors to perform the method described in any one of the preceding embodiments that reference the method. [Brief explanation of the drawing]

[0125] Further optional features and embodiments are disclosed in more detail in the description following the embodiments, preferably in conjunction with dependent claims, where each optional feature may be implemented in independent forms as well as in any feasible combination, as will be understood by those skilled in the art. The scope of the present invention is not limited by preferred embodiments. Embodiments are schematically shown in the figures, where the same reference numerals in these figures refer to the same or functionally equivalent elements.

[0126] In the diagram: [Figure 1] This figure shows an embodiment of the device according to the present invention. [Figure 2] Figures 2A and 2B show exemplary displays, their diffraction characteristics, and projector radiation angles. [Figure 3] Figures 3A-3C show exemplary light patterns (3A), diffraction patterns (3B), and resulting patterns in a user. [Figure 4] This is a flowchart illustrating an exemplary embodiment of a method for authenticating a device user. [Modes for carrying out the invention]

[0127] Detailed description of the embodiment Figure 1 shows a very schematic embodiment of the user authentication device 110 of the present invention.

[0128] For example, device 110 may be selected from the group consisting of: television devices; game consoles; personal computers; mobile devices, especially mobile phones and / or smartphones, and / or tablet computers, and / or laptops and / or tablets, and / or virtual reality devices and / or wearables such as smartwatches; or other types of portable computers.

[0129] Device 110 is: - A device 110 includes at least one projector 112 configured to project multiple light beams onto the user through at least one display 114 of the device 110, wherein the multiple light beams include a first light beam and a second light beam, and the display directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user. - At least one image generation unit 116 configured to generate a pattern image showing the projection of multiple light beams onto the user; - At least one processor 118 configured to extract biometric data from a pattern image and allow a user to perform an operation on a device 110 that requires authentication based on biometric data, It is equipped with.

[0130] The projector 112 may be configured to generate and / or provide at least one light pattern, in particular at least one infrared light pattern. The light pattern may be an infrared light pattern. Specifically, the projector 112 may be configured to project at least one infrared light pattern, and the projected light beam has wavelengths in the infrared spectral range, preferably 800 nm to 1300 nm, more preferably 900 nm to 1000 nm, and most preferably 1100 nm to 1200 nm.

[0131] The light projected by the projector 112 may be coherent. The light pattern may be a coherent, particularly infrared, light pattern. The projector 112 may be configured to emit light of a single wavelength, for example, a single wavelength in the near-infrared region. In other embodiments, the projector 112 may be adapted to emit light at multiple wavelengths, for example, to allow for additional measurements in other wavelength channels.

[0132] The light pattern may include regular and / or constant and / or periodic patterns such as triangular patterns, rectangular patterns, hexagonal patterns, or patterns including convex tiling. For example, the light pattern is a hexagonal pattern, preferably a hexagonal light pattern, and preferably a 2 / 5 hexagonal infrared light pattern. Using a periodic 2 / 5 hexagonal pattern makes it possible to distinguish noise from usable signals.

[0133] The light pattern may include at least one dot pattern. The light pattern has a low dot density. For example, the number of light beams projected onto the user is less than 5000, preferably less than 3000, more preferably less than 2000, even more preferably less than 1500, and most preferably less than 1000.

[0134] Projector 112 may include at least one emitter, in particular multiple emitters. The emitters may be selected from the group consisting of: at least one laser source such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separated-containment heterostructure laser, at least one quantum cascade laser, at least one dispersion Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode-excited laser, at least one dispersion feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface-emitting laser (VCSEL); at least one non-laser source such as at least one LED or at least one light bulb; and at least one edge-emitting laser.

[0135] The display 114 may be configured to correct spots as the spots generated by the projector 112 traverse the display 114, for example by increasing the number of spots. For example, the display 114 can function as a diffractive optical element (DOE). This saves resources because the display functions as a DOE, eliminating the need for additional DOEs. Furthermore, since the light beam is amplified by the display 114, fewer emitters, such as VCSEL cavities, are required. Additionally or alternatively, the projector 112 may include at least one optical element selected from the group consisting of: at least one lens; at least one microlens array (MLA); at least one diffractive optical element (DOE); and at least one metasurface element, configured to correct spots, for example by increasing the number of spots. The DOE and / or metasurface element may be configured to generate multiple light beams from a single incident light beam. For example, an optical element including a VCSEL projecting up to 2000 spots and multiple metasurface elements can be used to duplicate the number of spots. Further configurations are possible, in particular, including a different number of projected VCSELs and / or further configurations including at least one different optical element configured to increase the number of spots. Other multiplication factors are also possible. For example, one or more VCSELs may be used, and the generated laser spots may be replicated by using at least one DOE.

[0136] The device further comprises at least one flood light source 120 configured to emit flood light. The image generation unit 116 may be configured to generate at least one flood image while the flood light source 120 is emitting flood light. The flood light may have wavelengths in the infrared range, particularly in the near-infrared range. The flood light source 120 may include at least one LED or at least one VCSEL, preferably multiple VCSELs. The emission of flood light and the illumination of the light pattern may be performed consecutively or in a time that at least partially overlaps. For example, the flood light and the light pattern may be emitted simultaneously. For example, one of the flood light or the light pattern may be emitted at a lower intensity than the other.

[0137] The display 114, specifically the display panel, may be or comprise at least one organic light-emitting diode (OLED) display and / or at least one quantum dot light-emitting diode (QLED).

[0138] The display 114 may be at least partially transparent. The display 114 may be at least partially transparent in at least one continuous area covering the projector, flood source, and / or image generation unit. The transmittance of the display 114 may be 20% or less, preferably 15% or less, and more preferably 10% or less. The intensity of the light beam after projection through the display 114 may be 10% or less (≤10%) of the intensity of the light beam at the time of emission. The display 114 may have a transmittance of 10%. Preferably, the transmittance is less than 8%, more preferably less than 6%, even more preferably less than 5%, even more preferably less than 4%, even more preferably less than 3%, and most preferably less than 2.5%.

[0139] Display 114 in at least one contiguous area: - The light pattern incident on a continuous area while being projected from the projector 112 traverses the display 114; - Flood light incident on a continuous area while being irradiated from the flood source 120 traverses the display 114; - User light, generated by the light pattern and / or flood light shining on the user, incident on a continuous area, traverses the display 114 to collide with the image generation unit 116. It may be at least partially transparent so as to satisfy at least one of the following conditions.

[0140] For example, the display 114 may be semi-transparent in the near-infrared region. For example, the display 114 may have a transmittance of 20% to 50% in the near-infrared region. The display 114 may have different transmittances for different wavelength ranges. The present invention may propose a device 110 comprising an image generation unit 116 and a projector 112 that can be positioned behind the display 114 of the device. The transparent region of the display 114 allows the operation of the image generation unit 116 and the projector 112 behind the display 114.

[0141] The projector 112 is configured to guide a first light beam and a second light beam to illuminate an area of ​​the user that at least partially overlaps with the display 114. For example, the overlap is at least 10%, preferably at least 20%, and more preferably at least 50%. Higher overlaps (e.g., 100%) are also possible. For example, a first light spot (e.g., a light spot with a first radius r1) can illuminate a first area of ​​the user, and a second light spot with a second radius r2 (r1 ≤ r2) can illuminate a second area of ​​the user. For example, the first and second areas may be the same or different. For example, the first radius may be smaller than the second radius. For example, the first and second areas may be offset from each other. Any combination of these examples is possible. Because at least two light spots partially overlap, the intensity of the overlapping area is increased. This enables accurate, reliable, and secure authentication even behind a display with low transmittance. An example of light spots that at least partially overlap is shown in Figure 3C.

[0142] The display 114 can have diffraction characteristics. The display 114 can function as a grating having a periodic structure that diffracts the incident light from the projector 112 into a plurality of beams traveling in different directions, i.e., at different diffraction angles. FIG. 2A shows an exemplary display 114, where the incident light beam is diffracted at diffraction angles θ OLED,n (in this example n = 1, 2, 3). The diffraction angle can depend on the structure of the display 114 and the wavelength of the incident light. For example, the diffraction of the display 114 is: [Equation 1] which can be described by, where n is an integer, λ is the wavelength, d is the grating pitch, Θ OLED,n is the nth order diffraction angle and the optical axis (0th order). The pattern design of the projector 112 can be selected considering the diffraction characteristics of the display 114. FIG. 2B shows exemplary radiation angles θ proj,i (in this example i = 1, 2, 3) of the projector 112. In particular, the radiation angles of the projector 112 can be selected considering the diffraction characteristics of the display 114 such that the light beams corresponding to the minor diffraction orders from the display 114 coincide with other minor orders or the zero order.

[0143] The radiation angles of the projector 112 can be selected according to the following rule, [Equation 2] [Equation 3] where k is a natural number, θ proj,n is the radiation angle of the projector 112, and θ OLED,1 is the non - zero minimum diffraction angle. The distance d can refer to the distance between pixels in one dimension of the display 114, particularly the OLED display 114. Thus, the distance d can define the mesh size of the periodic (pixel) grid.

[0144] The radiation angle of the projector 112 can be selected by taking into account possible deviations from an ideal periodic pixel grid, particularly by using the rules described above. The relationship between the pixel arrangements of the display 114, defined by the distance d between at least two pixels in at least one dimension, can have a deviation of up to 50%, preferably up to 40%, and more preferably up to 25%. Thus, the radiation angle θ of the projector proj,n This can be selected considering a tolerance of up to 50%, preferably up to 40%, and more preferably up to 25%.

[0145] By using this rule, it becomes possible to match the minor diffraction order with the new or original (zero-order) light beam of projector 112.

[0146] Figure 3A shows an exemplary light pattern generated by the projector 112. Figure 3B shows an exemplary diffraction pattern of the display 114. Figure 3C shows the resulting pattern obtained at the user's end. In this example, primary and tertiary spots fall together, and secondary spots fall together. This can lead to the convergence of light spots, which can reduce the projector's radiant energy loss. Another advantage is that noise can be removed or utilized by adjusting the alignment of the projector and display so that nth-order spots converge at a single spatial position. This enables accurate, reliable, and secure authentication even behind displays with low transmittance.

[0147] The projector 112 can have an emitter with an inherently narrow radiation profile. For example, a combination of a VCSEL and optical elements such as an MLA, DOE, metasurface, or lens may be used. Such a configuration can have a narrower radiation profile than commercially available LEDs.

[0148] Returning to Figure 1, the image generation unit 116 may comprise at least one optical sensor, in particular at least one pixelated optical sensor. The image generation unit may comprise at least one CMOS sensor or at least one CCD chip. For example, the image generation unit 116 may comprise at least one CMOS sensor that may be sensitive to the infrared spectral range. The image may consist of raw image data or it may be a pre-processed image. For example, pre-processing may include applying at least one filter to the raw image data, and / or at least one background correction, and / or at least one background removal.

[0149] For example, the image generation unit 116 may include one or more of the following: at least one monochrome camera (e.g., including monochrome pixels), at least one color (e.g., RGB) camera (e.g., including color pixels), and at least one IR camera. The cameras may be CMOS cameras. The camera may include at least one monochrome camera chip (e.g., a CMOS chip). The camera may include at least one color camera chip (e.g., an RGB CMOS chip). The camera may include at least one IR camera chip (e.g., an IR CMOS chip). For example, the camera may include monochrome (e.g., black and white) pixels and color pixels. Color pixels and monochrome pixels can be combined within the camera. The camera may generally include a one-dimensional or two-dimensional array of image sensors such as pixels. For example, the camera may be an internal camera and / or an external camera of the device 110. As described above, the internal camera and / or external camera of the device may be accessed via a hardware and / or software interface used as the image generation unit 116. If device 110 is a smartphone, or includes a smartphone, the image generation unit may be the smartphone's front camera and / or back camera, such as the selfie camera.

[0150] The image generation unit 116 may have a field of view in the range of 10°×10° to 75°×75°, preferably in the range of 55°×65°. The image generation unit 116 may have a resolution of less than 2MP, preferably in the range of 0.3MP to 1.5MP.

[0151] The image generation unit 116 may include further elements such as one or more optical elements (e.g., one or more lenses). For example, the light sensor may be a fixed-focus camera having at least one lens fixedly tuned to the camera. Alternatively, the camera may have one or more variable lenses that are automatically or manually adjustable. The camera may have at least one optical filter, for example, at least one bandpass filter. The bandpass filter may be matched to the spectrum of the light emitter. However, other cameras are also possible.

[0152] A pattern image may include an image showing at least a portion of the user, particularly the user's face, while the user is illuminated by a light pattern, particularly each area of ​​interest included in the image. A pattern image can be generated by imaging and / or recording light reflected by an object and / or the user illuminated by an infrared light pattern. A pattern image showing the user may include at least a portion of the light pattern illuminated by at least a portion of the user. For example, projection by projector 112 and imaging by image generation unit 116 can be synchronized, for example, by using at least one control unit of device 110. A flood image may include an image showing the user, particularly the user's face, while the user is illuminated by flood light. A flood image can be generated by imaging and / or recording light reflected by an object and / or the user illuminated by flood light. A flood image showing the user may include at least a portion of the flood light on at least a portion of the user. For example, illumination by flood light source 120 and imaging by image generation unit 116 can be synchronized, for example, by using at least one control unit of device 110.

[0153] The image generation unit 116 may be configured to image and / or record the pattern image and the flood image simultaneously or at different timings. The image generation unit 116 may be configured to image and / or record the pattern image and the flood image in at least partially overlapping measurement areas or areas corresponding to measurement areas.

[0154] The device 110 may be configured to authenticate the user of the device 110 in order to perform at least one operation on a device requiring authentication. The device 110 may include at least one authentication unit (e.g., processor 118) configured to perform at least one user authentication process, particularly using flood images and pattern images. The authentication unit may be configured to use a facial recognition process that operates on flood images, pattern images, and / or extracted biometric data, particularly extracted biometric data derived from pattern images. The authentication unit may be at least one processor, or may include at least one processor, and / or may be designed as software or an application.

[0155] For example, an authentication unit can perform at least one face detection using a flood image. Face detection may be performed locally on the device. However, face identification, i.e., assigning an ID (identity) to the detected face, may be performed remotely, such as in the cloud, especially if identification, not just verification, is required. User templates can be stored on a remote device such as the cloud, eliminating the need for local storage. This can be advantageous in terms of storage capacity and security.

[0156] The authentication unit may be configured to identify users based on flood images. Therefore, in particular, the authentication unit can transfer data to a remote device. Alternatively or additionally, the authentication unit may perform user identification based on flood images, in particular by running appropriate computer programs having their respective functions. Identification includes assigning an identity to the detected face, and / or performing at least one identity check, and / or verifying the user's identity.

[0157] For example, the authentication process may include performing at least one face detection step. The face detection step may include analyzing the flood image. Furthermore, for example, the authentication process may include identification. Identification may include assigning an identity to the detected face and / or performing at least one identity verification and / or verifying the user's identity. Identification may include performing face verification to determine that the imaged face is the user's face. User identification may include matching the template with the flood image, for example, showing the contours of part of the user, particularly the contours of part of the face. User identification may include determining whether the imaged face is the user's face, and in particular whether the imaged face corresponds to an image of the user's face stored, for example, in at least one memory of the device. If the flood image does not match the image template, authentication may fail.

[0158] Analysis of flood images may include determining multiple facial features. Analysis may include comparing, in particular matching, the determined facial features with template features. Template features may be features extracted from at least one template. A template may be, or include, at least one image generated during the registration process (e.g., during device initialization). A template may be an image of an authorized user. Template features and / or facial features may include vectors. Feature matching may include determining the distance between vectors. User identification may include comparing the distance between vectors to at least one predefined threshold value, and user identification is successful if the distance is at least within an acceptable range and less than or equal to the predefined threshold value. Otherwise, the user is declining and / or rejected.

[0159] For example, image recognition may include using a trained model that includes at least one model, particularly at least one face recognition model. Flood image analysis can be performed by using a face recognition system such as FaceNet, as described, for example, Florian Schroff, Dmitry Kalenichenko, James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering" arXiv:1503.03832. The trained model may include at least one convolutional neural network. For example, a convolutional neural network may be designed as described in MD Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," CoRR, abs / 1311.2901, 2013, or C. Szegedy et al., "Going deeper with convolutions," CoRR, abs / 1409.4842, 2014. For more information on convolutional neural networks for face recognition systems, see Florian Schroff, Dmitry Kalenichenko, and James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," arXiv:1503.03832. Labeled image data from an image database can be used as training data.Specifically, labeled faces can be obtained from one or more of the following: the YouTube® Faces database, as described in "Labeled faces in the wild: A database for studying face recognition in unconstrained environments" by GB Huang, M. Ramesh, T. Berg, and E. Learned-Miller, Technical Report 07-49, University of Massachusetts, Amherst, October 2007; or in "Face recognition in unconstrained videos with matched background similarity" by Wolf, T. Hassner, and I. Maoz at the IEEE Conf. at CVPR in 2011; or the Google® Facial Expression Comparison dataset. Training of a convolutional neural network can be carried out as described in Florian Schroff, Dmitry Kalenichenko, and James Philbin, "FaceNet: A Unified Embedding for Face Recognition and Clustering," arXiv:1503.03832.

[0160] Figure 4 is a flowchart illustrating an exemplary embodiment of a method for authenticating the user of a device (specifically device 110 as described in Figures 1-4B). The method steps may be performed in a predetermined order or in a different order. Furthermore, there may be one or more additional method steps that are not listed. In addition, one, more than one, or even all of the method steps may be repeated.

[0161] Book: a. A projector 112 projects multiple light beams onto the user through a display 114, wherein the multiple light beams include a first light beam and a second light beam, and the display 114 directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, step (130) b. A step (132) of generating a pattern image showing the projection of the multiple light beams onto the user, c. A step (134) of extracting biological data from the pattern image, d. A step (136) that allows the user to perform an operation on a device that requires authentication based on the biometric data, Includes.

[0162] This method may be implemented by a computer.

[0163] Biometric data may be data that enables the distinction between living humans, particularly users, and non-living materials such as paper or 3D face masks. Biometric data may include blood perfusion data and / or material data. Extraction of biometric data may include extraction of material data and / or blood perfusion data. Biometric data may include information about the material of the user's surface onto which the spot is projected. Biometric data may include information about at least one vital sign.

[0164] This method may include, for example, extracting material data from a pattern image by beam profile analysis of a light spot using an authentication unit. For beam profile analysis, see WO2018 / 091649A1, WO2018 / 091638A1, and WO2018 / 091640A1, the full contents of which are included by reference. Extracting material data from a pattern image may include generating material types and / or generating data derived from material types. Preferably, the extraction of material data may be based on a pattern image. Material data may be extracted by using at least one model. Extracting material data may include providing a pattern image to a model and / or receiving material data from a model.

[0165] The authentication process may be validated based on extracted material data. In one embodiment, validating based on extracted material data may include determining whether the extracted material data corresponds to desired material data. Determining whether the extracted material data matches the desired material data may be referred to as validation. Allowing or denying at least one operation on a device requiring authentication based on the user and / or material data may include validating the authentication or authentication process. Validation may be performed based on material data and / or images. Determining whether the extracted material data corresponds to desired material data may include determining the similarity between the extracted material data and the desired material data. Determining the similarity between the extracted material data and the desired material data may include comparing the extracted material data with the desired material data. The desired material data may refer to predetermined material data. As an example, the desired material data may be skin. It can be determined whether the material data corresponds to the desired material data. In this example, the material data may be a non-skin material or silicone. Determining whether the material data corresponds to the desired material data may include comparing the material data with the desired material data. Comparing material data with desired material data may result in permitting and / or denying the user and / or object from performing at least one operation that requires authentication. In this example, skin as the desired material data may be compared with non-skin material or silicone as material data, and since silicone or non-skin material is different from skin, the result may be denial.

[0166] In addition to, or instead of using, material data, the method may include the extraction of hemoperfusion data. For example, the light beam projected by the projector may be coherent, patterned infrared irradiation. Extracting hemoperfusion data may include determining the speckle contrast of the patterned image and determining a hemoperfusion measurement based on the determined speckle contrast. The hemoperfusion measurement may depend on the determined speckle contrast. If the speckle contrast changes, the hemoperfusion measurement derived from the speckle contrast may change accordingly. The hemoperfusion measurement may be a single numerical value or value representing the probability that the object is living.

[0167] For example, the entire pattern image can be used to determine the speckle contrast. Alternatively, only a portion of the pattern image can be used to determine the speckle contrast. Preferably, the portion of the pattern image represents an area smaller than the area of ​​the entire pattern image. The portion of the pattern image can be obtained by cropping the pattern image.

[0168] In one embodiment, a data-driven model may be used to determine blood perfusion measurements.

[0169] The authentication process may be validated based on blood perfusion measurements. In one embodiment, validation based on blood perfusion measurements may include determining whether the blood perfusion measurements correspond to human blood perfusion measurements. Determining whether the blood perfusion measurements correspond to humans may be referred to as validation. Allowing or denying a user and / or object to perform at least one operation on a device that requires authentication based on blood perfusion measurements may include validating the authentication or authentication process. Validation may be based on blood perfusion measurements. Determining whether the blood perfusion measurements correspond to humans may include comparing the blood perfusion measurements with a range of at least one predefined or predetermined blood perfusion measurement values ​​stored, for example, in at least one database. If the extracted blood perfusion measurement value is at least within an acceptable range within the predefined or predetermined blood perfusion measurement value range, the authentication is considered validated; otherwise, it is invalid. If authentication is validated, the method may allow the user to perform at least one operation that requires authentication. On the other hand, if authentication is not validated, the method may deny the user to perform at least one operation that requires authentication. The method includes enabling a user to perform an operation on a device 110 that requires authentication based on biometric data, for example, by using at least one authorization unit 136. In particular, the method may include at least one authorization step, for example, by using at least one authorization unit. The authorization unit may be configured for access control. The authorization unit may include at least one processor, or may be designed as software or an application. The authorization unit and the authentication unit may be implemented as a single unit, for example, by using the same processor.The authorization unit may be configured to allow the user to perform at least one operation on the device (e.g., unlock device 110) if authentication is successful, and to deny the user to perform at least one operation on device 110 if authentication fails. This allows the user to be aware of the authentication result. This method may include displaying the authentication result on display 114.

[0170] At least one operation on a device requiring authentication may be access to the device (e.g., unlocking device 110), and / or access to an application (preferably one associated with device 110), and / or access to a portion of the application (preferably one associated with device 110). In one embodiment, allowing a user to access a resource may include allowing the user to perform at least one operation on the device and / or system. The resource may be a device, a system, a function of a device, a function of a system, and / or an entity. Additionally and / or alternatively, allowing a user to access a resource may include allowing the user to access an entity. The entity may be a physical entity and / or a virtual entity. A virtual entity may be, for example, a database. A physical entity may be an area with restricted access. An area with restricted access may be one of the following: a security area, a room, an apartment, a vehicle, and / or parts of the examples above. The device and / or system may be locked. The device and / or system may only be unlocked by an authorized user.

[0171] List of reference numbers 110 devices 112 Projectors 114 displays 116 Image Generation Unit 118 processors 120 Flood Irradiation Source 130 Projection 132 generation 134 Extracts 136 Permission

Claims

1. A method for authenticating a user of a device (110), wherein the method is: a. A projector (112) projects multiple light beams onto a user through a display (114), the multiple light beams including a first light beam and a second light beam, and the display (114) directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, step (130) b. A step (132) of generating a pattern image showing the projection of multiple light beams onto the user, c. Step (134) of extracting biological data from the pattern image, d. A step (136) of allowing the user to perform an operation on a device (110) that requires authentication based on the biometric data, Methods that include...

2. The method according to claim 1, wherein step a includes projecting a light pattern comprising a plurality of light spots, and the pattern image generated in step b is generated by an image generation unit (116) while the light pattern is being projected.

3. The method according to claim 1 or 2, wherein the radiation angle of the projector (112) is selected considering the diffraction characteristics of the display (114).

4. The method according to claim 3, wherein the radiation angle of the projector (112) is selected such that the light beam corresponding to a minor diffraction order from the display (114) matches another minor order or zero order, the minor diffraction order includes diffraction orders that are not equal to zero, and the minor diffraction orders are ±1st order, ±2nd order, ±3rd order diffractions.

5. The radiation angle of the projector (112) is selected according to the following rules: [Math 1] Here, n is an integer, k is a natural number, λ is the wavelength, d is the distance between pixels in one dimension of the display (114), and θ proj,n The method according to claim 1 or 2, wherein is the radiation angle of the projector (112).

6. The method according to claim 1 or 2, wherein the radiation angle of the projector (112) is selected considering a tolerance of 50% or less (≤50%), preferably 40% or less (≤40%), and more preferably 25% or less (≤25%).

7. The projector (112) comprises: at least one laser source such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separation-containment heterostructure laser, at least one quantum cascade laser, at least one dispersion Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode-excited laser, at least one dispersion feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface-emitting laser (VCSEL); at least one non-laser source such as at least one LED or at least one light bulb; and at least one emitter selected from the group consisting of at least one edge-emitting laser.

8. The method according to claim 1 or 2, wherein the display (114) is configured to correct the light spot generated by the projector (112) as it crosses the display, and / or the projector (112) includes at least one optical element selected from the group consisting of: at least one lens; at least one microlens array (MLA); at least one diffractive optical element (DOE); and at least one metasurface element.

9. The method according to claim 1 or 2, wherein the display (114) is at least partially transparent in at least one continuous area covering the projector (112) and / or image generation unit (116), and the display (114) has a transmittance of 10%, preferably less than 8%, more preferably less than 6%, even more preferably less than 5%, even more preferably less than 4%, even more preferably less than 3%, and most preferably less than 2.5%.

10. The method according to claim 1 or 2, wherein the number of light beams projected onto the user is less than 5,000, preferably less than 3,000, more preferably less than 2,000, even more preferably less than 1,500, and most preferably less than 1,000.

11. The method according to claim 1 or 2, wherein the projector (112) projects at least one infrared light pattern, and the projected light beam has wavelengths in the infrared spectral range, preferably 800 nm to 1300 nm, more preferably 900 nm to 1000 nm, and most preferably 1100 nm to 1200 nm.

12. A device (110) for authenticating a user, wherein the device (110): - At least one projector (112) configured to project a plurality of light beams onto a user through at least one display (114), wherein the plurality of light beams include a first light beam and a second light beam, and the display (114) directs the first light beam and the second light beam to illuminate at least partially overlapping areas of the user, - At least one image generation unit (116) configured to generate a pattern image showing the projection of multiple light beams onto the user; - At least one processor (118) configured to extract biometric data from the pattern image and allow the user to perform operations on a device (110) that requires authentication based on the biometric data, A device (110) comprising:

13. A computer program including instructions that, when executed by a device (110) according to claim 1 or 2 which the program references a device, causes the device (110) to perform the method according to claim 1 or 2 which references a method.

14. A computer-readable storage medium containing instructions, which, when executed by the device (110) according to claim 1 or 2 which references a device, causes the device (110) to perform the method according to claim 1 or 2 which references a method.

15. A non-temporary computer-readable medium containing instructions that, when executed by one or more processors, cause one or more processors to perform the method according to claim 1 or 2, which references the method.

Images