Enhanced Material Detection with Multi-Wavelength Projectors

The detector system uses a multi-wavelength approach with beam profile analysis and machine learning to reliably distinguish between skin and non-skin objects, addressing the challenge of spoofing attacks in security applications.

JP2024542985APending Publication Date: 2024-11-19TRINAMIX GMBH
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2024525068
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-10-26
Filing Date
2022-10-25
Publication Date
2024-11-19

Smart Images

  • Figure 2024542985000001_ABST
    Figure 2024542985000001_ABST
Patent Text Reader

Abstract

A detector (110) for material detection of at least one object (112) is disclosed, the detector (110) comprising: - at least one projector (116) for illuminating at least one object (112) with at least one illumination pattern, the illumination pattern including a plurality of illumination features, the illumination features having a first wavelength; - at least one flood light source (122) configured for scene illumination, the flood light source (122) configured to emit scene illumination having a second wavelength different from the first wavelength; at least one sensor element (128) having a matrix of optical sensors (132), each having a light-sensitive area, each optical sensor (132) designed to generate at least one sensor signal in response to illumination of its respective light-sensitive area by a light beam propagating from the object (112) to the detector (110); the sensor element (128) is configured to image at least one reflected image comprising a plurality of reflected features produced by the object (112) in response to the illumination pattern, each of the reflected features having a beam profile (134); at least one sensor element configured to image at least one scene image of the object illuminated by the scene illumination; at least one evaluation device (136), the evaluation device (136) is configured to determine at least one first material information of the object (112) by evaluating the beam profile (134) of at least one of the reflection features; the evaluation device (136) is configured to determine at least one second material information of the object (112) by evaluating the scene image; at least one evaluation device (136) configured to determine a material of the object (112) using the first material information and the second material information; Equipped with.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The invention relates to a detector for material detection of at least one object, to a method for material detection of at least one object and to various uses of the detector. The device, the method and the use according to the invention can in particular be employed in various fields, for example in everyday life, security technology, gaming, traffic technology, production technology, art, photography, such as digital photography or videography for documentary or technical purposes, safety technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology or science, although other applications are also possible. [Background technology]

[0002] Methods and devices for material detection are generally known. A reliable technique for material detection uses beam profile analysis, as described in WO2020 / 187719, the contents of which are incorporated by reference.

[0003] WO2020 / 187719 describes a detector for identifying at least one material characteristic m. The detector comprises at least one sensor element comprising a matrix of light sensors. The light sensors each have a light-sensitive area. The sensor elements are configured to record at least one reflected image of a light beam emanating from at least one object. The detector comprises at least one evaluation device configured to determine the material characteristic by evaluation of at least one beam profile of the reflected image. The evaluation device is configured to determine at least one distance characteristic by applying at least one distance-dependent image filter to the reflected image.

[0004] US 2020 / 311448 A1 describes a method that includes receiving, in one or more processing devices, data corresponding to a first image, and determining, based on the received data, by the one or more processing devices, that a first set of pixel values ​​of the first image correspond to illumination at a first representative wavelength and at least a second set of pixel values ​​of the first image correspond to illumination at a second representative wavelength. The illuminations at the first and second representative wavelengths constitute at least a part of a first illumination sequence pattern used in capturing the first image. The method also includes determining that the first illumination sequence pattern matches a second illumination sequence pattern associated with a device on which the first image is expected to be received, and in response thereto, initiating a biometric authentication process to authenticate a subject represented in the first image.

[0005] US2017 / 161557A9 describes a system, apparatus, and method for authenticating an individual or user using biometric features. The system for authenticating a user through identification of at least one biometric feature may include an active light source capable of emitting electromagnetic radiation having a peak emission wavelength of about 700 nm to about 1200 nm, the active light source being positioned to emit the electromagnetic radiation that impinges on at least one biometric feature of the user, and an image sensor having infrared light capture pixels positioned relative to the active light source to receive and detect the electromagnetic radiation upon reflection from the at least one biometric feature of the user. The system may further include an authentication module operatively coupled to the image sensor and operable to generate an electronic representation of the at least one biometric feature of the user from the detected electromagnetic radiation, and an authentication module operatively coupled to the processing module operatively operable to receive and compare the electronic representation to an authenticated standard of the at least one biometric feature of the user to provide authentication of the user.

[0006] However, for challenging targets such as distinguishing between skin and non-skin objects, reliable material detection remains a challenge, for example for security applications such as unlocking and / or accessing mobile devices, particularly one or more of a television device, a mobile phone, a smart phone, a gaming console, a tablet computer, a personal computer, a laptop, a tablet, a virtual reality device, or other types of portable computers. In particular, reliable detection of spoofing attacks using highly realistic 3D silicon masks remains challenging. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] WO2020 / 187719 [Patent Document 2] US2020 / 311448A1 [Patent Document 3] US2017 / 161557A9 Summary of the Invention [Problem to be solved by the invention]

[0008] It is therefore an object of the present invention to provide an apparatus and method for the above-mentioned technical problems of known apparatus and methods, in particular to provide an apparatus and method that allows reliable material detection, even in the case of difficult targets, preferably with low technical effort and with low demands in terms of technical resources and costs. [Means for solving the problem]

[0009] This problem is solved by the invention with the features of the independent patent claims. Advantageous developments of the invention, which can be realized individually or in combination, are set out in the dependent claims and / or in the following description and detailed embodiments.

[0010] In a first aspect of the present invention, a detector for material detection of at least one object is disclosed.

[0011] As used herein, a "detector" may generally refer to a device adapted to provide at least one item of information regarding the material of at least one object. The detector may be a fixed or mobile device. The detector may be a standalone device or may form part of another device, such as a computer, a vehicle, or other device. Furthermore, the detector may be a handheld device. For example, the detector may be a mobile device selected from the group consisting of a television device, a mobile phone, a smart phone, a game console, a tablet computer, a personal computer, a laptop, a tablet, a virtual reality device, or another type of portable computer. Other embodiments of the detector are possible.

[0012] An "object" may generally be any object selected from living and non-living objects. Thus, by way of example, at least one object may include one or more articles and / or one or more parts of an article. Additionally or alternatively, an object may be or include one or more living organisms and / or one or more parts thereof, such as a human being, e.g., a user, and / or one or more body parts of an animal.

[0013] As used herein, the term "material detection" may refer to determining at least one any material property that characterizes the material of an object. As used herein, the term "material property" refers to at least one any property of a material configured for characterization and / or identification and / or classification of the material. For example, the material property may be a property selected from the group consisting of reflectance, penetration depth of light into the material, roughness, specular reflectance, diffuse reflectance, surface properties, a measure of translucency, scattering, specifically backscattering behavior, and the like. For example, the at least one material property may be a property selected from the group consisting of scattering coefficient, translucency, transparency, deviation from Lambertian surface reflection, speckle, and the like.

[0014] The detector is: - at least one projector for illuminating at least one object with at least one illumination pattern, the illumination pattern including a plurality of illumination features, the illumination features having a first wavelength; and - at least one flood light source configured for scene illumination, the flood light source configured to emit scene illumination having a second wavelength different from the first wavelength; - comprising a matrix of light sensors, each of said light sensors being at least one sensor element having a light-sensitive area, each light sensor being designed to generate at least one sensor signal in response to illumination of its respective light-sensitive area by a light beam propagating from said object to said detector, the sensor elements are configured to image at least one reflected image comprising a plurality of reflective features generated by the object in response to the illumination pattern, each of the reflective features comprising a beam profile; at least one sensor element configured to image at least one scene image of the object illuminated by the scene illumination; at least one evaluation device, the evaluation device is configured to determine at least one first material information of the object by evaluating the beam profile of at least one of the reflective features; the evaluation device is configured to determine at least one second material information of the object by evaluating the scene image; at least one evaluation device configured to determine a material of the object using the first material information and the second material information; Equipped with.

[0015] As used herein, the term "projector," also referred to as light projector, may refer to an optical device configured to generate and project at least one illumination pattern onto an object, particularly onto a surface of the object.

[0016] As used herein, the term "pattern" may refer to any known or predetermined arrangement including a plurality of any geometric features, such as a symbol. A pattern may include a plurality of features. A pattern may include a periodic or non-periodic arrangement of features. As used herein, the term "at least one illumination pattern" may refer to at least one any pattern including illumination features adapted to illuminate at least a portion of an object.

[0017] As used herein, the term "illumination feature" refers to at least one feature that is at least partially spread out in a pattern. The illumination pattern includes a plurality of illumination features. The illumination pattern may be selected from the group consisting of at least one dot pattern; at least one line pattern; at least one stripe pattern; at least one checkerboard pattern; at least one pattern including a periodic or non-periodic arrangement of features. The illumination pattern may include a regular and / or regular and / or periodic pattern, such as a triangular pattern, a rectangular pattern, a hexagonal pattern, or even a convex tiling pattern. The illumination pattern may exhibit at least one illumination feature selected from the group consisting of at least one dot; at least one line; at least two lines, such as parallel or intersecting lines; at least one dot and one line; at least one arrangement of periodic or non-periodic features; at least one feature of any shape. The illumination pattern may include at least one pattern selected from the group consisting of: at least one dot pattern, in particular a pseudo-random dot pattern; a random dot pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one known feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including a convex tiling; at least one line pattern including at least one line; at least one line pattern including at least two lines, such as parallel or intersecting lines. For example, the projector may be configured to generate and / or project a cloud of point cloud features or non-point features. For example, the projector may be configured to generate a cloud of point cloud features or non-point features such that the illumination pattern may include multiple point cloud features or non-point features. The illumination pattern may include regular and / or regular and / or periodic patterns, such as triangular patterns, rectangular patterns, hexagonal patterns, or even patterns including convex tiling.The illumination pattern may include as many features per area as possible, such that a hexagonal pattern may be preferred. The distance between two features of each illumination pattern and / or the area of ​​at least one illumination feature may depend on a circle of confusion in the image determined by at least one detector. For example, the illumination pattern may include a periodic dot pattern.

[0018] As further used herein, the term "illuminating with at least one illumination pattern" refers to providing at least one illumination pattern to illuminate at least one object. As used herein, the term "ray" generally refers to a line perpendicular to the wavefront of light, pointing in the direction of energy flow. As used herein, the term "beam" generally refers to a collection of rays. In the following, the terms "ray" and "beam" are used synonymously. As further used herein, the term "light beam" generally refers to a quantity of light, specifically a quantity of light traveling in essentially the same direction, including the possibility that the light beam has an expansion or divergence angle. The light beam can have a spatial spread. Specifically, the light beam can have a non-Gaussian beam profile. The beam profile may be selected from the group consisting of a trapezoidal beam profile; a triangular beam profile; a conical beam. The trapezoidal beam profile can have a plateau region and at least one edge region. The light beam can specifically be a Gaussian light beam or a linear combination of Gaussian light beams, as outlined in more detail below. However, other embodiments are possible.

[0019] Furthermore, the projector may be configured to emit modulated or unmodulated light. When using multiple light sources, different light sources can have different modulation frequencies, which can then be used to differentiate the light beams.

[0020] The projector may comprise at least one array of emitters. The projector may comprise additional elements such as at least one transmission device.

[0021] As used herein, the term "emitter" may refer to at least one any device configured to provide at least one light beam for illuminating an object. Each of the emitters may be and / or include at least one element selected from the group consisting of at least one laser source, such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode pumped laser, at least one distributed feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL), in particular at least one VCSEL array; at least one non-laser light source, such as at least one LED or at least one light bulb.

[0022] The array of emitters may be a two-dimensional or one-dimensional array. The array may include a plurality of emitters arranged in a matrix. As further used herein, the term "matrix" may generally refer to an arrangement of a plurality of elements in a predetermined geometric order. The matrix may specifically be or include a rectangular matrix having one or more rows and one or more columns. The rows and columns may specifically be arranged in a rectangular manner. However, other arrangements are possible, such as non-rectangular arrangements. As an example, a circular arrangement is also possible, where the elements are arranged in concentric circles or ellipses around a central point.

[0023] For example, the emitter may be an array of VCSELs. As used herein, the term "vertical-cavity surface-emitting laser" refers to a semiconductor laser diode configured to emit a laser beam perpendicular to its top surface. Examples of VCSELs are described, for example, in en.wikipedia.org / wiki / Vertical-cavity_surface-emitting_laser. VCSELs are generally known to those skilled in the art, such as from WO2017 / 222618A. Each of the VCSELs is configured to generate at least one light beam. The VCSELs may be disposed on a common substrate or may be disposed on different substrates. The array may comprise up to 2500 VCSELs. For example, the array may comprise 38×25 VCSELs, such as a 3.5W high power array. For example, the array may comprise 10×27 VCSELs at 2.5W. For example, the array may comprise 96 VCSELs at 0.9W. The size of the array can be up to 2 mm x 2 mm, for example with 2500 elements.

[0024] The illumination feature has a first wavelength. The light beam emitted by each emitter may have a wavelength of 300 to 1100 nm, preferably 500 to 1100 nm. For example, light in the infrared spectral range may be used, for example in the range of 780 nm to 3.0 μm. In particular, light in the part of the near infrared range in which silicon photodiodes are applicable may be used, in particular in the range of 700 nm to 1100 nm. The emitters may be configured to generate at least one illumination pattern in the infrared range, in particular in the near infrared range. Using light in the near infrared range may enable light that is not detected or is only weakly detected by the human eye, but is still detectable by a silicon sensor, in particular a standard silicon sensor.

[0025] For example, the first wavelength may be 940 nm. This wavelength is considered advantageous because solar radiation on Earth has a minimum irradiance at this wavelength, as described, for example, in CIE 085-1989 "Solar spectral Irradiance". For example, the emitter may be an array of VCSELs. The VCSELs may be configured to emit light beams in the wavelength range of 800 nm to 1000 nm. For example, the VCSELs may be configured to emit light beams at 808 nm, 850 nm, 940 nm, or 980 nm. Preferably, the VCSELs emit light at 940 nm.

[0026] The projector may comprise at least one transfer device configured to generate illumination features from a light beam incident on the transfer device. The term "transfer device", also referred to as "transfer system", may generally refer to one or more optical elements adapted to modify a light beam, such as by modifying one or more of the beam parameters of the light beam, the width of the light beam, or the direction of the light beam. The transfer device may have at least one imaging optical device. The transfer device may specifically include at least one selected from the group consisting of: at least one lens, e.g. at least one lens selected from the group consisting of at least one focusable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system; at least one holographic optical element; at least one meta-optical element. In particular, the transfer device comprises at least one refractive optical lens stack. Thus, the transfer device may comprise a multi-lens system having refractive properties.

[0027] The one or more light beams generated by the projector may generally propagate parallel to the optical axis or may propagate at an angle to the optical axis, for example at an angle with the optical axis. The projector may be configured such that the one or more light beams propagate from the projector towards the scene along the optical axis. To this end, the projector may include at least one reflecting element, preferably at least one prism, for deflecting the illuminating light beam onto the optical axis. As an example, the one or more light beams, such as a laser light beam, and the optical axis may have an angle of less than 10°, preferably less than 5°, or even less than 2°. However, other embodiments are possible. Furthermore, the one or more light beams may be on the optical axis or off the optical axis. As an example, the one or more light beams may be parallel to the optical axis or even coincident with the optical axis with a distance of less than 10 mm, preferably less than 5 mm, or even less than 1 mm to the optical axis.

[0028] As used herein, the term "flood light source" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer to at least one any device adapted to provide at least one illumination light beam for illumination of an object, without being limited thereto. The flood light source is configured for scene illumination. As used herein, the term "scene illumination" may refer to diffuse and / or uniform illumination of a scene. As used herein, the term "scene" may refer to at least one any object or spatial region. A scene may include at least one object and its surrounding environment. The flood light source may be adapted to directly or indirectly illuminate the object, where the illumination is reflected or scattered by the surface of the object and thereby directed at least in part to the sensor element. The flood light source may be adapted to illuminate the object, for example, by directing a light beam at the object that reflects the light beam.

[0029] The flood light source is configured to emit scene illumination having a second wavelength different from the first wavelength. The flood light source may comprise at least one light emitting diode (LED). However, other embodiments are possible. For example, the flood light source may comprise at least one VCSEL as a light source and at least one diffuser. The flood light source may comprise a single light source, or multiple light sources. By way of example, the light emitted by the flood light source may have a wavelength of 300-1100 nm, in particular 500-1100 nm. Additionally or alternatively, light in the infrared spectral range may be used, for example light in the range of 780 nm-3.0 μm. In particular, a part of the near infrared range where silicon photodiodes are applicable, in particular light in the range of 700 nm-1100 nm, may be used. The flood light source may be configured to emit light at a single wavelength. In particular, the wavelength may be in the near infrared range. In other embodiments, the flood light source may be adapted to emit light at multiple wavelengths to enable additional measurements in other wavelength channels.

[0030] The first and second wavelengths can be selected to distinguish between materials. For example, two or more materials can have similar reflectances at a first wavelength, but different reflectances at a second wavelength. The first and second wavelengths can be different wavelengths in the infrared spectral range. For example, the first wavelength can be 940 nm and the second wavelength can be 850 nm.

[0031] The projector and flood light source may form a coordinate system, with the longitudinal coordinate being the coordinate along the optical axis. The coordinate system may be a polar coordinate system, with the optical axis forming the z-axis, and distance and polar angle from the z-axis being used as additional coordinates. Directions parallel or anti-parallel to the z-axis are considered longitudinal directions, and coordinates along the z-axis are considered longitudinal coordinates z. Any direction perpendicular to the z-axis may be considered transverse, and polar coordinates and / or polar angles may be considered transverse coordinates. As used herein, the term "depth information" may relate to information from which the longitudinal coordinate and / or longitudinal coordinate may be derived.

[0032] As used herein, the term "sensor element" generally refers to a device or a combination of devices configured to sense at least one parameter. In this case, the parameter may specifically be an optical parameter and the sensor element may specifically be an optical sensor element. The sensor element may be formed as an integral single device or as a combination of several devices. The sensor element comprises a matrix of optical sensors. The sensor element may comprise at least one CMOS sensor. The matrix may be composed of independent pixels, such as independent optical sensors. Thus, a matrix of inorganic photodiodes may be constituted. However, alternatively, one or more of commercially available matrices may be used, for example CCD detectors, such as CCD detector chips, and / or CMOS detectors, such as CMOS detector chips. Thus, in general, the sensor element may be and / or may include at least one CCD device and / or CMOS device, and / or the optical sensor may form a sensor array or be part of a sensor array, such as the matrix mentioned above. Thus, by way of example, the sensor element may comprise an array of pixels, such as a rectangular array with m rows and n columns, where m, n are independently positive integers. Preferably, a plurality of columns and a plurality of rows are provided, i.e. n>1, m>1. Thus, by way of example, n may be 2-16 or more, and m may be 2-16 or more. Preferably, the ratio of the number of rows to the number of columns is close to 1. By way of example, n and m may be selected such that 0.3≦m / n≦3, such as by selecting m / n=1:1, 4:3, 16:9 or similar. By way of example, the array may be a square array with equal numbers of rows and columns, such as by selecting m=2, n=2 or m=3, n=3, etc.

[0033] The matrix may be composed of independent pixels, such as independent photosensors. It may thus constitute a matrix of inorganic photodiodes. Alternatively, however, one or more commercially available matrices may be used, for example CCD detectors, such as CCD detector chips, and / or CMOS detectors, such as CMOS detector chips. In general, therefore, the photosensor may be and / or include at least one CCD and / or CMOS device and / or the photosensor may form or be part of a sensor array, such as the above-mentioned matrices.

[0034] The matrix may specifically be a rectangular matrix having at least one row, preferably multiple rows and multiple columns. By way of example, the rows and columns may be oriented in a substantially perpendicular direction. As used herein, the term "substantially perpendicular" refers to a perpendicular orientation state, for example with a tolerance of ±20° or less, preferably with a tolerance of ±10° or less, more preferably with a tolerance of ±5° or less. Similarly, the term "substantially parallel" refers to a parallel orientation state, for example with a tolerance of ±20° or less, preferably with a tolerance of ±10° or less, more preferably with a tolerance of ±5° or less. Thus, by way of example, a tolerance of less than 20°, specifically less than 10°, or even less than 5° may be tolerated. To provide a wide field of view, the matrix may specifically have at least 10 rows, preferably at least 500 rows, more preferably at least 1000 rows. Similarly, the matrix may have at least 10 columns, preferably at least 500 columns, more preferably at least 1000 columns. The matrix may include at least 50 light sensors, preferably at least 100,000 light sensors, more preferably at least 5,000,000 light sensors. The matrix may include a number of pixels in the range of several megapixels. However, other embodiments are possible. Thus, in configurations where axial rotational symmetry is expected, a circular or concentric arrangement of the photosensors of the matrix, which may also be called pixels, may be preferred.

[0035] Thus, by way of example, the sensor element may be part of or constitute a pixelated optical device. For example, the sensor element may be and / or include at least one CCD and / or CMOS device. By way of example, the sensor element may be part of or constitute at least one CCD and / or CMOS device having a matrix of pixels, each pixel forming a light-sensitive area. The sensor element may employ a rolling shutter scheme or a global shutter scheme for reading the matrix of light sensors.

[0036] As used herein, the term "light sensor" generally refers to a light-sensitive device for detecting a light beam, such as for detecting illumination and / or a light spot generated by at least one light beam. As further used herein, the term "light-sensitive area" generally refers to an area of ​​a light sensor that is externally illuminated by at least one light beam and generates at least one sensor signal in response to said illumination. The light-sensitive area may specifically be located on the surface of the respective light sensor. However, other embodiments are possible. The sensor element may include multiple light sensors, each having a light-sensitive area. As used herein, the term "light sensor each having at least one light-sensitive area" refers to a configuration with multiple single light sensors, each having one light-sensitive area, and a configuration with one combined light sensor having multiple light-sensitive areas. The term "light sensor" further refers to a light-sensitive device configured to generate one output signal. If the sensor element includes a plurality of light sensors, each light sensor may be embodied by providing exactly one light-sensitive area that can be illuminated, for example, so that exactly one light-sensitive area is present in each light sensor, and generates exactly one uniform sensor signal for the entire light sensor in response to illumination of the light-sensitive area. Each light sensor may therefore be a single-area light sensor. The use of a single-area light sensor, however, makes the construction of the detector particularly simple and efficient. Thus, by way of example, commercially available light sensors, such as commercially available silicon photodiodes, each having exactly one light-sensitive area, may be used in the construction. However, other embodiments are possible.

[0037] Preferably, the photosensitive area may be oriented substantially perpendicular to the optical axis, which may be a straight optical axis or may even be refracted or split, such as by using one or more deflection elements and / or one or more beam splitters, in which case the substantially perpendicular orientation refers with respect to the local optical axis of each branch or beam path of the optical arrangement.

[0038] The light sensor may in particular be or may include at least one photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, most preferably a silicon photodetector. In particular, the light sensor may have sensitivity in the infrared spectral range. All pixels of the matrix, or at least a group of light sensors of the matrix, may in particular be identical. Groups of identical pixels of the matrix may in particular be provided for different spectral ranges, or all pixels may be identical with respect to their spectral sensitivity. Furthermore, the pixels may be identical with respect to their size and / or their electronic or optoelectronic properties. In particular, the light sensor may be or may include at least one inorganic photodiode with sensitivity in the infrared spectral range, preferably in the range of 700 nm to 3.0 micrometers. In particular, the light sensor may have sensitivity in the part of the near infrared range, in particular in the range of 700 nm to 1100 nm, where silicon photodiodes are applicable. Infrared light sensors that may be used in the light sensor may be commercially available infrared light sensors, such as, for example, infrared light sensors marketed under the brand name Hertzstueck® by trinamiX GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, by way of example, the light sensor may include at least one light sensor of the intrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, HgCdTe photodiodes. Additionally or alternatively, the light sensor may include at least one light sensor of the extrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge:Au photodiodes, Ge:Hg photodiodes, Ge:Cu photodiodes, Ge:Zn photodiodes, Si:Ga photodiodes, Si:As photodiodes.Additionally or alternatively, the optical sensor may include at least one photoconductive sensor, such as a PbS or PbSe sensor, a bolometer, preferably a bolometer selected from the group consisting of a VO bolometer and an amorphous Si bolometer.

[0039] The optical sensor may be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. Specifically, the optical sensor may be sensitive in the visible spectral range of 500 nm to 780 nm, most preferably 650 nm to 750 nm, or 690 nm to 700 nm. Specifically, the optical sensor may be sensitive in the near infrared range. Specifically, the optical sensor may be sensitive in the portion of the near infrared range where silicon photodiodes are applicable, particularly in the range of 700 nm to 1000 nm. The optical sensor may be sensitive in the infrared spectral range, specifically in the range of 780 nm to 3.0 μm. For example, the optical sensors may each independently be or include at least one element selected from the group consisting of a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. For example, the sensor element may be or may include at least one element selected from the group consisting of a CCD sensor element, a CMOS sensor element, a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. Any other type of photosensitive element may be used. The photosensitive element may generally be made entirely or partially of inorganic materials and / or may be made entirely or partially of organic materials. Most commonly, one or more photodiodes may be used, such as commercially available photodiodes, e.g., inorganic semiconductor photodiodes.

[0040] The detector may further include at least one further redirecting device. The detector may further include one or more additional elements, such as one or more additional optical elements. The detector may include at least one optical element selected from the group consisting of redirecting devices, such as at least one lens and / or at least one lens system, at least one diffractive optical element. The "further redirecting device", also referred to as a "transfer system", may include one or more optical elements adapted to modify the light beam, such as by modifying one or more of the beam parameters of the light beam, the width of the light beam, or the direction of the light beam. The further redirecting device may be adapted to direct the light beam to a light sensor. The further transfer device may in particular comprise one or more of: at least one lens, for example at least one lens selected from the group consisting of at least one focusable lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflecting element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system. The further transfer device may have a focal length. As used herein, the term "focal length" of the further transfer device refers to the distance at which the incident parallel light rays that may impinge on the transfer device are focused to a "focus", also called a "focal point". The focal length therefore constitutes an indication of the ability of the further transfer device to converge the incident light beam. The further transfer device may therefore comprise one or more imaging elements that may have the effect of a focusing lens. By way of example, the further transfer device may comprise one or more lenses, in particular one or more refractive lenses, and / or one or more convex mirrors. In this example, the focal length can be defined as the distance from the center of the thin refractive lens to the principal focus of the thin lens. For a converging thin refractive lens, such as a convex or biconvex thin lens, the focal length can be considered positive and give the distance over which collimated light impinging on the thin lens as a transport device can be focused to a single spot.Furthermore, the further redirecting device may include at least one wavelength-selective element, for example at least one optical filter. Furthermore, the further redirecting device may be designed to apply a predefined beam profile to the electromagnetic radiation, for example at the position of the sensor region, in particular at the sensor area. Any of the above-mentioned embodiments of the further redirecting device may in principle be realized individually or in any desired combination.

[0041] The further transfer device may have an optical axis. As used herein, the term "optical axis of the further transfer device" generally refers to an axis of mirror symmetry or rotational symmetry of a lens or lens system. The further transfer system may, by way of example, include at least one beam path, in which the elements of the transfer system are arranged rotationally symmetrically with respect to the optical axis. Furthermore, one or more optical elements arranged in the beam path may be off-centered or tilted with respect to the optical axis. However, in this case, the optical axis may be defined in turn by interconnecting the centers of the optical elements in the beam path, such as by interconnecting the centers of the lenses, and in this context the optical sensor is not considered as an optical element. The optical axis may generally denote a beam path. There, the detector may have a single beam path along which the light beam travels from the object to the optical sensor, or may have multiple beam paths. By way of example, a single beam path may be provided, or the beam path may be divided into two or more partial beam paths. In the latter case, each partial beam path may have its own optical axis. In the case of several light sensors, the light sensors may be arranged in one and the same beam path or partial beam path, but alternatively the light sensors may also be arranged in different partial beam paths.

[0042] The further transfer device may configure a coordinate system where the vertical coordinate is the coordinate along the optical axis and d is the spatial offset from the optical axis. The coordinate system may be a polar coordinate system where the optical axis of the transfer device forms the z-axis and the distance and polar angle from the z-axis may be used as additional coordinates. Directions parallel or anti-parallel to the z-axis may be considered as vertical directions and coordinates along the z-axis may be considered as vertical coordinates. Any direction perpendicular to the z-axis may be considered as horizontal direction and the polar coordinate and / or polar angle may be considered as horizontal coordinates.

[0043] As used herein, a "sensor signal" generally refers to a signal generated by a light sensor and / or at least one pixel of the light sensor in response to illumination. In particular, the sensor signal may be or include at least one electrical signal, such as at least one analog electrical signal and / or at least one digital electrical signal. More particularly, the sensor signal may be or include at least one voltage signal and / or at least one current signal. More particularly, the sensor signal may include at least one photocurrent. Furthermore, a raw sensor signal may be used, or the detector, the light sensor, or any other element may be adapted to process or pre-process the sensor signal, thereby generating a secondary sensor signal that may also be used as a sensor signal, such as pre-processing by filtering, etc.

[0044] As used herein, the term "image" may specifically relate to, but is not limited to, data recorded by using a sensor element, such as a plurality of electronic readings from a sensor element, such as a pixel of a CCD or CMOS chip. The term "imaging at least one image" may refer to capturing and / or recording an image.

[0045] The sensor elements are configured to image at least one scene image of an object illuminated by the scene illumination. The scene image may be generated in response to diffuse and / or uniform illumination of the object by the scene illumination. The scene image may not include reflective features generated by the illumination pattern. The scene image may be at least one two-dimensional image. As used herein, the term "two-dimensional image" may generally refer to an image having information regarding lateral coordinates, such as height and width dimensions.

[0046] The sensor element is configured to image at least one reflected image including a plurality of reflective features generated by an object in response to the illumination pattern. Each of the reflective features includes a beam profile. As used herein, the term "reflective feature" may refer to a feature in an image plane generated by a scene, particularly an object, in response to illumination, specifically at least one illumination feature. Each of the reflective features includes at least one beam profile, also called a reflected beam profile. As used herein, the term "beam profile" of a reflective feature may generally refer to at least one intensity distribution of a reflective feature, such as a light spot on an optical sensor, as a function of pixels. The beam profile may be selected from the group consisting of a linear combination of a trapezoidal beam profile; a triangular beam profile; a conical beam profile and a Gaussian beam profile.

[0047] The evaluation device may be configured to perform an evaluation of the reflected image. The evaluation of the reflected image may include identifying a reflection feature of the reflected image. The evaluation device may be configured to perform at least one image analysis and / or image processing to identify the reflection feature. The image analysis and / or image processing may use at least one feature detection algorithm. The image analysis and / or image processing may include the following: filtering; selecting at least one region of interest; forming a difference image between an image generated by the sensor signal and at least one offset; inverting the sensor signal by inverting the image generated by the sensor signal; forming a difference image between images generated by the sensor signal at different times; background correction; decomposition into color channels; decomposition into hue; saturation; luminance channel; frequency decomposition; singular value decomposition; applying a blob detector; applying a corner detector; applying a determinant of a Hessian filter; applying a principal curvature based region detector; applying a maximum stable extremum region detector; applying a generalized Hough transform; applying a ridge detector; applying an affine invariant feature detector; applying an affine adaptive interest point operator; applying a Harris affine region detector; applying a Hessian affine region detector; applying a scale invariant feature transform. The method may include one or more of: applying a scale-space extremum detector; applying a local feature detector; applying a fast robust feature algorithm; applying a histogram of gradient position and orientation algorithm; applying a histogram of oriented gradient descriptors; applying a Deriche edge detector; applying a differential edge detector; applying a space-time interest point detector; applying a Moravec corner detector; applying a Canny edge detector; applying a Laplace operator of a Gaussian filter; applying a differential Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Schall operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; and generating a binary image. The region of interest may be determined manually by a user or automatically, such as by recognizing features in an image generated by an optical sensor.

[0048] As further used herein, the term "evaluation device" generally refers to any data processing device adapted to perform specified operations, for example by using at least one processor and / or at least one application specific integrated circuit. Thus, by way of example, the at least one evaluation device may include software code stored therein, including a multitude of computer commands. The evaluation device may provide one or more hardware elements for performing one or more of the specified operations, and / or may provide one or more processors having software running thereon for performing one or more of the specified operations. The operations (including evaluating the image) may be performed by the at least one evaluation device. Thus, by way of example, the one or more instructions may be implemented in software and / or hardware. Thus, by way of example, the evaluation device may comprise one or more computers, application specific integrated circuits (ASICs), digital signal processors (DSPs), or one or more programmable devices, such as field programmable gate arrays (FPGAs), configured to perform the above evaluations. However, additionally or alternatively, the evaluation device may also be fully or partially embodied by hardware. The evaluation device and the detector may be fully or partially integrated in a single device. Thus, in general, the evaluation device may also form part of the detector, or the evaluation device and the detector may be embodied, completely or partly, as separate devices.

[0049] The evaluation device may be or include one or more integrated circuits, such as one or more application specific integrated circuits (ASICs), and / or one or more computers, preferably one or more microcomputers, and / or one or more data processing devices, such as microcontrollers, field programmable gate arrays, or digital signal processors. Additional components may be included, such as data acquisition devices, such as one or more pre-processing devices, and / or one or more devices for receiving and / or pre-processing the sensor signals, such as one or more analog-to-digital converters and / or one or more filters. Furthermore, the evaluation device may comprise one or more measurement devices, such as one or more measurement devices for measuring current and / or voltage. Furthermore, the evaluation device may include one or more data storage devices. Furthermore, the evaluation device may include one or more interfaces, such as one or more wireless interfaces and / or one or more wired interfaces.

[0050] The evaluation device may be configured to display, visualize, analyze, distribute, communicate, or further process information, such as information obtained by the sensor elements. The evaluation device may be connected to or incorporate at least one of a display, projector, monitor, LCD, TFT, loudspeaker, multi-channel sound system, LED pattern, or further visualization device, as examples. It may further be connected to or incorporate at least one of a communication device or communication interface, connector, or port, capable of sending encrypted or unencrypted information using one or more of email, text message, telephone, Bluetooth, Wi-Fi, infrared, or Internet interfaces, ports, or connections. It may further be connected to or incorporate at least one of the following: a processor, a graphics processor, a CPU, an Open Multimedia Applications Platform (OMAP®), an integrated circuit, a product from the Apple A-series or the Samsung S3C2 series, a system on a chip such as a microcontroller or microprocessor, one or more memory blocks such as a ROM, a RAM, an EEPROM or a flash memory, a timing source such as an oscillator or a phase locked loop, a counter timer, a real time timer or a power-on-reset-generator, a voltage regulator, a power management circuit or a DMA controller. The individual units may further be connected to or integrated into an Internet of Things or Industry 4.0 type network by a bus such as the AMBA bus.

[0051] The evaluation device may be connected by or have a serial or parallel interface or port, a further external interface or port such as one or more of USB, Centronics Port, FireWire, HDMI, Ethernet, Bluetooth, RFID, Wi-Fi, USART, or SPI, or an analog interface or port such as one or more of a standardized interface or port to a further device such as a 2D camera device using an ADC, DAC, or RGB interface such as CameraLink. The evaluation device may be further connected by one or more of a processor-to-processor interface or port, an FPGA-FPGA-interface, or a serial or parallel interface port. The evaluation device may be further connected to one or more of an optical disk drive, a CD-RW drive, a DVD+RW drive, a flash drive, a memory card, a disk drive, a hard disk drive, a solid state disk, or a solid state hard disk.

[0052] The evaluation device may be connected by or may have one or more further external connectors, such as one or more of a phone connector, an RCA connector, a VGA connector, a hermaphrodite connector, a USB connector, an HDMI connector, an 8P8C connector, a BCN connector, an IEC60320 C14 connector, a fiber optic connector, a D-subminiature connector, an RF connector, a coaxial connector, a SCART connector, an XLR connector, and / or may incorporate at least one suitable socket for one or more of these connectors.

[0053] The evaluation device can be configured to determine a beam profile of each reflection feature. As used herein, the term "determine a beam profile" refers to identifying at least one reflection feature provided by the optical sensor and / or selecting at least one reflection feature provided by the optical sensor and evaluating at least one intensity distribution of the reflection feature. As an example, a region of the matrix can be used and evaluated to determine an intensity distribution, such as a three-dimensional intensity distribution or a two-dimensional intensity distribution, such as along an axis or line through the matrix. As an example, the center of illumination by the light beam can be determined, such as by determining at least one pixel with the highest illumination, and a cross-sectional axis can be selected through the center of illumination. The intensity distribution can be an intensity distribution as a function of coordinate along this cross-sectional axis through the center of illumination. Other evaluation algorithms are possible.

[0054] The reflection feature may cover or extend over at least one pixel of the image. For example, the reflection feature may cover or extend over a number of pixels. The evaluation device may be configured to determine and / or select all pixels that are connected and / or belong to the reflection feature, e.g. the light spot. The evaluation device may determine the center of intensity as:

number

[0055] The evaluation device is configured to determine at least one first material information of the object by evaluating the beam profile of the at least one reflection feature. The evaluation device may be configured to determine at least one first material information of the object by evaluating the beam profile of at least three or more reflection features, in particular the beam profiles of all reflection features.

[0056] The evaluation device can be configured to identify the reflection characteristic as generated by an item having a particular material characteristic if the reflected beam profile of the item meets at least one predetermined or predefined criterion. As used herein, the term "at least one predetermined or predefined criterion" refers to at least one characteristic and / or value suitable for distinguishing material characteristics. The predetermined or predefined criterion may be or may include at least one predetermined or predefined value and / or threshold and / or threshold range that refers to the material characteristic. The reflection characteristic can be displayed as generated by an item having a particular material characteristic if the reflected beam profile meets at least one predetermined or predefined criterion. As used herein, the term "display" refers to any display, such as an electronic signal and / or at least one visual or audio display.

[0057] As used herein, the term "determining at least one material information" may refer to assigning at least one material property to each reflectance characteristic. The evaluation device may include at least one database including a list and / or table, such as a look-up list or look-up table, of predefined and / or predetermined material properties. The list and / or table of material properties may be determined and / or generated by performing at least one test measurement, such as, for example, by performing a material test using a sample with known material properties. The list and / or table of material properties may be determined and / or generated at the manufacturer's site and / or by a user. The material properties may be further assigned to a material classifier, such as one or more of a material name, biological or non-biological material, translucent or non-translucent material, metal or non-metal, fur or non-fur, carpet or non-carpet, reflective or non-reflective, specular or non-specular, foam or non-foam, roughness group, etc. The evaluation device may include at least one database including a list and / or table including material names and / or material groups associated with material properties.

[0058] To determine the first material information, beam profile analysis may be used. Specifically, beam profile analysis utilizes the reflectance properties of coherent light projected onto the surface of an object to classify materials. Material classification may be performed as described in WO2020 / 187719, EP application 20159984.2 filed February 28, 2020 and / or EP application 20154961.5 filed January 31, 2020, and C. Lennartz, F. Schick, S. Metz, "Whitepaper-Beam Profile Analysis for 3D imaging and material detection," April 28, 2021, Ludwigshafen, Germany, the entire contents of which are incorporated by reference. In particular, a periodic grid of laser spots is projected, for example a hexagonal grid as described in EP application 20170905.2 filed on 22 April 2020, and the reflected image is recorded by a camera. Analysing the beam profile of each reflected feature recorded by the sensor element can be performed by feature-based methods and / or by using the reflected features of the reflected image based on convolutional neural network classification. Feature-based methods can be used in combination with machine learning methods that allow parameterisation of the classification model. Convolutional neural networks can be utilized to classify materials by using the reflected images as input.

[0059] The feature-based method may be described below. The evaluation device may be configured to compare the reflected beam profile with at least one pre-determined and / or pre-recorded and / or pre-defined beam profile. The pre-determined and / or pre-recorded and / or pre-defined beam profile may be stored in a table or a look-up table, may be determined, for example empirically, and may be stored, as an example, in at least one data storage device of the detector. For example, the pre-determined and / or pre-recorded and / or pre-defined beam profile may be determined upon initial start-up of the device performing the method according to the invention. For example, the pre-determined and / or pre-recorded and / or pre-defined beam profile may be stored in at least one data storage device of the evaluation device, for example by software, in particular by an app downloaded from an app store or the like. The reflection feature may be identified as being generated by an item having material property m if the reflected beam profile and the pre-determined and / or pre-recorded and / or pre-defined beam profile are identical. The comparison may include superimposing the reflected beam profile and the pre-determined or pre-defined beam profile such that their centers of intensity match. The comparison may comprise determining a deviation, e.g., a sum of squared inter-point distances, between the reflected beam profile and a pre-determined and / or pre-recorded and / or pre-defined beam profile. The evaluation device may be adapted to compare the determined deviation with at least one threshold value, and if the determined deviation is lower and / or equal to the threshold value, the surface is indicated as biological tissue and / or the detection of biological tissue is confirmed. The threshold value may be stored in a table or a look-up table, may be determined, e.g., empirically, and may be stored, as an example, in at least one data storage device of the evaluation device.

[0060] Additionally or alternatively, material properties can be determined by applying at least one image filter to the reflected image. As further used herein, the term "image" refers to a two-dimensional function f(x,y), where brightness and / or color values ​​are given for any x,y position in the image. The positions may be discretized corresponding to the recording pixels. The brightness and / or color may be discretized corresponding to the bit depth of the light sensor. As used herein, the term "image filter" refers to at least one mathematical operation applied to the beam profile and / or at least one specific region of the beam profile. In particular, the image filter Φ maps the image f or a region of interest in the image to a real number Φ(f(x,y))=φ, where φ denotes a feature, in particular a material feature. Images can be affected by noise, and the same is true for features. Thus, the features can be random variables. The features can follow a normal distribution. If the features do not follow a normal distribution, they can be transformed to follow a normal distribution, such as by Box-Cox-Transformation.

[0061] The evaluation device evaluates at least one material characteristic φ by applying at least one material-dependent image filter Φ2 to the image. 2m As used herein, the term “material dependent” image filter refers to an image having a material dependent output. The output of a material dependent image filter is referred to herein as a “material feature φ 2m " or "Material-dependent feature φ 2m The material feature may be or may include at least one piece of information regarding at least one material property of a surface in the scene that generated the reflectance feature.

[0062] The material dependent image filter may be an intensity filter; a spot shape filter; a squared norm gradient; a standard deviation; a smoothness filter such as a Gaussian filter or a median filter; a gray level generation based contrast filter; a gray level generation based energy filter; a gray level generation based uniformity filter; a gray level generation based dissimilarity filter; a rho energy filter; a threshold domain filter; or a linear combination thereof; or an intensity filter, a spot shape filter, a squared norm gradient, a standard deviation, a smoothness filter, a gray level generation based energy filter, a gray level generation based uniformity filter, a gray level generation based dissimilarity filter, a rho energy filter, or a threshold domain filter, or |ρ Ф2other,Фm These linear combinations (Φ m is a further material dependent image filter Φ, which is correlated to one or more of a brightness filter, a spot shape filter, a squared norm gradient, a standard deviation, a smoothness filter, a gray level generation based energy filter, a gray level generation based uniformity filter, a gray level generation based dissimilarity filter, a low energy filter, or a threshold area filter, or a linear combination thereof. 2other , . . . The further material-dependent image filter Φ 2other is the material-dependent image filter Φ m and |ρ Ф2other,Фm |≧0.60, preferably |ρ Ф2other,Фm |May be correlated by ≧0.80.

[0063] The material-dependent image filter may be at least one any filter Φ that passes hypothesis validation. As used herein, the term "passes hypothesis validation" refers to the fact that the null hypothesis H0 is rejected and an alternative hypothesis H1 is accepted. The hypothesis validation may include validating the material dependency of the image filter by applying the image filter to a predefined data set. The data set may include multiple beam profile images. As used herein, the term "beam profile image" refers to NB refers to the sum of Gaussian radial basis functions,

number

number

number

[0064] [Table 1]

[0065] The values ​​of x,y, are

number

[0066] Next, for each image fk For the feature value φ corresponding to the filter Φ k can be calculated,

number

number

number

number

number

number

[0067] Hypothesis testing may include determining the mean sum of squares between:

number

number

number

number

[0068] In the following, the image filter is described assuming that the reflection image contains at least one reflection feature, in particular a spot image.

number

[0069] For example, the material dependent image filter may be a luminance filter. The luminance filter may return a luminance measurement of a spot as a material characteristic. The material characteristic may be:

number

number

[0070]

number

[0071] For example, a material dependent image filter may be a filter that has an output that is dependent on the spot shape. The material dependent image filter may return a value as a material characteristic that correlates to the translucency of the material, which affects the shape of the spot. The material characteristic may be:

number

number

number

[0072] For example, the material dependent image filter may be a squared norm gradient, which may return a value that correlates to a measure of the soft-hard transition and / or roughness of the spot as a material characteristic.

number

number

[0073] In the formula, μ is

number

[0074] For example, the material dependent image filter may be a smoothness filter, such as a Gaussian filter or a median filter. In one embodiment of the smoothness filter, the image filter may refer to the observation that volumetric scattering exhibits less speckle contrast compared to diffuse scattering materials. The image filter may quantify the smoothness of the spots corresponding to the speckle contrast as a material characteristic. The material characteristic may be:

number

number

number

number

[0075] The material characteristics of this filter are:

number

[0076] In the formula, Var represents the variance function.

[0077] For example, the image filter may be a gray level generation based contrast filter. The material filter may be a gray level generation matrix

number

[0078] The material characteristics of the gray level generation based contrast filter are:

number

[0079] For example, the image filter may be a grey level generation based energy filter, which is based on the grey level generation matrix defined above.

[0080] The material characteristics of the gray level generation based energy filter are:

number

[0081] For example, the image filter may be a grey level generation based uniformity filter, which is based on the grey level generation matrix defined above.

[0082] The material characteristics of the gray level generation based uniformity filter are:

number

[0083] For example, the image filter may be a grey level generation based dissimilarity filter, which is based on the grey level generation matrix defined above.

[0084] The material features of the gray level generation based dissimilarity filter are

number

[0085] For example, the image filter may be a low energy filter. The material filter is a low vector L5=[1,4,6,4,1] and E5=[-1,-2,0,-2,-1] and a material L5 (E5) T and E5 (L5) T It is based on.

[0086] Image f k These matrices:

number

[0087] where the material characteristics of the low energy filter are:

number

[0088] For example, the material dependent image filter may be a threshold domain filter. This material feature may relate two areas in the image plane. The first area, Ω1, may be an area where the function f is greater than α times the maximum value of f. The second area, Ω2, may be an area where the function f is less than α times the maximum value of f, but greater than a threshold of ε times the maximum value of f. Preferably, α may be 0.5 and ε may be 0.05. Due to speckle or noise, the areas may not simply correspond to an inner and outer circle of the spot center. For example, Ω1 may include speckle or unconnected areas of the outer circle. The material feature may be:

number

[0089] The evaluation device calculates a material characteristic φ to determine the material properties of the surface that generated the reflection characteristic. 2m and a material property of the surface that generated the reflection feature. The predetermined relationship may be one or more of an empirical relationship, a semi-empirical relationship, and an analytically derived relationship. The evaluation device may comprise at least one data storage device for storing the predetermined relationship, such as, for example, a look-up list or a look-up table.

[0090] The determination of the at least one first material information may include using artificial intelligence, in particular a convolutional neural network. Using the reflection image as input of the convolutional neural network may allow to generate a classification model with sufficient accuracy to distinguish materials. By selecting important regions in the reflection image, only physically valid information is passed to the network, so that only a compact training data set may be required. Furthermore, a very compact network architecture may be generated.

[0091] Specifically, at least one parameterized classification model can be used to determine the at least one first material information. The parameterized classification model can be configured to classify the material by using the reflectance image as an input. The classification model can be parameterized by using one or more of machine learning, deep learning, neural networks, or other forms of artificial intelligence. As used herein, the term "machine learning" is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term can specifically refer to, without limitation, a method of using artificial intelligence (AI) for automatic model building, and in particular for model parameterization. The classification model can be a classification model configured to identify materials. The material properties can be determined by applying an optimization algorithm with respect to at least one optimization target to the classification model. The machine learning can be based on at least one neural network, in particular a convolutional neural network. The weights and / or topology of the neural network can be pre-determined and / or pre-defined. Specifically, training of the classification model can be performed using machine learning. The classification model can include at least one machine learning architecture and model parameters. For example, the machine learning architecture may be or include one or more of linear regression, logistic regression, random forest, naive Bayes classification, nearest neighbor, neural network, convolutional neural network, generative adversarial network, support vector machine, gradient boosting algorithm, etc. As used herein, the term "training", also written as "learning", is a broad term and should be given its ordinary and customary meaning to those skilled in the art and should not be limited to a special or customized meaning. The term may specifically refer, without limitation, to the process of building a model, and in particular the process of determining and / or updating the parameters of the model. The classification model may be at least partially data-driven.For example, the parameterized classification model may be based on experimental data. For example, training may include using at least one training data set, the training data set including images, particularly reflectance images, of a plurality of items having known material properties.

[0092] The evaluation device is configured to determine at least one second material information of the object by evaluating the scene image. The evaluation of the scene image may include determining at least one reflectance value of at least one region of interest. The region of interest may be selected manually by a user and / or automatically by using at least one object detection algorithm. The evaluation may include comparing the reflectance value with at least one pre-determined or predefined value and / or threshold and / or threshold range referring to material properties. The determination of the second material information may be performed using at least one parameterized classification model. The parameterized model may be a combined parameterized classification model for determining the first material information and the second material information and / or at least one further parameterized classification model in addition to the parameterized model used in particular for determining the first material information. The combined parameterized classification model and / or the further parameterized classification model may be at least partially data-driven. For example, the combined parameterized classification model and / or the further parameterized classification model may be based on experimental data. For example, the training may include using at least one training dataset that includes images, in particular scene images of a plurality of items having known material properties for the further parameterized model, and / or scene images and reflectance images for a combined parameterized classification model.

[0093] The determination of the second material information may be performed consecutively or simultaneously with the determination of the first material information. At least the second material information can be used separately from the first material information and / or together with the first material information. The evaluation of the scene images can be performed consecutively or simultaneously with the evaluation of the reflected images using at least one parameterized classification model. The evaluation device may be configured to determine a final material classification. The final material classification may be a combination of the results of the parameterized classification model using the first material information and the second material information.

[0094] For example, the evaluation device can be configured to sequentially use the first material information and at least one parameterized classification model providing at least one material confidence to determine whether at least one second material information needs to be evaluated. For example, if skin or non-skin needs to be classified and the first material information rejects skin with high confidence, the second material information may not be needed for the final classification decision. If the second material information is used, the decision may be evaluated using the at least one parameterized classification model.

[0095] For example, the evaluations can be performed simultaneously, for example, using combined parameterized classification models.

[0096] The evaluation device is configured to use the first material information and the second material information to determine a combination of parameterized classification model results, the combination of the first material information and the second material information may enable an increased confidence in the material classification.

[0097] The first material information may be a material property characterizing the material of the object. The evaluation device may be configured to use the second material information as an additional information channel to distinguish materials, such as in ambiguous cases. Using an additional information channel is advantageous in the case of difficult targets, such as distinguishing between skin and silicon. The detector may be configured to distinguish between biological and non-biological materials. The detector may be configured to distinguish between skin and non-skin objects.

[0098] The evaluation device is configured to determine the material of the object using the first material information and the second material information. The evaluation device may be configured to determine the material of the object by using only the first material information. This may be suitable for many materials. However, for different targets, i.e. targets with similar reflectivity at the first wavelength, the second material information can be taken into account by the evaluation device to distinguish the materials. This significantly improves the reliability of the material detection and therefore the possibility of identifying spoofing attacks.

[0099] The evaluation device is configured to determine a vertical coordinate of at least one reflection feature by analyzing the beam profile of each reflection feature. The evaluation device determines at least one vertical coordinate z for each reflection feature by analyzing the beam profile of each reflection feature. DPR The evaluation device may be configured to determine the vertical coordinate z of the reflective feature using the so-called photon ratio to depth technique, also called beam profile analysis. DPR For Depth from Photon Ratio (DPR) techniques, reference is made to WO2018 / 091649A1, WO2018 / 091638A1 and WO2018 / 091640A1, as well as C. Lennartz, F. Schick, S. Metz, "Whitepaper-Beam Profile Analysis for 3D imaging and material detection," April 28, 2021, Ludwigshafen, Germany, the entire contents of which are incorporated by reference.

[0100] Determining the longitudinal coordinates of the reflection features, i.e. the depth information, comprises evaluating the reflection images. The evaluation comprises, in particular for each reflection feature, at least one analysis of the respective beam profile using a depth from photon ratio technique. The analysis of the beam profile generally comprises evaluating the beam profile and may comprise at least one mathematical operation, and / or at least one comparison, and / or at least one symmetrization, and / or at least one filtering, and / or at least one normalization. For example, the analysis of the beam profile may comprise at least one of the following steps: histogram analysis, calculation of difference measurements, application of a neural network, application of a machine learning algorithm. The evaluation device may be configured to symmetrize and / or normalize and / or filter the beam profile, in particular to remove noise or asymmetries from the recordings at larger angles, edges, etc. The evaluation device may filter the beam profile by removing high spatial frequencies, such as by spatial frequency analysis and / or median filtering, etc. The aggregation may be performed by the center of intensity of the light spot and by averaging all intensities at the same distance to the center. The evaluation device may be configured to normalize the beam profile to a maximum intensity, in particular to take into account the recorded intensity differences over distance. The evaluation device may be configured to remove contributions from background light from the beam profile, for example by imaging without illumination.

[0101] The evaluation device may be configured to determine at least one first area and at least one second area of ​​each of the reflection features and / or the reflected beam profile of the reflection features in the at least one region of interest, the evaluation device being configured to integrate the first area and the second area.

[0102] The analysis of the beam profile of one of the reflection features may include determining at least one first area and at least one second area of ​​the beam profile. The first area of ​​the beam profile may be area A1 and the second area of ​​the beam profile may be area A2. The evaluation device may be configured to integrate the first area and the second area. The evaluation device may be configured to derive a combined signal, in particular a quotient Q, by one or more of: dividing the integrated first area and the integrated second area; dividing a multiple of the integrated first area and the integrated second area; dividing a linear combination of the integrated first area and the integrated second area. The evaluation device may be configured to determine at least two areas of the beam profile and / or to divide the beam profile into at least two segments having different areas of the beam profile, where overlap of the areas is possible as long as the areas do not coincide. For example, the evaluation device may be configured to determine a plurality of areas, such as two, three, four, five or up to ten areas. The evaluation device may be configured to divide the light spot into at least two areas of the beam profile and / or divide the beam profile into at least two segments comprising different areas of the beam profile. The evaluation device may be configured to determine, for at least two of the areas, integrals of the beam profile over the respective areas. The evaluation device may be configured to compare at least two of the determined integrals. In particular, the evaluation device may be configured to determine at least one first area and at least one second area of ​​the beam profile. As used herein, the term "area of ​​the beam profile" generally refers to any area of ​​the beam profile at the optical sensor location used to determine the quotient Q. The first area of ​​the beam profile and the second area of ​​the beam profile may be adjacent or overlapping areas or both. The first area of ​​the beam profile and the second area of ​​the beam profile may not be congruent in area.For example, the evaluation device may be configured to divide the sensor area of ​​the CMOS sensor into at least two sub-areas, and the evaluation device may be configured to divide the sensor area of ​​the CMOS sensor into at least one left part and at least one right part, and / or at least one upper part and at least one lower part, and / or at least one inner part and at least one outer part. Additionally or alternatively, the camera may comprise at least two light sensors, and the light-sensitive areas of the first light sensor and the second light sensor may be arranged such that the first light sensor is adapted to determine a first area of ​​the beam profile of the reflective feature, and the second light sensor is adapted to determine a second area of ​​the beam profile of the reflective feature. The evaluation device may be adapted to integrate the first area and the second area. The evaluation device may be configured to use at least one predetermined relationship between the quotient Q and the vertical coordinate to determine the vertical coordinate. The predetermined relationship may be one or more of an empirical relationship, a semi-empirical relationship, and an analytically derived relationship. The evaluation device may comprise at least one data store for storing the predetermined relationships, for example a look-up list or a look-up table.

[0103] The first area of ​​the beam profile may essentially include edge information of the beam profile, and the second area of ​​the beam profile may essentially include center information of the beam profile, and / or the first area of ​​the beam profile may essentially include information about the left part of the beam profile and the second area of ​​the beam profile may essentially include information about the right part of the beam profile. The beam profile may have a center, i.e., a maximum of the beam profile and / or a center point of the plateau of the beam profile and / or a geometric center of the light spot, and a falling edge extending from the center. The second area may include an inner area of ​​the cross section, and the first area may include an outer area of ​​the cross section. As used herein, the term "essentially center information" generally refers to a low percentage of edge information, i.e., a low percentage of the intensity distribution corresponding to the edge, compared to the percentage of center information, i.e., a percentage of the intensity distribution corresponding to the center. Preferably, the center information has a percentage of edge information of less than 10%, more preferably less than 5%, and most preferably, the center information does not include edge content. As used herein, the term "essentially edge information" generally refers to a low percentage of center information, compared to the percentage of edge information. The edge information may comprise information of the entire beam profile, in particular the central and edge regions. The edge information may comprise a percentage of central information less than 10%, preferably less than 5%, and more preferably the edge information does not include central content. If the beam profile is close to or around the center and essentially includes central information, at least one area of ​​the beam profile may be determined and / or selected as the second area of ​​the beam profile. If the beam profile includes at least a portion of the falling edge of the cross section, at least one area of ​​the beam profile may be determined and / or selected as the first area of ​​the beam profile. For example, the entire area of ​​the cross section may be determined as the first area.

[0104] Other choices of the first area A1 and the second area A2 may be possible. For example, the first area may essentially include an outer region of the beam profile and the second area may essentially include an inner region of the beam profile. For example, in case of a two-dimensional beam profile, the beam profile may be divided into a left part and a right part, where the first area may essentially include an area of ​​the left part of the beam profile and the second area may essentially include an area of ​​the right part of the beam profile.

[0105] The edge information may include information about the number of photons in a first area of ​​the beam profile, and the center information may include information about the number of photons in a second area of ​​the beam profile. The evaluation device may be configured to determine a surface integral of the beam profile. The evaluation device may be configured to determine the edge information by integration and / or summation of the first area. The evaluation device may be configured to determine the center information by integration and / or summation of the second area. For example, the beam profile may be a trapezoidal beam profile and the evaluation device may be configured to determine the integral of the trapezoid. Furthermore, if a trapezoidal beam profile is assumed, the determination of the edge and center signals may be replaced by an equivalent evaluation utilizing the properties of the trapezoidal beam profile, such as, for example, determining the slope and position of the edges and the height of the central plateau, as well as deriving the edge and center signals by geometrical considerations.

[0106] In one embodiment, A1 may correspond to the entire or complete area of ​​the feature on the optical sensor. A2 may be the central area of ​​the feature on the optical sensor. The central area may be a constant value. The central area may be smaller compared to the total area of ​​the feature. For example, for a circular feature, the central area may have a radius between 0.1 and 0.9 of the total radius of the feature, preferably between 0.4 and 0.6 of the total radius.

[0107] In an embodiment, the illumination pattern may include at least one line pattern. A1 may correspond to an area having a full line width of the line pattern on the light sensor, in particular on the light sensitive area of ​​the light sensor. The line pattern on the light sensor may be expanded and / or displaced compared to the line pattern of the illumination pattern, such that the line width on the light sensor is amplified. In particular, in case of a matrix of light sensors, the line width of the line pattern on the light sensor may vary from one row to another. A2 may be a central area of ​​the line pattern on the light sensor. The line width of the central area may be a constant value, in particular may correspond to the line width of the illumination pattern. The line width of the central area may be small compared to the full line width. For example, the central area may have a line width of 0.1 to 0.9 of the full line width, preferably 0.4 to 0.6 of the full line width. The line pattern may be segmented on the light sensor. Each column of the matrix of light sensors may include center information of the intensity of the central area of ​​the line pattern and edge information of the intensity from the area extending further outward from the central area of ​​the line pattern to the edge area.

[0108] In one embodiment, the illumination pattern may include at least one dot pattern. A1 may correspond to an area having a full radius of the dots of the dot pattern on the light sensor. A2 may be a central area of ​​the dots of the dot pattern on the light sensor. The central area may be a constant value. The central area may have a radius corresponding to the full radius. For example, the central area may have a radius of 0.1 to 0.9 of the full radius, preferably 0.4 to 0.6 of the full radius.

[0109] The illumination pattern may include both at least one dot pattern and at least one line pattern. In addition to or as an alternative to line and dot patterns, other embodiments are possible.

[0110] The evaluation device may be configured to derive the quotient Q by one or more of: dividing the integrated first area by the integrated second area; dividing the integrated first area by a multiple of the integrated second area; or dividing a linear combination of the integrated first area by the integrated second area.

[0111] The evaluation device may be configured to derive the quotient Q by one or more of: dividing the first area by the second area; dividing a multiple of the first area by the second area; or dividing a linear combination of the first area by the second area.

number

[0112] Additionally or alternatively, the evaluation device may be adapted to determine one or both of center and edge information from at least one slice or cut of the light spot. This may be achieved, for example, by replacing the surface integral of the quotient Q by a line integral along the slice or cut. To improve accuracy, several slices or cuts through the light spot may be used for averaging. In case of an elliptical spot profile, averaging over several slices or cuts may provide improved distance information.

[0113] For example, in the case of a light sensor having a matrix of pixels, the evaluation device - determining the pixels having the highest sensor signals and forming at least one center signal; - evaluating the matrix of sensor signals and forming at least one sum signal; - determining a quotient Q by combining the center signal and the sum signal; - determining at least one longitudinal coordinate z of the object by evaluating the quotient Q, thereby evaluating the beam profile.

[0114] The term "central signal" generally refers to at least one sensor signal that contains essentially central information of the beam profile. As used herein, the term "highest sensor signal" refers to one or both of the local maxima or maxima of the region of interest. For example, the central signal may be the signal of a pixel with the highest sensor signal among a plurality of sensor signals generated by the pixels of the entire matrix or a region of interest in the matrix, which may be predetermined or determinable in the image generated by the pixels of the matrix. The central signal may originate from a single pixel or from a group of optical sensors, in the latter case, as an example, the sensor signals of the group of pixels may be added, integrated or averaged to determine the central signal. The group of pixels from which the central signal originates may be a group of adjacent pixels, such as pixels that are located less than a predetermined distance away from the actual pixel with the highest sensor signal, or may be a group of pixels that generate sensor signals that are within a predetermined range from the highest sensor signal. The group of pixels from which the central signal originates may be selected to be as large as possible to allow a maximum dynamic range. The evaluation device may be adapted to determine the central signal by integrating a plurality of sensor signals, for example a plurality of pixels around the pixel with the highest sensor signal. For example, the beam profile may be a trapezoidal beam profile and the evaluation device may be adapted to determine the integral of the trapezoid, in particular the integral of the trapezoidal plateau.

[0115] As mentioned above, the central signal may generally be a single sensor signal, such as a sensor signal from a pixel at the center of the light spot, or may be a combination of multiple sensor signals, such as a combination of sensor signals arising from pixels at the center of the light spot, or may be a secondary sensor signal derived by processing sensor signals derived by one or more of the aforementioned possibilities. The determination of the central signal may be performed electronically, as comparison of sensor signals is fairly easily performed by conventional electronics, or may be performed fully or partially by software. Specifically, the center signal may be selected from the group consisting of: the highest sensor signal; the average of a group of sensor signals that are within a predetermined tolerance range of the highest sensor signal; the average of sensor signals from a group of pixels including the pixel with the highest sensor signal and a predetermined group of adjacent pixels; the sum of sensor signals from a group of pixels including the pixel with the highest sensor signal and a predetermined group of adjacent pixels; the sum of a group of sensor signals that are within a predetermined tolerance range of the highest sensor signal; the average of a group of sensor signals that exceed a predetermined threshold; the sum of a group of sensor signals that exceed a predetermined threshold; an integral of sensor signals from a group of optical sensors including the optical sensor with the highest sensor signal and a predetermined group of adjacent pixels; an integral of a group of sensor signals that are within a predetermined tolerance range of the highest sensor signal; an integral of a group of sensor signals that exceed a predetermined threshold.

[0116] Similarly, the term "sum signal" generally refers to a signal that essentially contains edge information of the beam profile. For example, the sum signal can be derived by summing, integrating, or averaging the sensor signals of the entire matrix or of a region of interest in the matrix, the region of interest being pre-determined or determinable in the image generated by the optical sensor of the matrix. When summing, integrating, or averaging the sensor signals, the actual optical sensor where the sensor signal is generated may be excluded from the summing, integrating, or averaging, or may be included in the summing, integrating, or averaging. The evaluation device may be adapted to determine the sum signal by integrating the signals of the entire matrix or of the region of interest in the matrix. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be adapted to determine the integral of the entire trapezoid. Furthermore, if a trapezoidal beam profile is assumed, the determination of the edge signal and the center signal may be replaced by an equivalent evaluation that utilizes the properties of the trapezoidal beam profile, such as, for example, determining the slope and position of the edges and the height of the central plateau, and deriving the edge signal and the center signal by geometrical considerations.

[0117] Similarly, the center signal and the edge signal can also be determined by using a segment of the beam profile, such as a circular segment of the beam profile. For example, the beam profile can be divided into two segments by a dividing line or chord that does not pass through the center of the beam profile. Thus, one segment will substantially contain edge information and the other segment will substantially contain center information. For example, the edge signal can be further subtracted from the center signal to further reduce the amount of edge information in the center signal.

[0118] The quotient Q may be a signal generated by combining the center signal and the sum signal. In particular, the determination may include one or more of: forming a quotient of the center signal and the sum signal, or its inverse; forming a quotient of a multiple of the center signal and a multiple of the sum signal, or its inverse; forming a quotient of a linear combination of the center signals and a linear combination of the sum signal, or its inverse. Additionally or alternatively, the quotient Q may include any signal or combination of signals that includes at least one item of information regarding the comparison between the center signal and the sum signal.

[0119] As used herein, the term "vertical coordinate of a reflectance feature" refers to the distance between the light sensor and the point of the scene emitting the corresponding illumination feature. The evaluation device may be configured to use at least one predetermined relationship between the quotient Q and the vertical coordinate to determine the vertical coordinate. The predetermined relationship may be one or more of an empirical relationship, a semi-empirical relationship, and an analytically derived relationship. The evaluation device may comprise at least one data storage device for storing the predetermined relationship, such as, for example, a look-up list or a look-up table.

[0120] The evaluation unit may be configured to execute at least one depth from photon ratio algorithm that calculates distances for all zeroth and higher order reflectance features.

[0121] In a further aspect, the present invention discloses a method for detecting the material of at least one object, in which a detector according to the present invention is used. The method comprises the following steps: - illuminating an object with at least one illumination pattern generated by at least one projector, the illumination pattern comprising a plurality of illumination features, the illumination features having a first wavelength; - illuminating an object with scene illumination produced by at least one flood light source, the flood light source being configured to emit scene illumination having a second wavelength different from the first wavelength; - imaging, using a sensor element, at least one reflected image comprising a plurality of reflected features produced by the object in response to the illumination pattern, each of the reflected features comprising a beam profile; - imaging at least one scene image of an object illuminated by said scene illumination using sensor elements; - determining at least one first material information of the object by evaluating the beam profile of at least one of the reflective features using an evaluation device; - determining at least one second material information of the object by evaluating the scene image using the evaluation device; - determining, using the evaluation device, a material of the object using the first material information and the second material information; Includes.

[0122] The method steps may be performed in a given order or in a different order. Furthermore, there may be one or more additional method steps not listed. Furthermore, one, more than one, or even all of the method steps may be performed repeatedly. For details, options and definitions, reference may be made to the detectors described above. Thus, in particular, as outlined above, the method may include using a detector according to the invention, such as according to one or more of the embodiments given above or given in more detail below.

[0123] The method may further include evaluating the sensor signals, thereby determining a combined signal Q, and determining a longitudinal coordinate of the at least one reflection feature by analysis of the respective beam profile. The analysis of the beam profile may include evaluating the combined signal Q from the sensor signals associated with the reflection feature. The evaluation device may be configured to use at least one predetermined relationship between the combined signal Q and the longitudinal coordinate to determine the longitudinal coordinate.

[0124] In a further aspect, a computer program is disclosed comprising computer executable instructions for carrying out the method according to the invention when the program is run on a computer or a computer network.

[0125] In a further aspect of the invention, the use of a detector according to the invention, such as according to one or more embodiments given above or in more detail below, is proposed for use purposes selected from the group consisting of: localization in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photography applications; imaging or camera applications; mapping applications for generating a map of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotics applications; quality control applications; manufacturing applications; automotive applications.

[0126] For example, the detector can be used in vehicle applications such as driver monitoring, personalized vehicles, etc.

[0127] For further uses of the detectors and devices of the present invention, reference is made to WO2018 / 091649A1, WO2018 / 091638A1 and WO2018 / 091640A1 and C. Lennartz, F. Schick, S. Metz, "Whitepaper - Beam Profile Analysis for 3D imaging and material detection", April 28, 2021, Ludwigshafen, Germany, the contents of which are incorporated by reference.

[0128] Overall, in the context of the present invention, the following embodiments are considered to be preferred: Embodiment 1. A detector for material detection of at least one object, said detector comprising: - at least one projector for illuminating at least one object with at least one illumination pattern, the illumination pattern including a plurality of illumination features, the illumination features having a first wavelength; and - at least one flood light source configured for scene illumination, the flood light source configured to emit scene illumination having a second wavelength different from the first wavelength; - comprising a matrix of light sensors, each of said light sensors being at least one sensor element having a light-sensitive area, each light sensor being designed to generate at least one sensor signal in response to illumination of its respective light-sensitive area by a light beam propagating from said object to said detector, the sensor elements are configured to image at least one reflected image comprising a plurality of reflective features generated by the object in response to the illumination pattern, each of the reflective features comprising a beam profile; at least one sensor element configured to image at least one scene image of the object illuminated by the scene illumination; at least one evaluation device, the evaluation device is configured to determine at least one first material information of the object by evaluating the beam profile of at least one of the reflective features; the evaluation device is configured to determine at least one second material information of the object by evaluating the scene image; at least one evaluation device configured to determine a material of the object using the first material information and the second material information; The detector comprises:

[0129] Embodiment 2. The detector of the preceding embodiment, wherein the evaluation device is configured to use the second material information as an additional information channel for distinguishing materials.

[0130] Embodiment 3. The detector of any one of the preceding embodiments, wherein the detector is configured to distinguish between biological and non-biological materials.

[0131] Embodiment 4. The detector of any one of the preceding embodiments, wherein the detector is configured to distinguish between skin and non-skin.

[0132] Embodiment 5. The detector of any one of the preceding embodiments, wherein the first wavelength and the second wavelength are different wavelengths in the infrared spectral range.

[0133] Embodiment 6. The detector of any one of the preceding embodiments, wherein the first wavelength is 940 nm and the second wavelength is 850 nm.

[0134] Embodiment 7. The detector of any one of the preceding embodiments, wherein the projector comprises a plurality of emitters, the emitters comprising at least one emitter selected from the group consisting of at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one transistor laser, at least one diode pumped laser, at least one distributed feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one gallium arsenide laser, at least one semiconductor ring laser, at least one extended cavity diode laser, at least one vertical cavity surface emitting laser (VCSEL).

[0135] Embodiment 8. The detector of any one of the preceding embodiments, wherein the flood light source comprises at least one light emitting diode (LED).

[0136] Embodiment 9. A detector as described in any one of the preceding embodiments, wherein the evaluation device is configured to determine a vertical coordinate of at least one of the reflection features by analysis of a respective beam profile, the analysis of the beam profile including evaluating a combined signal Q from a sensor signal associated with the reflection feature, and the evaluation device is configured to use at least one predetermined relationship between the combined signal Q and the vertical coordinate to determine the vertical coordinate.

[0137] Embodiment 10. A detector as described in the preceding embodiment, wherein the evaluation device is configured to derive a combined signal Q by one or more of dividing the sensor signal, dividing a multiple of the sensor signal, or dividing a linear combination of the sensor signals.

[0138] Embodiment 11. A detector according to any one of the preceding embodiments, wherein the sensor element comprises at least one CCD chip and / or at least one CMOS chip.

[0139] Embodiment 12. A method for detecting the material of at least one object using at least one detector according to any one of the preceding embodiments, the method comprising the steps of: - illuminating an object with at least one illumination pattern generated by at least one projector, the illumination pattern comprising a plurality of illumination features, the illumination features having a first wavelength; - illuminating an object with scene illumination produced by at least one flood light source, the flood light source being configured to emit scene illumination having a second wavelength different from the first wavelength; - imaging, using a sensor element, at least one reflected image comprising a plurality of reflected features produced by the object in response to the illumination pattern, each of the reflected features comprising a beam profile; - imaging at least one scene image of an object illuminated by said scene illumination using sensor elements; - determining at least one first material information of the object by evaluating the beam profile of at least one of the reflective features using an evaluation device; - determining at least one second material information of the object by evaluating the scene image using the evaluation device; - determining, using the evaluation device, a material of the object using the first material information and the second material information; A method comprising:

[0140] Embodiment 13. The method according to the preceding embodiment, further comprising evaluating a sensor signal, thereby determining a combined signal Q, and determining a vertical coordinate of at least one of the reflection features by analysis of a respective beam profile, the analysis of the beam profile comprising evaluating the combined signal Q from a sensor signal associated with the reflection feature, and the evaluation device configured to use at least one predetermined relationship between the combined signal Q and the vertical coordinate to determine the vertical coordinate.

[0141] Embodiment 14. Use of a detector described in any one of the preceding embodiments relating to a detector, wherein the purpose of use is selected from the group consisting of: position measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photography applications; imaging or camera applications; mapping applications for generating a map of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotics applications; quality control applications; manufacturing applications; automotive applications.

[0142] As used herein, the terms "having", "comprises" or "including", or any grammatical variants thereof, are used in a non-exclusive manner. Thus, these terms may refer both to the situation where, apart from the features introduced by these terms, no further features are present in the entity described in this context, and to the situation where one or more further features are present. As an example, the expressions "A has B", "A comprises B" and "A includes B" may refer both to the situation where no other elements are present in A apart from B (i.e., A is solely and exclusively composed of B) and to the situation where, in addition to B, one or more elements are present in entity A, such as element C, elements C and D, or further elements.

[0143] Furthermore, it should be noted that the terms "at least one," "one or more," or similar language indicating that a feature or element may be present more than one time are typically used only once when introducing each feature or element. It should be noted that in most cases below, when referring to each feature or element, the language "at least one" or "one or more" will not be repeated, despite the fact that the feature or element may appear more than one time.

[0144] Furthermore, as used herein, the terms "preferably", "more preferably", "particularly", "more particularly", "particularly", "more particularly" or similar terms are used in connection with any feature without limiting the possibility of alternatives. Thus, features introduced by these terms are optional features and are not intended to limit the scope of the claims in any way. The invention can be practiced with alternative features, as will be recognized by those skilled in the art. Similarly, features introduced by "in one embodiment of the invention" or similar expressions are intended to be optional features, without any limitation on alternative embodiments of the invention, without any limitation on the scope of the invention, and without any limitation on the possibility of combining the feature introduced in such a way with other optional or non-optional features of the invention. [Brief description of the drawings]

[0145] Further optional details and features of the invention are evident from the following description of preferred exemplary embodiments in conjunction with the dependent claims. In this context, certain features can be implemented in isolation or in combination with other features. The invention is not limited to the exemplary embodiments. The exemplary embodiments are illustrated diagrammatically in the figures. Identical reference numerals in the individual figures refer to identical elements or elements with identical functions or elements which correspond to each other in terms of function.

[0146] Specifically, in the drawing: [Figure 1] FIG. 1 illustrates an embodiment of a detector according to the present invention. [Diagram 2] FIG. 1 shows a mobile device equipped with a detector. [Diagram 3] FIG. 13 is a diagram showing a reflectance distribution. [Figure 4] FIG. 1 illustrates an embodiment of a method according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0147] Detailed Description of the Preferred Embodiments Fig. 1 shows very diagrammatically an embodiment of a detector 110 for material detection of at least one object 112 according to the present invention. An exemplary object 112 is shown in Fig. 2. As further shown in Fig. 2, the detector 110 may be either attached to or integrated into a mobile device 114, such as a mobile phone or a smartphone. The detector 110 may be integrated into the mobile device 114, for example, within a housing of the mobile device 114. The mobile device 114 is one or more of a mobile communication device, such as a mobile phone or a smartphone, a tablet computer, a portable computer.

[0148] The detector 110 comprises at least one projector 116 for illuminating the object 112 with at least one illumination pattern. The illumination pattern includes a plurality of illumination features. The illumination features have a first wavelength.

[0149] The illumination pattern may be selected from the group consisting of at least one dot pattern; at least one line pattern; at least one stripe pattern; at least one checkerboard pattern; at least one pattern including a periodic or non-periodic arrangement of features. The illumination pattern may include regular and / or regular and / or periodic patterns such as triangular patterns, rectangular patterns, hexagonal patterns, or even convex tiling patterns. The illumination pattern may exhibit at least one illumination feature selected from the group consisting of at least one dot; at least one line; at least two lines, such as parallel or intersecting lines; at least one dot and one line; at least one arrangement of periodic or non-periodic features; at least one feature of any shape. The illumination pattern may include at least one pattern selected from the group consisting of: at least one dot pattern, in particular a pseudo-random dot pattern; a random dot pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one known feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including a convex uniform tiling; at least one line pattern including at least one line; at least one line pattern including at least two lines, such as parallel or intersecting lines. For example, the projector 116 may be configured to generate and / or project a point cloud feature or a non-point feature. For example, the projector 116 may be configured to generate a point cloud feature or a non-point feature such that the illumination pattern may include multiple point cloud features or non-point features. The illumination pattern may include a regular and / or regular and / or periodic pattern, such as a triangular pattern, a rectangular pattern, a hexagonal pattern, or even a pattern including a convex tiling. The illumination pattern may contain as many features per area as possible, such that a hexagonal pattern may be preferred.The distance between the two features of each illumination pattern and / or the area of ​​the at least one illumination feature may depend on a circle of confusion in the image determined by the at least one detector. For example, the illumination pattern may include a periodic dot pattern.

[0150] Projector 116 includes at least one array 118 of emitters 120, as shown, for example, in Figure 3. Each of emitters 120 is configured to generate at least one light beam. Each of the emitters 120 may be and / or include at least one element selected from the group consisting of at least one laser source, such as at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one gallium arsenide laser, at least one transistor laser, at least one diode pumped laser, at least one distributed feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one semiconductor ring laser, at least one vertical cavity surface emitting laser (VCSEL), in particular at least one VCSEL array; and at least one non-laser light source, such as at least one LED or at least one light bulb.

[0151] The array 118 of emitters 120 may be a two-dimensional or one-dimensional array. The array 118 may include a plurality of emitters 120 arranged in a matrix. As shown in FIG. 3, the matrix may be or may include a rectangular matrix having one or more rows and one or more columns. The rows and columns may be arranged in a rectangular manner. However, other arrangements are possible, such as non-rectangular arrangements. As an example, a circular arrangement is also possible, where the elements are arranged in concentric circles or ellipses around a central point.

[0152] For example, the emitter 120 may be an array 118 of VCSELs. Examples of VCSELs are described, for example, in en.wikipedia.org / wiki / Vertical-cavity_surface-emitting_laser. VCSELs are generally known to those skilled in the art, for example from WO2017 / 222618A. Each of the VCSELs is configured to generate at least one light beam. The VCSELs may be disposed on a common substrate or may be disposed on different substrates. The array 118 may comprise up to 2500 VCSELs. For example, the array 118 may comprise 38×25 VCSELs, such as a high-power array of 3.5 W. For example, the array may comprise 10×27 VCSELs at 2.5 W. For example, the array may comprise 96 VCSELs at 0.9 W. The size of the array may be up to 2 mm×2 mm, for example, for a 2500 element array.

[0153] The illumination characteristic has a first wavelength. The light beam emitted by each emitter 120 may have a wavelength between 300 and 1100 nm, preferably between 500 and 1100 nm. For example, light in the infrared spectral range may be used, for example in the range of 780 nm to 3.0 μm. In particular, light in the part of the near infrared range in which silicon photodiodes are applicable may be used, in particular in the range of 700 nm to 1100 nm. The emitter 120 may be configured to generate at least one illumination pattern in the infrared range, in particular in the near infrared range. Using light in the near infrared range may enable light that is not detected or is only weakly detected by the human eye, but is still detectable by a silicon sensor, in particular a standard silicon sensor.

[0154] For example, the first wavelength may be 940 nm. This wavelength is considered advantageous because solar radiation on Earth has a minimum irradiance at this wavelength, as described, for example, in CIE 085-1989 "Solarspectral Irradiance". For example, the emitter 120 may be an array of VCSELs. The VCSELs may be configured to emit light beams in the wavelength range of 800 nm to 1000 nm. For example, the VCSELs may be configured to emit light beams at 808 nm, 850 nm, 940 nm, or 980 nm. Preferably, the VCSELs emit light at 940 nm.

[0155] The detector comprises at least one flood light source 122 configured for scene illumination. The flood light source 122 is configured to emit scene illumination having a second wavelength different from the first wavelength. The flood light source 122 may be adapted to provide at least one illumination light beam for illuminating the object. The flood light source 122 is configured for scene illumination. The scene illumination may be a diffuse illumination and / or a uniform illumination of the scene. The scene may include at least one object 112 and a surrounding environment. The flood light source 122 may be adapted to directly or indirectly illuminate the object 112, the illumination being reflected or scattered by a surface of the object 112 and thereby directed at least in part to a sensor element of the detector 110. The flood light source 122 may be adapted to illuminate the object 112 reflecting the light beam, for example by directing a light beam towards the object 112.

[0156] The flood light source 122 is configured to emit scene illumination having a second wavelength different from the first wavelength. The flood light source 122 may comprise at least one light emitting diode (LED). The flood light source 122 may comprise a single light source, or multiple light sources. By way of example, the light emitted by the flood light source 122 may have a wavelength of 300-1100 nm, in particular 500-1100 nm. Additionally or alternatively, light in the infrared spectral range may be used, for example light in the range of 780 nm-3.0 μm. In particular, a part of the near infrared range where silicon photodiodes are applicable, in particular light in the range of 700 nm-1100 nm, may be used. The flood light source 122 may be configured to emit light of a single wavelength. In particular, the wavelength may be in the near infrared range. In other embodiments, the flood light source 122 may be adapted to emit light at multiple wavelengths to enable additional measurements in other wavelength channels.

[0157] The first and second wavelengths can be selected to distinguish between the materials. For example, two or more materials can have similar reflectances at a first wavelength, but different reflectances at a second wavelength. The first and second wavelengths can be different wavelengths in the infrared spectral range. For example, the first wavelength can be 940 nm and the second wavelength can be 850 nm.

[0158] The detector 110 may comprise at least one control unit 124 configured to control the light emission of the projector 116 and / or the flood light source 122. For example, the projector 116 comprises at least one shutter 126. The shutter 126 may be an optical element configured to block light passing therethrough. The shutter 126 may be configured to temporarily block light from one of the emitters 120 from passing therethrough.

[0159] The detector 110 comprises at least one sensor element 128 with a matrix of light sensors 132, for example as part of at least one camera 130. The light sensors 132 each have a light-sensitive area. Each light sensor 132 is designed to generate at least one sensor signal in response to illumination of the respective light-sensitive area by a reflected light beam propagating from the object 112 to the sensor element 128. The sensor element 128 may be or comprise at least one imaging element configured to record or capture spatially resolved one-, two- or three-dimensional optical data or information. As an example, the sensor element 128 may comprise at least one camera chip, such as at least one CCD chip and / or at least one CMOS chip, configured to record an image.

[0160] The light sensor 132 may in particular be or may include at least one photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, most preferably a silicon photodetector. In particular, the light sensor 132 may have sensitivity in the infrared spectral range. All pixels of the matrix, or at least a group of the light sensors of the matrix, may in particular be identical. Groups of identical pixels of the matrix may in particular be provided for different spectral ranges, or all pixels may be identical with respect to their spectral sensitivity. Furthermore, the pixels may be identical with respect to their size and / or their electronic or optoelectronic properties. In particular, the light sensor 132 may be or may include at least one inorganic photodiode with sensitivity in the infrared spectral range, preferably in the range of 700 nm to 3.0 micrometers. In particular, the light sensor 132 may have sensitivity in the part of the near infrared range, in particular in the range of 700 nm to 1100 nm, where silicon photodiodes are applicable. Infrared light sensors that may be used for the light sensor may be commercially available infrared light sensors, such as, for example, infrared light sensors available under the brand name Hertzstueck® from trinamiX GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, by way of example, the light sensor 132 may include at least one light sensor of the intrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, and HgCdTe photodiodes. Additionally or alternatively, the light sensor 132 may include at least one light sensor of the extrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge:Au photodiodes, Ge:Hg photodiodes, Ge:Cu photodiodes, Ge:Zn photodiodes, Si:Ga photodiodes, and Si:As photodiodes.Additionally or alternatively, the optical sensor 132 may include at least one photoconductive sensor, such as a PbS or PbSe sensor, a bolometer, preferably a bolometer selected from the group consisting of a VO bolometer and an amorphous Si bolometer.

[0161] The light sensor 132 may be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. Specifically, the light sensor may be sensitive in the visible spectral range of 500 nm to 780 nm, most preferably 650 nm to 750 nm, or 690 nm to 700 nm. Specifically, the light sensor 132 may be sensitive in the near infrared range. Specifically, the light sensor 132 may be sensitive in the portion of the near infrared range where silicon photodiodes are applicable, particularly in the range of 700 nm to 1000 nm. The light sensor 132 may be sensitive in the infrared spectral range, specifically in the range of 780 nm to 3.0 μm. For example, the light sensors may each independently be or include at least one element selected from the group consisting of a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. For example, the light sensor 132 may be or may include at least one element selected from the group consisting of a CCD sensor element, a CMOS sensor element, a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. Any other type of photosensitive element may be used. The photosensitive element may generally be made entirely or partially of inorganic materials and / or may be made entirely or partially of organic materials. Most commonly, one or more photodiodes may be used, such as commercially available photodiodes, e.g., inorganic semiconductor photodiodes.

[0162] The sensor element 128 comprises a matrix of pixels. Thus, by way of example, the light sensor 132 may be part of or constitute a pixelated optical device. For example, the light sensor 132 may be and / or constitute at least one CCD and / or CMOS device. By way of example, the light sensor 132 may be part of or constitute at least one CCD and / or CMOS device having a matrix of pixels, each pixel forming a light-sensitive area. The sensor element may be formed as a single device or as a combination of multiple devices. The matrix may in particular be or include a rectangular matrix having one or more rows and one or more columns. The rows and columns may in particular be arranged in a rectangular manner. However, other arrangements are possible, such as non-rectangular arrangements. By way of example, a circular arrangement is also possible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, the matrix may be a single row of pixels. Other arrangements are possible.

[0163] The pixels of the matrix may in particular be equal in size, sensitivity, and / or other optical, electrical, mechanical properties. The light-sensitive areas of all light sensors 132 of the matrix may in particular be arranged in a common plane, which may preferably face the scene, such that a light beam propagating from the object 112 to the detector 110 generates a light spot on said common plane. The light-sensitive areas may in particular be arranged on the surface of the respective light sensor 132. However, other embodiments are also possible. The sensor elements 128 may, for example, comprise at least one CCD and / or CMOS device. By way of example, the sensor elements 128 may be part of or may constitute a pixelated optical device. By way of example, the light sensor 132 may be part of or may constitute at least one CCD and / or CMOS device having a matrix of pixels, each pixel forming a light-sensitive area.

[0164] The sensor elements 128 are configured to image at least one scene image of the object 112 illuminated by the scene illumination. The scene image may be generated in response to diffuse and / or uniform illumination of the object 112 by the scene illumination. The scene image may not include reflective features generated by the illumination pattern. The scene image may be at least one two-dimensional image.

[0165] The sensor element 128 is configured to image at least one reflected image including a plurality of reflective features generated by the object 112 in response to the illumination pattern. The reflective features may be features in the image plane generated by the scene in response to the illumination, specifically at least one illumination feature. Each of the reflective features includes at least one beam profile 134, also called a reflected beam profile (see exemplarily FIG. 2). The beam profile 134 of the reflective feature may generally refer to at least one intensity distribution of the reflective feature, such as a light spot on the light sensor 132, as a function of pixels. The beam profile 134 may be selected from the group consisting of a linear combination of a trapezoidal beam profile; a triangular beam profile; a conical beam profile and a Gaussian beam profile.

[0166] The detector 110 comprises at least one evaluation device 136. The evaluation device 136 is configured to determine at least one first material information of the object by evaluating at least one beam profile 134 of the reflection feature.

[0167] The evaluation unit 136 may be configured to perform an evaluation of the reflected image. The evaluation of the reflected image may include identifying a reflection feature in the reflected image. The evaluation unit 136 may be configured to perform at least one image analysis and / or image processing to identify the reflection feature. The image analysis and / or image processing may use at least one feature detection algorithm. The image analysis and / or image processing may include: filtering; selecting at least one region of interest; forming a difference image between an image generated by the sensor signal and at least one offset; inverting the sensor signal by inverting the image generated by the sensor signal; forming a difference image between images generated by the sensor signal at different times; background correction; decomposition into color channels; decomposition into hue; saturation; luminance channel; frequency decomposition; singular value decomposition; applying a blob detector; applying a corner detector; applying a determinant of a Hessian filter; applying principal curvature based region detector; applying maximally stable extremum region detector; applying a generalized Hough transform; applying a ridge detector; applying an affine invariant feature detector; applying an affine adaptive interest point operator; applying a Harris affine region detector; applying a Hessian affine region detector; applying a scale invariant feature transform The method may include one or more of: applying a scale-space extremum detector; applying a local feature detector; applying a fast robust feature algorithm; applying a histogram of gradient position and orientation algorithm; applying a histogram of oriented gradient descriptors; applying a Deriche edge detector; applying a differential edge detector; applying a space-time interest point detector; applying a Moravec corner detector; applying a Canny edge detector; applying a Laplace operator of a Gaussian filter; applying a differential Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Schall operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; and generating a binary image. The region of interest may be determined manually by a user or automatically, such as by recognizing features in an image generated by an optical sensor.

[0168] The evaluation device 136 can be configured to determine a beam profile 134 of each reflection feature. Determining the beam profile 134 can include identifying at least one reflection feature provided by the optical sensor 132 and / or selecting at least one reflection feature provided by the optical sensor 132 and evaluating an intensity distribution of at least one of the reflection features. As an example, a region of the matrix can be used and evaluated to determine an intensity distribution, such as a three-dimensional intensity distribution or a two-dimensional intensity distribution, such as along an axis or line through the matrix. As an example, the center of illumination by the light beam can be determined, such as by determining at least one pixel with the highest illumination, and a cross-sectional axis can be selected through the center of illumination. The intensity distribution can be an intensity distribution as a function of coordinate along this cross-sectional axis through the center of illumination. Other evaluation algorithms are possible.

[0169] The evaluation device 136 is configured to determine at least one first material information of the object by evaluating at least one beam profile 134 of the reflection features. The evaluation device 136 may be configured to determine at least one first material information of the object 112 by evaluating at least three or more beam profiles of the reflection features, in particular the beam profiles of all the reflection features.

[0170] The evaluation device 136 can be configured to identify the reflection characteristic as being generated by an item having a particular material characteristic if its reflected beam profile meets at least one predetermined or predefined criterion. The at least one predetermined or predefined criterion can be at least one characteristic and / or value suitable for distinguishing material characteristics. The predetermined or predefined criterion can be or include at least one predetermined or predefined value and / or threshold and / or threshold range referring to the material characteristic. The reflection characteristic can indicate that it is generated by an item having a particular material characteristic if its reflected beam profile meets at least one predetermined or predefined criterion. The indication can include any indication, such as an electronic signal and / or at least one visual or audio indication, for example via a display of the mobile device 114.

[0171] Determining the at least one material information may include assigning at least one material property to each reflectance characteristic. The evaluation device 136 may include at least one database including a list and / or table of predefined and / or predetermined material properties, e.g., a look-up list or look-up table. The list and / or table of material properties may be determined and / or generated by performing at least one test measurement, e.g., by performing a material test using a sample with known material properties. The list and / or table of material properties may be determined and / or generated at the manufacturer's site and / or by a user. The material properties may be further assigned to a material classifier, such as a material name, one or more of material groups, such as biological or non-biological, translucent or non-translucent, metal or non-metal, fur or non-fur, carpet or non-carpet, reflective or non-reflective, specular or non-specular, foam or non-foam, roughness group, etc. The evaluator 136 may include at least one database including lists and / or tables containing material names and / or material groups associated with material properties.

[0172] To determine the first material information, beam profile analysis may be used. Specifically, beam profile analysis utilizes the reflectance properties of coherent light projected onto the surface of an object to classify materials. Material classification may be performed as described in WO2020 / 187719, EP application 20159984.2 filed February 28, 2020 and / or EP application 20154961.5 filed January 31, 2020, and C. Lennartz, F. Schick, S. Metz, "Whitepaper-Beam Profile Analysis for 3D imaging and material detection," April 28, 2021, Ludwigshafen, Germany, the entire contents of which are incorporated by reference. In particular, a periodic grid of laser spots is projected, for example a hexagonal grid as described in EP application 20170905.2 filed on 22 April 2020, and a reflected image is recorded with a camera. Analysing the beam profile of each reflected feature recorded by the sensor element can be performed by feature-based methods and / or by using convolutional neural network based classification of the reflected features of the reflected image. Feature-based methods can be used in combination with machine learning methods that allow parameterisation of the classification model. Convolutional neural networks can be utilized to classify materials by using the reflected images as input.

[0173] The evaluation device 136 is configured to determine at least one second material information of the object by evaluating the scene image. The evaluation of the scene image may include determining at least one reflectance value of at least one region of interest. The region of interest may be manually selected by a user and / or automatically selected by using at least one object detection algorithm. The evaluation may include comparing the reflectance value to at least one predetermined or predefined value and / or threshold and / or threshold range referring to a material property.

[0174] The determination of the second material information may be performed using at least one parameterized classification model. The parameterized model may be a combined parameterized classification model for determining the first material information and the second material information, and / or at least one further parameterized classification model in addition to the parameterized model used in particular for determining the first material information. The combined parameterized classification model and / or the further parameterized classification model may be at least partially data-driven. For example, the combined parameterized classification model and / or the further parameterized classification model may be based on experimental data. For example, the training may include using at least one training data set, the training data set including images, in particular scene images of a plurality of items with known material properties of the further parameterized model, and / or scene images and reflection images of the combined parameterized classification model.

[0175] The determination of the second material information may be performed consecutively or simultaneously with the determination of the first material information. At least the second material information can be used separately and / or together with the first material information. The evaluation of the scene images can be performed consecutively or simultaneously with the evaluation of the reflected images using at least one parameterized classification model. The evaluation unit 136 may be configured to determine a final material classification. The final material classification may be a combination of the results of the parameterized classification model using the first material information and the second material information.

[0176] For example, the evaluation device 136 can be configured to sequentially use the first material information and at least one parameterized classification model providing at least one material confidence to determine whether at least one second material information needs to be evaluated. For example, if skin or non-skin needs to be classified and the first material information rejects skin with high confidence, the second material information may not be needed for the final classification decision. If the second material information is used, the decision may be evaluated using the at least one parameterized classification model.

[0177] For example, the evaluations can be performed simultaneously, for example, using combined parameterized classification models.

[0178] The evaluation unit 136 is configured to use the first material information and the second material information to determine a combination of parameterized classification model results. The combination of the first material information and the second material information may enable more reliable material classification.

[0179] The first material information may be a material property characterizing the material of the object. The evaluation device 136 may be configured to use the second material information as an additional information channel to distinguish materials, such as in ambiguous cases. Using an additional information channel is advantageous in the case of difficult targets, such as distinguishing between skin and silicon. The detector 110 may be configured to distinguish between biological and non-biological materials. The detector 110 may be configured to distinguish between skin and non-skin.

[0180] The evaluation device 136 is configured to determine the material of the object using the first material information and the second material information. The evaluation device 136 may be configured to determine the material of the object 112 by using only the first material information. This may be suitable for many materials. However, for different targets, i.e. targets with similar reflectivity at the first wavelength, the second material information can be taken into account by the evaluation device 136 to distinguish the materials. This significantly improves the reliability of material detection and increases the possibility of identifying spoofing attacks.

[0181] The evaluation device 136 is configured to determine at least one vertical coordinate of the reflection features by analyzing the beam profile of each of the reflection features. The evaluation device 136 determines at least one vertical coordinate z for each of the reflection features by analyzing the beam profile of each of the reflection features. DPR The evaluation unit 136 may be configured to determine the vertical coordinate z of the reflective feature using the so-called photon ratio to depth technique, also called beam profile analysis. DPR For Depth from Photon Ratio (DPR) techniques, reference is made to WO2018 / 091649A1, WO2018 / 091638A1 and WO2018 / 091640A1, as well as C. Lennartz, F. Schick, S. Metz, “Whitepaper-Beam Profile Analysis for 3D imaging and material detection,” April 28, 2021, Ludwigshafen, Germany, the entire contents of which are incorporated by reference.

[0182] Figure 3 shows the distribution of reflectance R as a function of wavelength λ. The first and second wavelengths are denoted as λ1 and λ2, respectively. The distribution is shown for two different materials, shown as dotted and solid lines. Figure 3 shows that even though the two materials have similar values ​​at the first wavelength λ1, they can be clearly distinguished using the second wavelength λ2.

[0183] 4 shows an embodiment of a material detection method according to the invention, in which a detector 110 according to the invention is used. The method comprises the following steps: - illuminating the object 112 with at least one illumination pattern generated by at least one projector 116 (denoted by reference numeral 138), the illumination pattern comprising a plurality of illumination features, the illumination features having a first wavelength, and imaging using a sensor element 128 at least one reflected image comprising a plurality of reflected features generated by the object 112 in response to the illumination pattern, each of the reflected features comprising a beam profile 134; - illuminating the object 112 with scene illumination produced by at least one flood light source 122 (depicted with reference numeral 140), said flood light source 122 being configured to emit scene illumination having a second wavelength different from said first wavelength, and imaging at least one scene image of the object illuminated by said scene illumination using a sensor element 128; - determining at least one first material information of said object by evaluating at least one beam profile 134 of said reflection features using an evaluation device 136 (depicted by reference number 142); - determining at least one second material information of said object 112 by evaluating said scene image using an evaluation device 136 (denoted by reference number 144); - determining, using an evaluation device 136 (reference numeral 146), the material of said object 112 using said first material information and said second material information; Includes.

[0184] The method steps may be performed in a given order or in a different order. Additionally, there may be one or more additional method steps not listed. Additionally, one, more than one, or even all of the method steps may be performed repeatedly. For details, options, and definitions, reference may be made to the detector 110 described with respect to FIGS. 1 and 2. [Explanation of symbols]

[0185] 110 Detector 112 Object 114 Mobile Devices 116 Projector 118 Array 120 Emitter 122 Flood Light Source 124 Control Unit 126 Shutter 128 Sensor Element 130 Camera 132 Optical Sensor 134 Beam Profile 136 Evaluation Equipment 138 Irradiating with at least one illumination pattern and imaging at least one reflected image 140 Illuminate with scene illumination and image at least one scene image. 142 Determining at least one first material 144 Determining at least one second material 146 Determining the material of an object

Claims

1. A detector (110) for material detection of at least one object (112), said detector (110) comprising: at least one projector (116) for illuminating at least one object (112) with at least one illumination pattern, the illumination pattern including a plurality of illumination features, each of the illumination features being at least one at least partially extended feature of the illumination pattern, the illumination features having a first wavelength; at least one flood light source (122) configured for scene illumination, said flood light source (122) being configured to emit scene illumination having a second wavelength different from said first wavelength; at least one sensor element (128) having a matrix of light sensors (132), each having a light-sensitive area, each light sensor (132) designed to generate at least one sensor signal in response to illumination of the respective light-sensitive area by a light beam propagating from said object (112) to said detector (110); The sensor element (128) is configured to image at least one reflected image comprising a plurality of reflected features generated by the object (112) in response to the illumination pattern, the reflected features being features in an image plane generated by the object (112) in response to illumination of at least one illumination feature, each of the reflected features having a beam profile (134); at least one sensor element configured to image at least one scene image of the object illuminated by the scene illumination; at least one evaluation device (136), the evaluation device (136) is configured to determine at least one first material information of the object (112) by evaluating the beam profile (134) of at least one of the reflection features; the evaluation device (136) is configured to determine at least one second material information of the object (112) by evaluating the scene image; at least one evaluation device configured to determine a material of the object using the first material information and the second material information; A detector (110).

2. The detector of claim 1 , wherein the evaluation device is configured to use the second material information as an additional information channel for distinguishing materials.

3. The detector (110) of claim 1 or 2, wherein the detector (110) is configured to distinguish between biological and non-biological materials.

4. The detector (110) of claim 1 or 2, wherein the detector (110) is configured to distinguish between skin and non-skin.

5. 3. The detector (110) of claim 1 or 2, wherein the first wavelength and the second wavelength are different wavelengths in the infrared spectral range.

6. 3. The detector (110) of claim 1 or 2, wherein the first wavelength is 940 nm and the second wavelength is 850 nm.

7. 3. The detector of claim 1, wherein the projector comprises a plurality of emitters, the emitters comprising at least one emitter selected from the group consisting of at least one semiconductor laser, at least one double heterostructure laser, at least one external cavity laser, at least one separate confinement heterostructure laser, at least one quantum cascade laser, at least one distributed Bragg reflector laser, at least one polariton laser, at least one hybrid silicon laser, at least one extended cavity diode laser, at least one quantum dot laser, at least one volume Bragg grating laser, at least one indium arsenide laser, at least one transistor laser, at least one diode pumped laser, at least one distributed feedback laser, at least one quantum well laser, at least one interband cascade laser, at least one gallium arsenide laser, at least one semiconductor ring laser, at least one extended cavity diode laser, at least one vertical cavity surface emitting laser (VCSEL).

8. The detector (110) of claim 1 or 2, wherein the flood light source (122) comprises at least one light emitting diode (LED).

9. 3. The detector (110) of claim 1 or 2, wherein the evaluation device (136) is configured to determine a vertical coordinate of at least one of the reflection features by analysis of a respective beam profile (134), the analysis of the beam profile comprising evaluating a combined signal Q from a sensor signal associated with the reflection feature, and the evaluation device (136) is configured to use at least one predetermined relationship between the combined signal Q and the vertical coordinate to determine the vertical coordinate.

10. 10. The detector of claim 9, wherein the evaluation device is configured to derive the combined signal Q by one or more of: dividing the sensor signal, dividing a multiple of the sensor signal, or dividing a linear combination of the sensor signals.

11. Detector (110) according to claim 1 or 2, wherein the sensor element (128) comprises at least one CCD chip and / or at least one CMOS chip.

12. 3. A method for detecting a material of at least one object (112) using at least one detector (110) according to claim 1 or 2, said method comprising the following steps: - illuminating an object (112) with at least one illumination pattern generated by at least one projector (116), said illumination pattern comprising a plurality of illumination features, each of said illumination features being at least one at least partially extended feature of said illumination pattern, said illumination features having a first wavelength; - illuminating said object (112) with scene illumination produced by at least one flood light source (122), said flood light source (122) being configured to emit scene illumination having a second wavelength different from said first wavelength; - imaging, using a sensor element (128), at least one reflected image comprising a plurality of reflected features produced by said object in response to said illumination pattern, said reflected features being features in an image plane produced by said object in response to illumination of at least one illumination feature, each of said reflected features comprising a beam profile (134); - imaging, using a sensor element (128), at least one scene image of said object (112) illuminated by said scene illumination; - determining at least one first material property of said object (112) by evaluating said beam profile (134) of at least one said reflection feature using an evaluation device (136); - determining at least one second material information of said object (112) by evaluating said scene image using said evaluation device (136); - determining, using an evaluation device (136), a material of said object (112) using said first material information and said second material information; A method comprising:

13. 13. The method of claim 12, further comprising evaluating a sensor signal, thereby determining a combined signal Q, and determining a vertical coordinate of at least one of the reflection features by analysis of a respective beam profile, the analysis of the beam profile comprising evaluating the combined signal Q from the sensor signal associated with the reflection feature, and the evaluation device (136) configured to use at least one predetermined relationship between the combined signal Q and the vertical coordinate to determine the vertical coordinate.

14. Use of the detector (110) according to claim 1 or 2 in relation to a detector, the use being selected from the group consisting of: position measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human machine interface applications; tracking applications; photography applications; imaging or camera applications; mapping applications for generating a map of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotics applications; quality control applications; manufacturing applications; automotive applications.

Citation Information

Patent Citations

  • Biomedical imaging apparatus and related methods

    JP2014525091A

  • Face authentication apparatus, face authentication method, and face authentication program

    JP2019139433A

  • Biometric Imaging Devices and Associated Methods

    US20150356351A1

  • Biometric Methods for Online User Authentication

    US20190207932A1

  • Using an illumination sequence pattern for biometric authentication

    US20200311448A1