Gesture recognition

Abstract

A detector (110) for gesture detection comprises - at least one illumination source (112) configured for projecting at least one illumination pattern comprising a plurality of illumination features onto at least one region (114) comprising at least one object (116), wherein the object (116) comprises at least partially at least one human hand; - at least one optical sensor (118) having at least one light-sensitive area (126), wherein the optical sensor (118) is configured for determining at least one image (128) of the region, wherein the image (128) comprises a plurality of reflection features (130) resulting from the region (114) in response to the illumination by the illumination features; - at least one evaluation device (136), wherein the evaluation device (136) is configured for determining at least one depth map of the region by determining at least one depth information for each reflection feature (130), wherein the evaluation device (136) is configured for finding the object (116) by identifying the reflection features (130) resulting from the illumination of biological tissue, wherein the evaluation device (136) is configured for determining at least one reflection beam profile for each reflection feature (130), wherein the evaluation device (136) is configured for identifying a reflection feature (130) as resulting from the illumination of biological tissue in case the reflection beam profile of the reflection feature (130) fulfills at least one predetermined or predefined criterion, wherein the evaluation device (136) is configured for identifying the reflection feature (130) as background otherwise, wherein the evaluation device (136) is configured for segmenting the image (128) of the region by using at least one segmentation algorithm, wherein the reflection features (130) identified as resulting from the illumination of biological tissue are used as seed points and the reflection features (130) identified as background are used as background seed points for the segmentation algorithm, wherein the evaluation device (136) is configured for determining a position and / or an orientation of the object (116) in space taking into account the segmented image and the depth map.

Description

Technical Field

[0001] This invention relates to a detector for posture detection, a method for posture detection, and various uses of the detector. The apparatus, method, and uses according to the invention can be specifically used, for example, in various fields such as daily life, security technology, gaming, transportation technology, production technology, photography such as digital or video photography for artistic purposes, documentation or technical purposes, security technology, information technology, agriculture, crop protection, maintenance, cosmetics, medical technology, or science. However, other applications are also possible. Background Technology

[0002] The trend toward autonomous vehicles is not only driving autonomous driving but also impacting in-vehicle assistance features. Driver monitoring, in-vehicle monitoring, and posture tracking are fundamental concepts for next-generation cars. Early systems are already available, relying on 2D images, 3D depth maps, or a combination of both.

[0003] From a technical perspective, detecting the posture and pose of people in a car interior can be very challenging. Existing methods based on 2D image processing rely on color features, such as ARVárkonyi-Kóczy, B. Tusor, “Human-Computer Interaction for Smart Environment Applications Using Fuzzy Hand Posture and Gesture Models”, IEEE Trans. Instrumentation and Measurement, vol. 60, no. 5, pp. 1505-1514, 2011, or CCHsieh, DH Liou, “Novel Haar features for real-time hand gesture recognition”, J. Real Time Image Processing, vol. 10, pp. 357-370, 2015. As described in Processing, vol. 10, pp. 357-370, 2015; and shape features, see, for example, E. Ong, R. Bowden, “A boosted classifier tree for hand shape detection,” Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004, Proceedings, Seoul, South Korea, pp. 889-894.However, the robustness of these methods is rather limited due to the wide variation in hand appearance, shadows, and a broad range of lighting conditions. See M. Bergh, L. Gool, “Combining RGB and ToF cameras for real-time 3D hand gesture interaction”, In: Workshop on applications of computer vision (WACV), IEEE, pp. 66–72, 2011.

[0004] For example, motion analysis using optical flow can fill this gap under certain conditions, such as R. Cutler and M. Turk's "View-based Interpretation of Realtime Optical Flow for Gesture Recognition," Proc. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 416–421, 1998, but still lacks static scenes or has insufficient contrast. Depth maps, such as those from time-of-flight cameras or stereo methods, provide supplementary data that can significantly improve detection accuracy. The combination of 2D cameras and 3D sensors provides more reliable detection accuracy, but requires a comprehensive fusion concept, careful calibration, and results in significantly higher costs. Furthermore, separating multiple objects located at similar depths remains challenging.

[0005] In traditional 2D or 3D pose detection setups, hands must be identified from 2D images or 3D point clouds. Hand recognition alone is the challenging part because the contrast with the background in a 2D image can be low, and hand color can vary significantly depending on skin type and lighting conditions. Especially since the hand must be identified as a hand, model data such as shape and color needs to be matched to it. However, shape varies greatly because shape conveys some information in pose. Fingers may even be permanently lost or simply occluded by other objects.

[0006] Furthermore, WO 2018 / 091640 and WO 2019 / 042956 describe the use of a single CMOS sensor combined with a laser grid projector for 2D image and depth measurement to accurately detect distance. All data originates from a single CMOS sensor, allowing for easy and rapid integration. Therefore, system complexity is significantly reduced compared to a combination of separate image and depth sensors.

[0007] US 2016 / 253821 A1 describes an object that is identified or tracked within a volume by projecting a beam of light, encoded with one or more predefined properties, into the volume with a predefined optical structure. A detector captures light reflected from the volume from the predefined optical structure. The object is segmented from the volume by analyzing one or more properties of the light reflected from the object from the predefined optical structure.

[0008] The problem solved by this invention

[0009] Therefore, the object of the present invention is to provide apparatus and methods that address the aforementioned technical challenges of known devices and methods. Specifically, the object of the present invention is to provide apparatus and methods that allow reliable posture detection via a display with low technical effort and low requirements in terms of technical resources and cost. Summary of the Invention

[0010] This problem is solved by the present invention, which features independent patent claims. Advantageous developments of the invention, which can be implemented individually or in combination, are presented in the dependent claims and / or in the following description and detailed embodiments.

[0011] As used below, the terms “have,” “include,” or “contain,” or any grammatical variations thereof, are used in a non-exclusive manner. Thus, these terms can refer to a situation where an entity described in this context has no other features besides those introduced by these terms, or to a situation where one or more other features exist. For example, the statements “A has B,” “A includes B,” and “A contains B” can refer to a situation where A has no other elements besides B (i.e., A consists solely and exclusively of B), or to a situation where entity A has one or more other elements besides B (such as element C, elements C and D, or even other elements).

[0012] Furthermore, it should be noted that the terms "at least one," "one or more," or similar expressions indicating that a feature or element may exist once or more will generally be used only once when the corresponding feature or element is introduced. In the following text, in most cases, when referring to the corresponding feature or element, the expression "at least one" or "one or more" will not be repeated, but the fact that the corresponding feature or element may exist once or more will be acknowledged.

[0013] Furthermore, as used below, the terms “preferredly,” “more preferably,” “particularly,” “more particularly,” “specifically,” “more specifically,” or similar terms may be used in combination with optional features without limiting the possibility of substitution. Therefore, features introduced by these terms are optional features and are not intended to limit the scope of the claims in any way. As those skilled in the art will recognize, the invention can be practiced using alternative features. Similarly, features introduced by phrases such as “in embodiments of the invention” are intended to be optional features, without limiting alternative embodiments of the invention, without limiting the scope of the invention, and without limiting the possibility of combining features introduced in this manner with other optional or non-optional features of the invention.

[0014] In a first aspect of the invention, an apparatus for posture detection is disclosed. As used herein, the term "posture" can refer to at least one state and / or at least one movement of at least a part of the human body. Posture can include body postures, such as body postures of at least a part of the body, particularly gestures. Posture can be static or dynamic. Posture can include movement of at least a part of the hand, such as movement of the fingers, movement of one or both hands, the face, or other parts of the body. As used herein, the term "posture detection" can refer to determining the presence or absence of a posture and / or posture recognition. As used herein, the term "posture recognition" can refer to interpreting a human posture using a mathematical algorithm. As used herein, the term "detector for posture detection" can refer to at least one arbitrary device configured to detect at least one posture.

[0015] Detectors used for pose detection include

[0016] - At least one illumination source configured to project at least one illumination pattern comprising a plurality of illumination features onto at least one area comprising at least one object, wherein the object at least partially comprises at least one human hand;

[0017] - At least one optical sensor having at least one photosensitive area, wherein the optical sensor is configured to determine at least one image of the area, wherein the image includes a plurality of reflection features generated by the area in response to illumination by illumination features;

[0018] - At least one evaluation device, wherein the evaluation device is configured to determine at least one depth map of the area by determining at least one depth information for each reflection feature.

[0019] The evaluation device is configured to locate an object by identifying reflective features produced by irradiating biological tissue. The evaluation device is configured to determine at least one reflective beam profile for each reflective feature. The evaluation device is configured to identify a reflective feature as produced by irradiating biological tissue if the reflective beam profile of the reflective feature meets at least one predetermined or predefined criterion. Otherwise, the evaluation device is configured to identify the reflective feature as background.

[0020] The evaluation device is configured to segment the image of the region using at least one segmentation algorithm, wherein reflectance features identified as being produced by irradiation of biological tissue are used as seed points, and reflectance features identified as background are used as background seed points for the segmentation algorithm.

[0021] The evaluation device is configured to take into account segmented images and depth maps to determine the position and / or orientation of an object in space.

[0022] As used herein, the term "irradiation source" can generally refer to at least one arbitrary device configured to generate at least one irradiation pattern. The irradiation source can be configured to provide an irradiation pattern for irradiating the area. The irradiation source can be adapted to directly or indirectly irradiate the area, wherein the irradiation pattern is reflected or scattered by a surface of the area and thus at least partially directed towards an optical sensor. The irradiation source can be configured to irradiate the area, for example, by directing a light beam toward the area, which then reflects the beam. The irradiation source can be configured to generate an irradiation beam irradiating the area.

[0023] As used herein, the term "area" can refer to at least one arbitrary object or region of space. The area may include at least one object and its surrounding environment. As used herein, the term "object" refers to a point or area that emits at least one beam of light, particularly at least one reflective pattern. For example, the object may be at least one object selected from the group consisting of: a scene, a person such as a human, wood, carpet, foam, an animal such as a cow, a plant, a block of tissue, metal, a toy, a metallic object, a beverage, food such as fruit, meat, or fish, a plate, cosmetics, applied cosmetics, clothing, fur, hair, skincare products, plants, a body, a part of a body, organic materials, inorganic materials, reflective materials, a screen, a display, a wall, a piece of paper, such as a photograph. The object may include at least one surface on which the illumination pattern is projected. The surface may be adapted to at least partially reflect the illumination pattern back to the detector. The object may specifically be a human body or at least a part of a human body, such as at least one arm, at least one hand, at least one finger, or a face. The object at least partially includes at least one human hand. As used herein, the term "at least partially at least one human hand" refers to embodiments in which the object is a human hand and / or embodiments in which the object is at least one finger and / or at least a portion of the palm.

[0024] The illumination source may include at least one light source. The illumination source may include multiple light sources. The illumination source may include artificial illumination sources, particularly at least one laser source and / or at least one incandescent lamp and / or at least one semiconductor light source, such as at least one light-emitting diode, particularly an organic and / or inorganic light-emitting diode. As an example, the light emitted by the illumination source may have a wavelength of 300 to 1100 nm, particularly 500 to 1100 nm. Additionally or alternatively, light in the infrared spectral range may be used, such as in the range of 780 nm to 3.0 μm. Specifically, light in a portion of the near-infrared region may be used, in which silicon photodiodes are specifically suited for the range of 700 nm to 1100 nm. The illumination source may be configured to generate at least one illumination pattern in the infrared region. Using light in the near-infrared region allows the light to be detected by the human eye only weakly and not at all, but still detectable by a silicon sensor, particularly a standard silicon sensor.

[0025] As used herein, the term "ray" generally refers to a line perpendicular to the wavefront of light and pointing in the direction of energy flow. As used herein, the term "bundle" generally refers to a collection of rays. Hereinafter, the terms "ray" and "bundle" will be used as synonyms. As further used herein, the term "beam" generally refers to the amount of light, specifically the amount of light traveling substantially in the same direction, including the possibility of beams having a diffusion angle or widening angle. A beam may have spatial extension. Specifically, a beam may have a non-Gaussian beam profile. The beam profile may be selected from the group consisting of: trapezoidal beam profile; triangular beam profile; conical beam profile. A trapezoidal beam profile may have a plateau region and at least one edge region. A beam may specifically be a Gaussian beam or a linear combination of Gaussian beams, as will be outlined in more detail below. However, other embodiments are possible.

[0026] The illumination source can be configured to emit light of a single wavelength. Specifically, this wavelength can be in the near-infrared region. In other embodiments, the illumination source can be configured to emit light with multiple wavelengths, thereby allowing for additional measurements in other wavelength channels.

[0027] The illumination source may be or may include at least one multi-beam light source. For example, the illumination source may include at least one laser source and one or more diffractive optical elements (DOEs). Specifically, the illumination source may include at least one laser and / or laser source. Various types of lasers can be used, such as semiconductor lasers, dual heterostructure lasers, external cavity lasers, split-confined heterostructure lasers, quantum cascade lasers, distributed Bragg reflector lasers, polaron lasers, hybrid silicon lasers, extended cavity diode lasers, quantum dot lasers, volume Bragg grating lasers, indium arsenide lasers, transistor lasers, diode-pumped lasers, distributed feedback lasers, quantum well lasers, interband cascade lasers, gallium arsenide lasers, semiconductor ring lasers, extended cavity diode lasers, or vertical cavity surface-emitting lasers. Additionally or alternatively, non-laser light sources, such as LEDs and / or bulbs, may be used. The illumination source may include one or more diffractive optical elements (DOEs) suitable for generating an illumination pattern. For example, the illumination source can be adapted to generate and / or project a point cloud. For instance, the illumination source may include one or more of the following: at least one digital light processing projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one light-emitting diode array; at least one laser source array. Using at least one laser source as the illumination source is particularly preferred due to its typically defined beam profile and other characteristics of operability. The illumination source can be integrated into the detector housing.

[0028] Furthermore, the illumination source can be configured to emit modulated or unmodulated light. When using multiple illumination sources, the different sources can have different modulation frequencies, which can then be used to distinguish the light beams.

[0029] One or more light beams generated by an illumination source can typically propagate parallel to or tilted relative to the optical axis, for example, by an angle relative to the optical axis. A detector can be configured such that one or more light beams propagate from the detector toward the region along the optical axis of the detector. For this purpose, the detector may include at least one reflective element, preferably at least one prism, for deflecting the illumination beam onto the optical axis. As an example, one or more light beams, such as a laser beam, and the optical axis may include an angle of less than 10°, preferably less than 5° or even less than 2°. However, other embodiments are also possible. Moreover, one or more light beams can be on or off the optical axis. As an example, one or more light beams can be parallel to the optical axis, having a distance of less than 10 mm from the optical axis, preferably less than 5 mm or even less than 1 mm from the optical axis, or even coincident with the optical axis.

[0030] As used herein, the term "at least one illumination pattern" means at least one arbitrary pattern that includes at least one illumination feature suitable for illuminating at least a portion of the area. As used herein, the term "illumination feature" means at least one at least partially extended feature of the pattern. An illumination pattern may include a single illumination feature. An illumination pattern may include multiple illumination features. An illumination pattern may be selected from the group consisting of: at least one dot pattern; at least one line pattern; at least one stripe pattern; at least one checkerboard pattern; at least one pattern including an arrangement of periodic or aperiodic features. An illumination pattern may include regular and / or constant and / or periodic patterns, such as triangular patterns, rectangular patterns, hexagonal patterns, or patterns including further convex tiles. An illumination pattern may exhibit at least one illumination feature selected from the group consisting of: at least one dot; at least one line; at least two lines, such as parallel lines or intersecting lines; at least one dot and one line; at least one arrangement of periodic or aperiodic features; at least one arbitrarily shaped feature. The illumination pattern may include at least one pattern selected from the group consisting of: at least one point pattern, particularly a pseudo-random point pattern; a random point pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one predictable feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including convex uniform tiles; at least one line pattern including at least one line; at least one line pattern including at least two lines, such as parallel or intersecting lines. For example, the illumination source may be adapted to generate and / or project a point cloud. The illumination source may include at least one light projector adapted to generate a point cloud such that the illumination pattern may include multiple point patterns. The illumination source may include at least one mask adapted to generate the illumination pattern from at least one light beam generated by the illumination source.

[0031] The distance between two features of the illumination pattern and / or the area of ​​at least one illumination feature may depend on the blurred circle in the image. As outlined above, the illumination source may include at least one light source configured to generate the at least one illumination pattern. Specifically, the illumination source includes at least one light source and / or at least one laser diode designated for generating laser radiation. The illumination source may include at least one diffractive optical element (DOE). The detector may include at least one light projector, such as at least one laser source and DOE, adapted to project at least one periodic dot pattern.

[0032] As used further herein, the term “projecting at least one illumination pattern” means providing at least one illumination pattern for illuminating at least one area.

[0033] As used herein, "optical sensor" generally refers to a photosensitive device for detecting a light beam, such as for detecting illumination and / or a light spot produced by at least one light beam. As further used herein, "photosensitive area" generally refers to a region of an optical sensor that can be illuminated from the outside by at least one light beam, generating at least one sensor signal in response to such illumination. The photosensitive area may specifically be located on the surface of the respective optical sensor. However, other embodiments are also possible. The detector may include multiple optical sensors, each having a photosensitive area. As used herein, the term "each optical sensor having at least one photosensitive area" refers to a configuration where each of a plurality of individual optical sensors has one photosensitive area, and a configuration where a combined optical sensor has multiple photosensitive areas. Furthermore, the term "optical sensor" refers to a photosensitive device configured to generate an output signal. In the case where the detector includes multiple optical sensors, each optical sensor can be implemented such that a precise photosensitive area exists within the respective optical sensor, such as by precisely providing a photosensitive area that can be illuminated, generating a precise, uniform sensor signal for the entire optical sensor in response to such illumination. Thus, each optical sensor may be a single-area optical sensor. However, the use of a single-area optical sensor makes detector setup particularly simple and efficient. Therefore, as an example, commercially available photoelectric sensors, such as commercially available silicon photodiodes, each having exactly one photosensitive region, can be used in the setup. However, other embodiments are also feasible.

[0034] Preferably, the photosensitive region can be oriented substantially perpendicular to the optical axis of the detector. The optical axis can be a straight optical axis, or it can be curved, or even split, for example by using one or more deflecting elements and / or by using one or more beam splitters, wherein in the latter case, the substantially perpendicular orientation can indicate the local optical axis in the corresponding branch or beam path of the optical setup.

[0035] The optical sensor may specifically be or may include at least one photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, and most preferably a silicon photodetector. Specifically, the optical sensor may be sensitive in the infrared spectral range. The optical sensor may include at least one sensor element comprising a pixel matrix. All pixels in the matrix or at least one group of optical sensors in the matrix may specifically be identical. The same group of pixels in the matrix may be specifically provided for different spectral ranges, or all pixels may have the same spectral sensitivity. Furthermore, the pixels may have the same size and / or electronic or optoelectronic properties. Specifically, the optical sensor may be or may include at least one inorganic photodiode sensitive in the infrared spectral range, preferably in the range of 700 nm to 3.0 micrometers. Specifically, the optical sensor may be sensitive in a portion of the near-infrared region, in which the silicon photodiode is specifically suited for the range of 700 nm to 1100 nm. Infrared optical sensors that can be used in the optical sensor are commercially available infrared optical sensors, such as the TrinamiX from Ludwigshafen am Rhein (Germany) D-67056. TM The trademark name launched by GmbH is Hertzstueck TM Commercially available infrared optical sensors are available. Therefore, as an example, the optical sensor may include at least one optical sensor of an inherent photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, and HgCdTe photodiodes. Additionally or alternatively, the optical sensor may include at least one optical sensor of an inherent photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: Ge:Au photodiodes, Ge:Hg photodiodes, Ge:Cu photodiodes, Ge:Zn photodiodes, Si:Ga photodiodes, and Si:As photodiodes. Additionally or alternatively, the optical sensor may include at least one photoconductivity sensor, such as a PbS or PbSe sensor, or a radiative thermal meter, preferably selected from V0 radiative thermal meters and amorphous Si radiative thermal meters.

[0036] Optical sensors can be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. Specifically, optical sensors can be sensitive in the visible spectral range from 500 nm to 780 nm, most preferably in the range of 650 nm to 750 nm or 690 nm to 700 nm. Specifically, optical sensors can be sensitive in the near-infrared region. Specifically, optical sensors can be sensitive in a portion of the near-infrared region, in which the silicon photodiode is specifically suited for the range of 700 nm to 1000 nm. Optical sensors can be sensitive in the infrared spectral range, specifically in the range of 780 nm to 3.0 micrometers. For example, each optical sensor individually can be or can include at least one element selected from the group consisting of: photodiodes, photovoltaic cells, photoconductors, phototransistors, or any combination thereof. For example, optical sensors can be or can include at least one element selected from the group consisting of: CCD sensor elements, CMOS sensor elements, photodiodes, photovoltaic cells, photoconductors, phototransistors, or any combination thereof. Any other type of photosensitive element can be used. Photosensitive elements can typically be made entirely or partially of inorganic materials and / or entirely or partially of organic materials. Most commonly, one or more photodiodes, such as commercially available photodiodes, for example, inorganic semiconductor photodiodes, can be used.

[0037] An optical sensor may include at least one sensor element comprising a pixel matrix. Therefore, as an example, an optical sensor may be part of or constitute a pixelated optical device. For example, an optical sensor may be and / or may include at least one CCD and / or CMOS device. As an example, an optical sensor may be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, with each pixel forming a photosensitive area.

[0038] As used herein, the term "sensor element" generally refers to a device or combination of devices configured to sense at least one parameter. In this case, the parameter may specifically be an optical parameter, and the sensor element may specifically be an optical sensor element. Sensor elements may be formed as a single device or a combination of several devices. Sensor elements include matrices of optical sensors. Sensor elements may include at least one CMOS sensor. Matrix may include individual pixels, such as independent optical sensors. Thus, a matrix of inorganic photodiodes may be formed. However, alternatively, one or more of commercially available matrices, such as CCD detectors (e.g., CCD detector chips) and / or CMOS detectors (e.g., CMOS detector chips), may be used. Thus, generally, sensor elements may be and / or may include at least one CCD and / or CMOS device, and / or optical sensors may form a sensor array or may be part of a sensor array, such as the matrix mentioned above. Thus, as an example, sensor elements may include pixel arrays, such as a rectangular array with m rows and n columns, where m and n are independently positive integers. Preferably, more than one column and more than one row are given, i.e., n>1, m>1. Therefore, as an example, n can be 2 to 16 or higher, and m can be 2 to 16 or higher. Preferably, the ratio of the number of rows to the number of columns is close to 1. As an example, n and m can be chosen such that 0.3 ≤ m / n ≤ 3, such as by choosing m / n = 1:1, 4:3, 16:9, or similar values. As an example, the array can be a square array with equal numbers of rows and columns, such as by choosing m = 2, n = 2 or m = 3, n = 3, etc.

[0039] The matrix may include individual pixels, such as independent optical sensors. Therefore, it may include a matrix of inorganic photodiodes. However, alternatively, commercially available matrices may be used, such as one or more of CCD detectors (such as CCD detector chips) and / or CMOS detectors (such as CMOS detector chips). Thus, typically, the optical sensor may be and / or may include at least one CCD and / or CMOS device, and / or the optical sensor of the detector may form a sensor array or may be part of a sensor array, such as the matrix mentioned above.

[0040] Specifically, the matrix can be a rectangular matrix having at least one row, preferably multiple rows, and multiple columns. As an example, the rows and columns can be substantially vertically oriented. As used herein, the term "substantially vertical" refers to a vertical orientation condition with a tolerance of, for example, ±20° or less, preferably ±10° or less, more preferably ±5° or less. Similarly, the term "substantially parallel" refers to a parallel orientation condition with a tolerance of, for example, ±20° or less, preferably ±10° or less, more preferably ±5° or less. Thus, as an example, tolerances less than 20°, specifically less than 10°, or even less than 5° are acceptable. To provide a wide field of view, the matrix can specifically have at least 10 rows, preferably at least 500 rows, more preferably at least 1000 rows. Similarly, the matrix can have at least 10 columns, preferably at least 500 columns, more preferably at least 1000 columns. The matrix can include at least 50 optical sensors, preferably at least 100,000 optical sensors, more preferably at least 5,000,000 optical sensors. The matrix can include multiple pixels spanning millions of pixels. However, other embodiments are also feasible. Therefore, in a configuration where axial rotational symmetry is desired, a circular or concentric arrangement of the optical sensors (also referred to as pixels) within the matrix is ​​preferred.

[0041] Therefore, as an example, the sensor element may be part of or constitute a pixelated optics. For example, the sensor element may be and / or may include at least one CCD and / or CMOS device. As an example, the sensor element may be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, with each pixel forming a photosensitive area. The sensor element may employ a rolling shutter or global shutter method to read out the matrix of the optical sensor.

[0042] The detector may also include at least one delivery device. The detector may also include one or more additional elements, such as one or more additional optical elements. The detector may include at least one optical element selected from the group consisting of: delivery devices, such as at least one lens and / or at least one lens system, at least one diffractive optical element. The term "delivery device" (also referred to as "delivery system") may generally refer to one or more optical elements adapted to modify a beam, such as by modifying one or more of the beam parameters, beam width, or beam direction. The delivery device may be adapted to guide the beam onto an optical sensor. Specifically, the delivery device may include one or more of the following: at least one lens, for example, at least one lens selected from the group consisting of: at least one focusable lens, at least one aspherical lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one reflector; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting reflector; at least one multi-lens system. As used herein, the term "focal length" of a transmission device refers to the distance at which collimated incident light rays incident on the transmission device are "focused" (which can also be represented as "focal point"). Therefore, the focal length constitutes a measure of the transmission device's ability to converge an incident beam. Thus, a transmission device may include one or more imaging elements that can have the effect of a converging lens. For example, a transmission device may have one or more lenses, particularly one or more refracting lenses, and / or one or more convex lenses. In this example, the focal length may be defined as the distance from the center of the thin refracting lens to the principal focal point of the thin lens. For converging thin refracting lenses, such as convex or biconvex thin lenses, the focal length can be considered positive and can provide a distance at which a collimated beam incident on the thin lens acting as a transmission device can be focused into a single spot. Furthermore, the transmission device may include at least one wavelength selection element, such as at least one optical filter. Additionally, the transmission device may be designed to imprint a predefined beam profile onto electromagnetic radiation, for example, at a location in the sensor region and, particularly, the sensor area. In principle, the above-described alternative embodiments of the transmission device can be implemented individually or in any desired combination.

[0043] The transmission device may have an optical axis. Specifically, the detector and the transmission device share a common optical axis. As used herein, the term "optical axis of the transmission device" generally refers to the mirror symmetry or rotational symmetry axis of a lens or lens system. The optical axis of the detector may be the line of symmetry of the detector's optical arrangement. The detector may include at least one transmission device, preferably at least one transmission system having at least one lens. As an example, the transmission system may include at least one beam path, wherein the elements of the transmission system in the beam path are positioned in a rotationally symmetrical manner about the optical axis. However, as will be outlined in more detail below, one or more optical elements located within the beam path may also be off-center or tilted about the optical axis. However, in this case, the optical axis may be defined sequentially, such as by connecting the centers of the optical elements in the beam path to each other, for example by connecting the centers of the lenses, wherein, in this context, the optical sensor is not considered an optical element. The optical axis may generally represent the beam path. The detector may have a single beam path along which the beam travels from the object to the optical sensor, or it may have multiple beam paths. As an example, a single beam path may be given, or the beam path may be divided into two or more partial beam paths. In the latter case, each partial beam path may have its own optical axis. In the case of multiple optical sensors, the optical sensors can be located in the same beam path or part of the beam path. Alternatively, however, the optical sensors can also be located in different part of the beam path.

[0044] The transmission device can form a coordinate system, where the longitudinal coordinate is the coordinate along the optical axis, and where d is the spatial offset from the optical axis. The coordinate system can also be a polar coordinate system, where the optical axis of the transmission device forms the z-axis, and where the distance from the z-axis and the polar angle can be used as additional coordinates. Directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered longitudinal coordinates. Any direction perpendicular to the z-axis can be considered a transverse direction, and polar coordinates and / or polar angles can be considered transverse coordinates.

[0045] The detectors can form a coordinate system, where the optical axes of the detectors form the z-axis, and furthermore, x-axis and y-axis can be provided that are perpendicular to the z-axis and perpendicular to each other. As an example, the detectors and / or a portion of the detectors can be positioned at a specific point in this coordinate system, such as the origin. In this coordinate system, directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered longitudinal coordinates. Any direction perpendicular to the longitudinal direction can be considered transverse directions, and x and / or y coordinates can be considered transverse coordinates.

[0046] Alternatively, other types of coordinate systems can be used. Thus, as an example, a polar coordinate system can be used, where the optical axis forms the z-axis, and the distance from the z-axis and the polar angle can be used as additional coordinates. Furthermore, directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered longitudinal coordinates. Any direction perpendicular to the z-axis can be considered transverse directions, and polar coordinates and / or polar angles can be considered transverse coordinates.

[0047] An optical sensor is configured to determine at least one image of the region. When the region is illuminated by an illumination feature, the region, i.e., the portion of the region illuminated by the illumination feature, reflects and / or re-emits the incident beam of the illumination feature toward a detector, which can thus image these reflected and / or re-emitted illumination features. Therefore, the region can produce reflective features. The image includes multiple reflective features produced by the region in response to illumination by the illumination feature. As used herein, but not limited to, the term "image" can specifically refer to data recorded using an optical sensor, such as multiple electronic readings from an imaging device (such as pixels of a sensor element). Thus, the image itself can include pixels, the pixels of the image being associated with pixels of a matrix of sensor elements. Therefore, when referring to "pixel," either reference is made to a unit of image information generated by a single pixel of a sensor element, or directly to a single pixel of a sensor element. The image can be at least one two-dimensional image. As used herein, the term "two-dimensional image" can generally refer to an image having information about lateral coordinates (such as dimensions of height and width). The image can be an RGB (red, green, blue) image. As used herein, the term "reflection feature" can refer to a feature in the image plane produced by the region in response to illumination specifically using at least one illumination feature.

[0048] The detector includes at least one evaluation device. The evaluation device is configured to evaluate an image. As used further herein, the term "evaluation device" generally refers to any means suitable for performing a specified operation, preferably performed by using at least one data processing means, more preferably by using at least one processor and / or at least one application-specific integrated circuit (ASIC). Thus, as an example, at least one evaluation device may include at least one data processing means on which software code containing a large number of computer commands is stored. The evaluation device may provide one or more hardware elements for performing one or more of the specified operations, and / or may provide software to one or more processors to run thereon to perform one or more of the specified operations, including evaluating an image. Specifically, instructions for determining the beam profile and surface may be performed by at least one evaluation device. Thus, as an example, one or more instructions may be implemented in software and / or hardware. Thus, as an example, the evaluation device may include one or more programmable means configured to perform the above evaluation, such as one or more computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs). However, additionally or alternatively, the evaluation device may also be embodied entirely or partially in hardware.

[0049] The evaluation device and detector can be fully or partially integrated into a single device. Therefore, the evaluation device can often be part of the detector. Alternatively, the evaluation device and detector can be implemented as separate devices, either fully or partially. The detector may include further components.

[0050] The evaluation apparatus may be or may include one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs), and / or one or more data processing devices, such as one or more computers, preferably one or more microcomputers and / or microcontrollers, field-programmable arrays, or digital signal processors. Additional components may be included, such as one or more preprocessing devices and / or data acquisition devices, such as one or more devices for receiving and / or preprocessing sensor signals, such as one or more analog-to-digital converters and / or one or more filters. Further, the evaluation apparatus may include one or more measuring devices, such as one or more measuring devices for measuring current and / or voltage. Further, the evaluation apparatus may include one or more data storage devices. Further, the evaluation apparatus may include one or more interfaces, such as one or more wireless interfaces and / or one or more wired interfaces.

[0051] The evaluation device may be connected to or may include at least one further data processing device, which may be used for one or more of the following: display, visualization, analysis, distribution, communication, or further processing of information (such as information obtained by optical sensors and / or by the evaluation device). As an example, the data processing device may be connected to or include at least one of the following: a display, projector, monitor, LCD, TFT, speaker, multi-channel sound system, LED pattern, or further visualization device. The data processing device may also be connected to or include at least one of the following: a communication device or communication interface, connector, or port, capable of sending encrypted or unencrypted information using one or more of the following: email, text message, telephone, Bluetooth, Wi-Fi, infrared, or Internet interface, port, or connection. The data processing device may also be connected to or include at least one of the following: a processor; a graphics processor; a CPU; an Open Multimedia Application Platform (OMAP). TM Integrated circuits; systems-on-a-chip such as those from Apple's A-series or Samsung's S3C2 series; microcontrollers or microprocessors; one or more blocks of memory such as ROM, RAM, EEPROM, or flash memory; timing sources such as oscillators or phase-locked loops, counter timers, real-time timers, or power-on reset generators; voltage regulators; power management circuits; or DMA controllers. Individual units can also be connected by a bus (such as an AMBA bus) or integrated into IoT or Industry 4.0 type networks.

[0052] The evaluation device and / or data processing device may be connected to or have further external interfaces or ports, such as serial or parallel interfaces or ports, USB, parallel ports, FireWire, HDMI, Ethernet, Bluetooth, RFID, Wi-Fi, USART, or SPI, or analog interfaces or ports, such as ADCs, DACs, or standardized interfaces or ports to further devices (such as 2D camera devices using RGB interfaces, such as CameraLink). The evaluation device and / or data processing device may also be connected by one or more of the following: inter-processor interfaces or ports, FPGA-FPGA interfaces, or serial or parallel interface ports. The evaluation device and data processing device may also be connected to one or more of the following: optical disc drives, CD-RW drives, DVD+RW drives, flash drives, memory cards, disk drives, hard disk drives, solid-state drives, or solid-state drives.

[0053] The evaluation device and / or data processing device may be connected by one or more further external connectors or have one or more further external connectors, such as one or more of the following: telephone connector, RCA connector, VGA connector, male and female connector, USB connector, HDMI connector, 8P8C connector, BCN connector, IEC 60320C14 connector, fiber optic connector, D miniature connector, RF connector, coaxial connector, SCART connector, XLR connector, and / or may include at least one suitable receptacle for one or more of these connectors.

[0054] The evaluation apparatus is configured to determine at least one depth map of the region by determining at least one depth information for each reflection feature. The term "each reflection feature" may refer to each reflection feature of an image or a region of interest within an image. As used herein, the term "depth" may refer to the distance between the object and the optical sensor and may be given by a longitudinal coordinate. As used herein, the term "depth map" may refer to the spatial distribution of depth. The evaluation apparatus may be configured to determine the depth information for each reflection feature using one or more of the following techniques: depth-from-photon-ratio, structured light, beam profilometry, time-of-flight, shape-from-motion, depth-from-focus, triangulation, defocusing, and stereo sensors. The evaluation apparatus may be configured to take depth information into account to segment objects from the background based on the depth map, particularly hand regions in an image.

[0055] The following describes beam profilometry, particularly the so-called photon ratio depth sounding (DPR) technique. For further details, see WO 2018 / 091649 A1, WO 2018 / 091638 A1 and WO 2018 / 091640 A, the entire contents of which are included by reference, which describe beam profilometry in more detail, particularly the determination of combined signals and the determination of depth from them.

[0056] For example, the evaluation apparatus can be configured to determine depth information for each reflection feature using photon ratio depth sounding. Each reflection feature may include at least one beam profile, also referred to as a reflection beam profile. As used herein, the term "beam profile" for a reflection feature can generally refer to at least one intensity distribution of the reflection feature as a function of pixels, such as the intensity distribution of a light spot on an optical sensor. The beam profile can be selected from linear combinations of trapezoidal beam profiles, triangular beam profiles, conical beam profiles, and Gaussian beam profiles. The evaluation apparatus can be configured to determine depth information for each reflection feature by analyzing their beam profiles.

[0057] The evaluation apparatus can be configured to determine at least one longitudinal coordinate z for each reflection feature by analyzing their beam profiles. DPR As used herein, the term "beam profile analysis" can generally refer to the evaluation of a beam profile and may include at least one mathematical operation and / or at least one comparison and / or at least one symmetry and / or at least one filtering and / or at least one normalization. For example, beam profile analysis may include at least one of histogram analysis steps, calculation of a difference metric, application of a neural network, or application of a machine learning algorithm. The evaluation apparatus can be configured to symmetrize and / or normalize and / or filter the beam profile, particularly to remove noise or asymmetry from recordings at large angles, recording edges, etc. The evaluation apparatus can filter the beam profile by removing high spatial frequencies, such as through spatial frequency analysis and / or median filtering. A summary can be performed by averaging the intensity center of the beam and all intensities at the same distance from the center. The evaluation apparatus can be configured to normalize the beam profile to maximum intensity, particularly taking into account intensity differences attributable to distances from the recording. The evaluation apparatus can be configured to remove the effects of background light from the beam profile, for example, through imaging without illumination.

[0058] A reflection feature may cover at least one pixel of an image or may extend over at least one pixel of an image. For example, a reflection feature may cover multiple pixels or may extend over multiple pixels. An evaluation device may be configured to determine and / or select all pixels connected to and / or belonging to a reflection feature (e.g., a light spot). The evaluation device may be configured to determine the intensity center using the following formula:

[0059]

[0060] Where R coi It is the location of the intensity center, r pixel It is the pixel position, and i = ∑ j I total , where j is the number of pixels connected to and / or belonging to the reflection feature, and I total It is the total intensity.

[0061] The evaluation device can be configured to determine the longitudinal coordinate z for each reflection feature using photon ratio depth sounding. DPR Regarding photon ratio depth sounding (DPR) technology, see WO 2018 / 091649 A1, WO 2018 / 091638 A1 and WO 2018 / 091640 A1, the entire contents of which are incorporated herein by reference.

[0062] The evaluation apparatus can be configured to determine the beam profile for each reflection feature. As used herein, the term "determine beam profile" refers to identifying at least one reflection feature provided by an optical sensor and / or selecting at least one reflection feature provided by an optical sensor and evaluating at least one intensity distribution of the reflection feature. As an example, a region of a matrix can be used and evaluated to determine the intensity distribution, such as a three-dimensional intensity distribution or a two-dimensional intensity distribution, such as along an axis or line through the matrix. As an example, the illumination center of the beam can be determined, such as by determining at least one pixel with the highest illumination, and a cross-sectional axis through the illumination center can be selected. The intensity distribution can be an intensity distribution as a function of coordinates along this cross-sectional axis through the illumination center. Other evaluation algorithms are feasible.

[0063] Photon ratio sounding can include at least one first region and at least one second region for determining the reflection beam profile of each reflection feature and / or multiple reflection features in at least one region of interest. Photon ratio sounding can also include integrating the first and second regions. Furthermore, photon ratio sounding can include deriving the quotient Q by one or more of the following: dividing the integrated first and second regions, dividing multiples of the integrated first and second regions, or dividing linear combinations of the integrated first and second regions.

[0064] The evaluation apparatus can be configured to determine at least one first region and at least one second region of the reflection beam profile of each reflection feature and / or multiple reflection features in at least one region of interest. The evaluation apparatus is configured to integrate the first and second regions.

[0065] Analysis of the beam profile, one of the reflection characteristics, may include determining at least one first region and at least one second region of the beam profile. The first region of the beam profile may be region A1, and the second region of the beam profile may be region A2. An evaluation device may be configured to integrate the first and second regions. The evaluation device may be configured to derive a combined signal, particularly a quotient Q, by one or more of the following: division of the integrated first and second regions, division of multiples of the integrated first and second regions, and division of a linear combination of the integrated first and second regions. The evaluation device may be configured to determine at least two regions of the beam profile and / or segment the beam profile into at least two segments comprising different regions of the beam profile, wherein overlap of regions may be possible, provided the regions are not congruent. For example, the evaluation device may be configured to determine multiple regions, such as two, three, four, five, or up to ten regions. The evaluation device may be configured to segment the beam spot into at least two regions of the beam profile and / or segment the beam profile into at least two segments comprising different regions of the beam profile. The evaluation device may be configured to determine the integration of the beam profile over the respective regions for at least two of the regions. The evaluation apparatus can be configured to compare at least two of the determined integrals. Specifically, the evaluation apparatus can be configured to determine at least one first region and at least one second region of the beam profile. As used herein, the term "region of beam profile" generally refers to any region of the beam profile at the location of the optical sensor used to determine the quotient Q. The first region and the second region of the beam profile can be one or both of adjacent or overlapping regions. The first region and the second region of the beam profile may not be congruent in area. For example, the evaluation apparatus can be configured to divide the sensor region of a CMOS sensor into at least two sub-regions, wherein the evaluation apparatus can be configured to divide the sensor region of the CMOS sensor into at least one left portion and / or at least one right portion and / or at least one upper portion and at least one lower portion and / or at least one inner portion and at least one outer portion. Additionally or alternatively, the detector may include at least two optical sensors, wherein the photosensitive regions of the first and second optical sensors can be arranged such that the first optical sensor is adapted to determine the first region of the beam profile of the reflection characteristics, and the second optical sensor is adapted to determine the second region of the beam profile of the reflection characteristics. The evaluation apparatus can be adapted to integrate the first and second regions. The evaluation apparatus can be configured to determine the vertical coordinate using at least one predetermined relationship between the quotient Q and the vertical coordinate. This predetermined relationship can be one or more of empirical, semi-empirical, and analytically derived relationships. The evaluation apparatus may include at least one data storage device for storing the predetermined relationship, such as a lookup list or lookup table.

[0066] The first region of the beam profile may include substantially edge information of the beam profile, and the second region of the beam profile includes substantially center information of the beam profile, and / or the first region of the beam profile may include information substantially about the left side of the beam profile, and the second region of the beam profile includes information substantially about the right side of the beam profile. The beam profile may have a center, i.e., the maximum value of the beam profile and / or the center point of the beam profile's flat top and / or the geometric center of the spot, and a descending edge extending from the center. The second region may include an inner region of the cross-section, and the first region may include an outer region of the cross-section. As used herein, the term "substantially center information" generally refers to a lower proportion of edge information (i.e., a lower proportion of the intensity distribution corresponding to the center) compared to the proportion of center information (i.e., the proportion of the intensity distribution corresponding to the center). Preferably, the center information has a proportion of less than 10%, more preferably less than 5% of the edge information, and most preferably, the center information does not include edge content. As used herein, the term "substantially edge information" generally refers to a lower proportion of center information compared to the proportion of edge information. Edge information may include information about the entire beam profile, particularly information from the center region and the edge regions. The proportion of center information in the edge information is less than 10%, preferably less than 5%, and more preferably, the edge information does not include any center content. If at least one region of the bundle profile is close to or surrounds the center and includes substantially center information, that region can be identified and / or selected as a second region of the bundle profile. If at least one region of the bundle profile includes at least some portions of the descending edge of the cross section, that region can be identified and / or selected as a first region of the bundle profile. For example, the entire region of the cross section can be identified as the first region.

[0067] Other options for the first region A1 and the second region A2 are also possible. For example, the first region may include a substantially outer region of the bundle profile, and the second region may include a substantially inner region of the bundle profile. For example, in the case of a two-dimensional bundle profile, the bundle profile may be divided into a left portion and a right portion, wherein the first region may include a region of the substantially left portion of the bundle profile, and the second region may include a region of the substantially right portion of the bundle profile.

[0068] Edge information may include information relating to the number of photons in a first region of the beam profile, while center information may include information relating to the number of photons in a second region of the beam profile. An evaluation device may be configured to determine the area integral of the beam profile. The evaluation device may be configured to determine the edge information by integrating and / or summing the first region. The evaluation device may be configured to determine the center information by integrating and / or summing the second region. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the integral of the trapezoid. Furthermore, when assuming a trapezoidal beam profile, the determination of the edge and center signals can be replaced by deriving the edge and center signals through geometric considerations using an equivalent evaluation of the characteristics of the trapezoidal beam profile (e.g., determining the slope and position of the edges and the height of the central flat top).

[0069] In one embodiment, A1 may correspond to the entire or complete area of ​​the feature point on the optical sensor. A2 may be the central area of ​​the feature point on the optical sensor. The central area may be a constant value. Compared to the entire area of ​​the feature point, the central area may be smaller. For example, in the case of a circular feature point, the central area may have a radius from 0.1 to 0.9 of the full radius of the feature point, preferably from 0.4 to 0.6 of the full radius.

[0070] In one embodiment, the illumination pattern may include at least one line pattern. A1 may correspond to a region having the full line width of the line pattern on the optical sensor (particularly the photosensitive area of ​​the optical sensor). Compared to the line pattern of the illumination pattern, the line pattern on the optical sensor may be widened and / or shifted to increase the line width on the optical sensor. In particular, in the case of a matrix of optical sensors, the line width of the line pattern on the optical sensor may vary column by column. A2 may be the central region of the line pattern on the optical sensor. The line width of the central region may be a constant value and may specifically correspond to the line width in the illumination pattern. The central region may have a line width smaller than the full line width. For example, the central region may have a line width from 0.1 to 0.9 of the full line width, preferably from 0.4 to 0.6 of the full line width. The line pattern may be segmented on the optical sensor. Each column of the matrix of optical sensors may include center information of the intensity in the central region of the line pattern and edge information of the intensity from the region extending further outward from the central region of the line pattern to the edge region.

[0071] In one embodiment, the illumination pattern may include at least a dot pattern. A1 may correspond to the region of the full radius of the dots in the dot pattern on the optical sensor. A2 may be the central region of the dots in the dot pattern on the optical sensor. The central region may be a constant value. The central region may have a radius compared to the full radius. For example, the central region may have a radius from 0.1 to 0.9 of the full radius, preferably from 0.4 to 0.6 of the full radius.

[0072] The illumination pattern may include at least one dot pattern and at least one line pattern. Other embodiments besides or alternatives to line and dot patterns are possible.

[0073] The evaluation device can be configured to derive the quotient Q by one or more of the following: performing division on the first region and the second region of the integral, performing division on multiples of the first region and the second region of the integral, and performing division on linear combinations of the first region and the second region of the integral.

[0074] The evaluation device can be configured to derive the quotient Q by one or more of the following: division of the first and second regions, division of multiples of the first and second regions, and division of linear combinations of the first and second regions. The evaluation device can also be configured to derive the quotient Q using the following formula:

[0075]

[0076] Where x and y are the lateral coordinates, A1 and A2 are the first and second regions of the bundle profile, respectively, and E(x,y) represents the bundle profile.

[0077] Additionally or alternatively, the evaluation device may be adapted to determine one or both of center information and edge information from at least one slice or cut of the light spot. This can be achieved, for example, by replacing the area integral in the quotient Q with a line integral along the slice or cut. To improve accuracy, several slices or cuts of the light spot can be used and averaged. In the case of an elliptical light spot profile, averaging over several slices or cuts improves the distance information.

[0078] For example, in the case of an optical sensor having a pixel matrix, the evaluation device can be configured to evaluate the beam profile through the following steps:

[0079] - Identify the pixel with the highest sensor signal and form at least one center signal;

[0080] - Evaluate the sensor signals of the matrix and form at least one sum signal;

[0081] - The quotient Q is determined by combining the center signal and the signal; and

[0082] - Determine at least one longitudinal coordinate z of the object by evaluating the quotient Q.

[0083] As used herein, "sensor signal" generally refers to a signal generated by an optical sensor and / or at least one pixel of an optical sensor in response to illumination. Specifically, a sensor signal can be or may include at least one electrical signal, such as at least one analog electrical signal and / or at least one digital electrical signal. More specifically, a sensor signal can be or may include at least one voltage signal and / or at least one current signal. More specifically, a sensor signal may include at least one photocurrent. Further, the original sensor signal can be used, or the display device, optical sensor, or any other element can be adapted to process or preprocess the sensor signal to generate a secondary sensor signal that can also be used as the sensor signal, such as through preprocessing like filtering. The term "center signal" generally refers to at least one sensor signal that includes substantially center information of the bundle profile. As used herein, the term "maximum sensor signal" refers to one or both of a local maximum value or a maximum value in the region of interest. For example, a center signal can be the signal of a pixel having the highest sensor signal among a plurality of sensor signals generated by pixels of the entire matrix or a region of interest within the matrix, wherein the region of interest can be predetermined or determinable within an image generated by the pixels of the matrix. The center signal can originate from a single pixel or a group of optical sensors. In the latter case, for example, the sensor signals of the group of pixels can be added, integrated, or averaged to determine the center signal. The group of pixels from which the center signal originates can be a group of adjacent pixels, such as pixels having a distance less than a predetermined distance from the actual pixel having the highest sensor signal, or it can be a group of pixels generating sensor signals within a predetermined range from the highest sensor signal. The group of pixels from which the center signal originates can be selected as large as possible to allow for the maximum dynamic range. The evaluation apparatus can be adapted to determine the center signal by integrating multiple sensor signals, for example, multiple pixels surrounding the pixel having the highest sensor signal. For example, the beam profile can be a trapezoidal beam profile, and the evaluation apparatus can be adapted to determine the integral of the trapezoid (especially the flat top of the trapezoid).

[0084] As outlined above, the center signal can typically be a single sensor signal, such as a sensor signal from a pixel at the center of the spot, or a combination of multiple sensor signals, such as a combination of sensor signals originating from a pixel at the center of the spot, or a secondary sensor signal derived by processing sensor signals derived from one or more of the aforementioned possibilities. The determination of the center signal can be performed electronically, as the comparison of sensor signals can be achieved fairly easily using conventional electronics, or it can be performed entirely or partially by software. Specifically, the center signal can be selected from the group including the following: the highest sensor signal; the average of a group of sensor signals within a predetermined tolerance range from the highest sensor signal; the average of sensor signals from a group of pixels including the pixel with the highest sensor signal and a predetermined adjacent pixel group; the sum of sensor signals from a group of pixels including the pixel with the highest sensor signal and a predetermined adjacent pixel group; the sum of a group of sensor signals within a predetermined tolerance range from the highest sensor signal; the average of a group of sensor signals greater than a predetermined threshold; the sum of a group of sensor signals greater than a predetermined threshold; the integral of sensor signals from a group of optical sensors including the optical sensor with the highest sensor signal and a predetermined adjacent pixel group; the integral of a group of sensor signals within a predetermined tolerance range from the highest sensor signal; and the integral of a group of sensor signals greater than a predetermined threshold.

[0085] Similarly, the term "sum signal" generally refers to a signal that includes essentially edge information of the bundle profile. For example, the sum signal can be derived by adding, integrating, or averaging sensor signals over a region of interest (ROI) within the entire matrix or the matrix, where the ROI can be predetermined or determinable within an image generated by the matrix's optical sensors. When adding, integrating, or averaging sensor signals, the actual optical sensors that generated the signals can be excluded from, or may be included in, the addition, integration, or averaging. The evaluation apparatus can be adapted to determine the sum signal by integrating the signal over the entire matrix or the ROI within the matrix. For example, the bundle profile can be a trapezoidal bundle profile, and the evaluation apparatus can be adapted to determine the integral over the entire trapezoid. Furthermore, when assuming a trapezoidal bundle profile, the determination of the edge and center signals can be substituted by deriving the edge and center signals through geometric considerations using an equivalent evaluation of the characteristics of the trapezoidal bundle profile (e.g., determining the slope and position of the edges and the height of the central flat top).

[0086] Similarly, center and edge signals can be determined by segmenting the bundle profile (such as circular segments). For example, the bundle profile can be divided into two segments by a secant or chord that does not pass through the center of the bundle profile. Thus, one segment essentially contains edge information, while the other essentially contains center information. For example, to further reduce the amount of edge information in the center signal, the edge signal can be further subtracted from the center signal.

[0087] The quotient Q can be a signal generated by combining a center signal and a sum signal. Specifically, the determination can include one or more of the following: forming a quotient of the center signal and the sum signal, or vice versa; forming a quotient of a multiple of the center signal and a multiple of the sum signal, or vice versa; forming a quotient of a linear combination of the center signals and a linear combination of the sum signals, or vice versa. Additionally or alternatively, the quotient Q can include any signal or combination of signals containing at least one piece of information regarding the comparison between the center signal and the sum signal.

[0088] As used herein, the term "longitudinal coordinate of an object" refers to the distance between the optical sensor and the object. The evaluation apparatus may be configured to determine the longitudinal coordinate using at least one predetermined relationship between the quotient Q and the longitudinal coordinate. This predetermined relationship may be one or more of empirical, semi-empirical, and analytically derived relationships. The evaluation apparatus may include at least one data storage device for storing the predetermined relationship, such as a lookup list or lookup table.

[0089] The evaluation device can be configured to perform at least one photon ratio depth sounding algorithm that calculates the distance with all reflectivity features of zero order and higher.

[0090] Image evaluation may include identifying the reflection features of the image. The evaluation apparatus may be configured to perform at least one image analysis and / or image processing to identify reflection features. Image analysis and / or image processing may use at least one feature detection algorithm. Image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image created by a sensor signal and at least one offset; inverting a sensor signal by inverting an image created by a sensor signal; forming a difference image between images created by a sensor signal at different times; background correction; decomposition into color channels; decomposition into hue, saturation, and luminance channels; frequency decomposition; singular value decomposition; applying a speckle detector; applying a corner detector; applying a Hessian filter determinant; applying a principal curvature-based region detector; applying a maximum stable extremum region detector; applying a generalized Hough transform; applying a ridge detector; applying an affine invariant feature detector; applying an affine-adapted interest point operator; applying a Harris affine region detector; applying a Hessian affine... Region detectors; application of scale-invariant feature transform; application of scale-space extremum detectors; application of local feature detectors; application of accelerated robust feature algorithms; application of gradient location and orientation histogram algorithms; application of histograms using orientation gradient descriptors; application of Deriche edge detectors; application of differential edge detectors; application of spatiotemporal interest point detectors; application of Moravec corner detectors; application of Canny edge detectors; application of Gaussian Laplacian filters; application of Gaussian difference filters; application of Sobel operators; application of Laplacian operators; application of Scharr operators; application of Prewitt operators; application of Roberts operators; application of Kirsch operators; application of high-pass filters; application of low-pass filters; application of Fourier transforms; application of Radon transforms; application of Hough transforms; application of wavelet transforms; thresholding; creation of binary images. Regions of interest can be determined manually by the user or automatically, such as by identifying features within images generated by optical sensors.

[0091] For example, the illumination source can be configured to generate and / or project a point cloud, such that multiple illumination regions are generated on an optical sensor (e.g., a CMOS detector). Furthermore, interference may exist on the optical sensor, such as interference caused by speckle and / or external light and / or multiple reflections. The evaluation apparatus can be adapted to determine at least one region of interest, for example, one or more pixels illuminated by the beam used to determine the longitudinal coordinates of an object. For example, the evaluation apparatus can be adapted to perform filtering methods, such as speckle analysis and / or edge filtering and / or object recognition methods.

[0092] The evaluation apparatus can be configured to perform at least one image correction. Image correction may include at least one background subtraction. The evaluation apparatus may be adapted to remove the effects of background light from the beam profile, for example, by imaging without further illumination.

[0093] The evaluation device is configured to locate an object by identifying reflective features produced by irradiated biological tissue. The evaluation device is configured to determine at least one reflective beam profile for each reflective feature. The evaluation device is configured to identify a reflective feature as produced by irradiated biological tissue if the reflective beam profile of the reflective feature meets at least one predetermined or predefined criterion. Otherwise, the evaluation device is configured to identify the reflective feature as background.

[0094] As used herein, the term "biological tissue" generally refers to biological material containing living cells. A detector can be a device for detecting biological tissue (particularly human skin), especially optical detection. The term "identification" arising from biological tissue can refer to determining and / or verifying whether a surface to be examined or tested is or includes biological tissue, particularly human skin, and / or distinguishing biological tissue, particularly human skin, from other tissues, particularly other surfaces, and / or distinguishing different types of biological tissue, such as distinguishing different types of human tissue, such as muscle, fat, organs, etc. For example, biological tissue can be or may include human tissue or a portion thereof, such as skin, hair, muscle, fat, organs, etc. For example, biological tissue can be or may include animal tissue or a portion thereof, such as skin, fur, muscle, fat, organs, etc. For example, biological tissue can be or may include plant tissue or a portion thereof. A detector can be adapted to distinguish animal tissue or a portion thereof from one or more of inorganic tissue, metallic surfaces, and plastic surfaces, such as those found on agricultural machinery or milking machines. A detector can be adapted to distinguish plant tissue or a portion thereof from one or more of inorganic tissue, metallic surfaces, and plastic surfaces, such as those found on agricultural machinery. The detector can be adapted to distinguish food and / or beverages from plates and / or glasses. It can be adapted to distinguish different types of food, such as fruit, meat, and fish. It can be adapted to distinguish cosmetics and / or applied cosmetics from human skin. It can be adapted to distinguish human skin from foam, paper, wood, displays, and screens. It can be adapted to distinguish human skin from clothing. It can be adapted to distinguish materials used in maintenance products and machine components, such as metal components. It can be adapted to distinguish between organic and inorganic materials. It can be adapted to distinguish human biological tissue from the surfaces of artificial or inanimate objects. The detector can be used, in particular, for non-therapeutic and non-diagnostic applications.

[0095] As used herein, the term "at least one predetermined or predefined criterion" refers to at least one property and / or value suitable for distinguishing biological tissue, particularly human skin, from other materials. A predetermined or predefined criterion may be, or may include, at least one predetermined or predefined value and / or threshold and / or threshold range relating to material properties. If the reflected beam profile satisfies at least one predetermined or predefined criterion, the reflective feature may be indicated as being generated by biological tissue. As used herein, the term "indication" refers to any indication, such as an electronic signal and / or at least one visual or auditory indication.

[0096] The evaluation apparatus can be configured to determine at least one material property m of an object by evaluating the beam profile of its reflection characteristics. As used herein, the term "material property" refers to at least one arbitrary characteristic of a material configured for characterizing and / or identifying and / or classifying the material. For example, a material property can be a property selected from: roughness, depth of light penetration through the material, characteristics that characterize the material as biological or non-biological, reflectivity, specular reflectivity, diffuse reflectivity, surface characteristics, translucency measures, scattering, specifically backscattering behavior, etc. At least one material property can be a property selected from: scattering coefficient, translucency, transparency, deviation from Lambertian surface reflection, speckle, etc. As used herein, the term "identifying at least one material property" means one or more of the following: determining the material property and assigning the material property to an object. The detector may include at least one database comprising lists and / or tables of predefined and / or predetermined material properties, such as lookup lists or lookup tables. The lists and / or tables of material properties can be determined and / or generated by performing at least one test measurement using the detector according to the invention, for example, by performing material testing using a sample having known material properties. The list and / or table of material properties may be determined and / or generated at the manufacturer's site and / or by the user of the detector. Material properties may be additionally assigned to a material classifier, such as one or more of the following: material name, material group, such as biological or non-biological material, translucent or non-translucent material, metallic or non-metallic, skin or non-skin, fur or non-fur, carpet or non-carpet, reflective or non-reflective, specular or non-specular, foam or non-foam, hair or non-hair, roughness group, etc. The detector may include at least one database containing lists and / or tables of material properties and associated material names and / or material groups.

[0097] For example, to avoid being bound by this theory, human skin can have a reflective profile, also represented as a backscattering profile, comprising the portion produced by backscattering of light from the surface, represented as surface reflection, and the portion produced by very diffuse reflection of light penetrating the skin, represented as the diffuse portion of backscattering. For the reflective profile of human skin, see “Lasertechnik in der Medizin: Grundlagen, Systeme, Anwendungen”, “Wirkung von Laserstrahlung auf Gewebe”, 1991, pp. 10 171-266, Jürgen Eichler, Theo Seiler, Springer Verlag, ISBN 0939-0979. The surface reflectance of skin increases with increasing wavelength towards the near-infrared. Furthermore, the penetration depth increases with increasing wavelength from visible light to the near-infrared. The diffuse portion of backscattering increases with the penetration depth of light. By analyzing the backscattering profile, these properties can be used to distinguish skin from other materials.

[0098] Specifically, the evaluation device can be configured to compare the reflected beam profile with at least one predetermined and / or pre-recorded and / or predefined beam profile. The predetermined and / or pre-recorded and / or predefined beam profile can be stored in a table or lookup table and can be determined empirically, for example, and, as an example, can be stored in at least one data storage device of the detector. For example, the predetermined and / or pre-recorded and / or predefined beam profile can be determined during the initial startup of the mobile device including the detector. For example, the predetermined and / or pre-recorded and / or predefined beam profile can be stored in at least one data storage device of the mobile device, for example, via software, specifically via an application downloaded from an app store, etc. If the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile are identical, the reflection feature can be identified as being generated by biological tissue. The comparison may include overlaying the reflected beam profile and the predetermined or predefined beam profile such that their intensity centers match. The comparison may include determining the deviation between the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile, such as the sum of squares of point-to-point distances. The evaluation device may be adapted to compare a determined deviation with at least one threshold, wherein if the determined deviation is less than and / or equal to the threshold, the surface is indicated as biological tissue and / or the detection of biological tissue is confirmed. The threshold may be stored in a table or lookup table and may be determined empirically, for example, and, as an example, may be stored in at least one data storage device of the detector.

[0099] Additionally or alternatively, to identify whether reflective features are generated by biological tissue, the evaluation device can be configured to apply at least one image filter to the image of the region. As further used herein, "image" refers to a two-dimensional function f(x, y), where a brightness and / or color value is given for any x, y location in the image. This can correspond to discretization of the location corresponding to the recorded pixels. Brightness and / or color can correspond to discretization of the bit depth of the optical sensor. As used herein, the term "image filter" refers to at least one mathematical operation applied to at least one specific region of the beam profile and / or beam profile. Specifically, the image filter Ф maps the image f or the region of interest in the image to a real number... in, Representing features, especially material features. Images can be limited by noise, and the same applies to features. Therefore, features can be random variables. Features can be normally distributed. If a feature is not normally distributed, it can be transformed into a normal distribution, such as through a Box-Cox transformation.

[0100] The evaluation device can be configured to determine at least one material feature by applying at least one material-related image filter Ф2 to an image. As used herein, the term "material-correlated" image filter refers to an image with material-correlated output. The output of a material-correlated image filter in this paper represents "material features". "or material-related characteristics" The material characteristics may be, or may include, at least one piece of information about at least one material property of the surface of the area where the reflective characteristics have been generated.

[0101] Material-related image filters can be at least one filter selected from the following: brightness filter; speckle shape filter; square norm gradient; standard deviation; smoothness filter, such as a Gaussian filter or median filter; contrast filter based on gray-level occurrence; energy filter based on gray-level occurrence; homogeneity filter based on gray-level occurrence; dissimilarity filter based on gray-level occurrence; law-based energy filter; threshold area filter; or linear combinations thereof; or further material-related image filters. 2other The further material-related image filter Ф 2other via |ρФ 2other，Фm |≥0.40 is related to one or more of the following, or linear combinations thereof: brightness filter, speckle shape filter, square norm gradient, standard deviation, smoothness filter, energy filter based on gray-level occurrence, homogeneous filter based on gray-level occurrence, dissimilar filter based on gray-level occurrence, law-based energy filter, or threshold region filter, where Ф mIt is one of the following: a brightness filter, a speckle shape filter, a square norm gradient filter, a standard deviation filter, a smoothness filter, an energy filter based on gray-level occurrence, a homogeneity filter based on gray-level occurrence, a dissimilarity filter based on gray-level occurrence, a law-based energy filter, or a threshold region filter, or a linear combination thereof. This further describes the material-related image filter Ф. 2other It can be done through |ρ Ф2other,Фm |≥0.60, preferably through |ρ Ф2other,Фm |≥0.80 is correlated with one or more of the material-related image filters Фm.

[0102] The material-dependent image filter can be at least one arbitrary filter Ф that passes the hypothesis test. As used herein, the term "passes the hypothesis test" means the fact that the null hypothesis H0 is rejected and the alternative hypothesis H1 is accepted. The hypothesis test may include testing the material-dependent nature of the image filter by applying it to a predefined dataset. The dataset may include multiple beam profile images. As used herein, the term "beam profile image" refers to N B The sum of Gaussian radial basis functions,

[0103]

[0104]

[0105] Where, N B Each of the Gaussian radial basis functions is derived from the center (x) lk ,y lk ), pre-factor a lk The exponential factor α = 1 / ∈ is defined. The exponential factor is the same for all Gaussian functions in all images. Center position x lk ,y lk For all images f k : They are all the same. Each bundle contour image in the dataset corresponds to a material classifier and a distance. The material classifier can be a label, such as "material A", "material B", etc. This can be achieved by using the above-described method targeting f. k The formula for (x,y) combined with the following parameter table generates the bundle profile image:

[0106]

[0107] The values ​​of x and y are integers corresponding to pixels, where Images can have a pixel size of 32x32. A dataset of bundle contour images can be obtained by using the methods described above for f. k The formula is generated by combining the parameter set to obtain f. k A continuous description. The value of each pixel in a 32x32 image can be obtained by using f. kThe value f is obtained by inserting integer values ​​from 0, ..., 31 for x and y. For example, for pixel (6, 9), the value f can be calculated. k (6, 9).

[0108] Subsequently, for each image f k It is possible to calculate the eigenvalues ​​corresponding to filter Φ. Among them, z k It corresponds to the image f from a predefined dataset. k The distance value. This produces the corresponding eigenvalues. The dataset. The hypothesis test can use a null hypothesis that the filter does not distinguish between material classifiers. The null hypothesis can be given by H0: μ1 = μ2 = ... = μ J Given, where μ m It corresponds to the eigenvalue The expected value for each material group. The index m represents the material group. The hypothesis test can be used as an alternative hypothesis for the filter to distinguish between at least two material classifiers. The alternative hypothesis can be derived from H1: μ m ≠μ m′ As used herein, the term "not distinguishable between material classifiers" means that the expected values ​​of the material classifiers are the same. As used herein, the term "distinguishing material classifiers" means that at least two expected values ​​of the material classifiers are different. As used herein, "distinguishing at least two material classifiers" is used synonymously with "suitable material classifier". Hypothesis testing may include at least one analysis of variance (ANOVA) on the generated eigenvalues. In particular, hypothesis testing may include determining the mean of the eigenvalues ​​for each material J, i.e., the total J mean. For m∈[0,1,…,J-1], where N m Give the number of eigenvalues ​​for each material J in a predefined dataset. Assume the test may include determining the average of all N eigenvalues. Hypothesis testing may include determining the mean sum of squares within the following:

[0109]

[0110] Assume the test may include the average sum of squares among the following:

[0111]

[0112] Assume that the test may include performing the F test:

[0113] ο Where d1 = NJ, d2 = J-1,

[0114] οF(x)=1–CDF(x)

[0115] οp = F(mssb / mssw)

[0116] In this article, I x It is a regularized incomplete beta function. Among them, Euler-Beta function as well as It is an incomplete beta function. The image filter passes the hypothesis test if the p-value is less than or equal to a predefined significance level. The filter passes the hypothesis test if p ≤ 0.075, preferably p ≤ 0.05, more preferably p ≤ 0.025, and most preferably p ≤ 0.01. For example, with a predefined significance level of α = 0.075, if the p-value is less than α = 0.075, the image filter passes the hypothesis test. In this case, the null hypothesis H0 can be rejected, and the alternative hypothesis H1 can be accepted. Therefore, the image filter distinguishes at least two material classifiers. Thus, the image filter passes the hypothesis test.

[0117] In the following description, we assume that the reflective image includes at least one reflective feature, specifically a blob image, and then describe the image filter. The blob image f can be expressed by the function f:R 2 →R ≥0 Given, where the background of image f may have been subtracted. However, other reflection features may be possible.

[0118] For example, a material-related image filter can be a brightness filter. A brightness filter can return a measure of the brightness of a spot as a material feature. The material feature can be determined by the following formula:

[0119]

[0120] Where f is the speckle image. The distance between the specks is denoted by z, where z can be obtained, for example, by using defocusing depth sounding or photon ratio depth sounding techniques and / or by using triangulation techniques. The surface normal of the material is given by n∈R. 3 Given and obtainable as the normal to the surface spanned by at least three measurement points. Vector d ray ∈R 3 d is the direction vector of the light source. Since the position of the spot is known using defocusing depth sounding or photon ratio depth sounding and / or using triangulation techniques, where the position of the light source is known as a parameter of the detector system, d ray It is the difference vector between the spot and the position of the light source.

[0121] For example, a material-related image filter could be a filter with an output that depends on the shape of the speckle. This material-related image filter could return a value related to the translucency of the material as a material feature. The translucency of the material affects the shape of the speckle. The material feature can be given by the following formula:

[0122]

[0123] Where 0 < α, β < 1 are the weights of the speckle height h, and H represents the symmetry function, i.e., H(x) = 1 : x ≥ 0, H(x) = 0 : x < 0. The speckle height h can be determined by the following formula:

[0124]

[0125] Among them, B r It is the inner circle of a spot with radius r.

[0126] For example, a material-related image filter can be a squared norm gradient. This material-related image filter can return values ​​as material features related to measures of the soft and hard transitions and / or roughness of the speckle. Material features can be defined by the following formula:

[0127]

[0128] For example, a material-related image filter can be a standard deviation. The standard deviation of a blob can be determined by the following formula:

[0129]

[0130] Where μ is the average value given by μ=∫(f(x))dx.

[0131] For example, a material-related image filter can be a smoothness filter, such as a Gaussian filter or a median filter. In one embodiment of the smoothness filter, the image filter may reference observations of volume scattering exhibiting less speckle contrast compared to diffuse scattering materials. This image filter can quantify the smoothness of the speckle corresponding to the speckle contrast as a material feature. The material feature can be determined by the following formula:

[0132]

[0133] Where F is a smoothness function, such as a median filter or a Gaussian filter. This image filter may include division by a distance z, as described in the formula above. The distance z can be determined, for example, using defocusing depth sounding or photon ratio depth sounding techniques and / or by using triangulation techniques. This allows the filter to be distance-sensitive. In one embodiment of the smoothness filter, the smoothness filter may be based on the standard deviation of the extracted speckle noise pattern. The speckle noise pattern N can be empirically described by the following formula:

[0134] f(x) = f0(x)·(N(X)+1),

[0135] Where f0 is the despeckle image, and N(X) is the noise term used to model the speckle pattern. Computing the despeckle image can be computationally difficult. Therefore, the despeckle image can be approximated using a smoothed version of f, i.e., f0 ≈ F(f), where F is a smoothing operator similar to a Gaussian filter or median filter. Thus, the approximation of the speckle pattern can be given by the following equation:

[0136]

[0137] The material characteristics of the filter can be determined by the following formula:

[0138]

[0139] Where Var represents the variance function.

[0140] For example, an image filter can be a contrast filter based on grayscale occurrence. This material filter can be based on the grayscale occurrence matrix M. f,ρ (g1g2)=[p g1,g2 ], while p g1,g2 The occurrence rate of the gray-scale combination is (g1,g2) = [f(x1,y1),f(x2,y2)], and the relation ρ defines the distance between (x1,y1) and (x2,y2), which is ρ(x,y) = (x+a,y+b), where a and b are selected from 0 and 1.

[0141] The material characteristics of a contrast filter based on grayscale appearance can be given by the following formula:

[0142]

[0143] For example, an image filter can be an energy filter based on grayscale occurrence. This material filter is based on the grayscale occurrence matrix defined above.

[0144] The material characteristics of an energy filter based on grayscale appearance can be given by the following formula:

[0145]

[0146] For example, an image filter can be a homogeneous filter based on grayscale occurrence. This material filter is based on the grayscale occurrence matrix defined above.

[0147] The material characteristics of a homogeneous filter based on grayscale appearance can be given by the following formula:

[0148]

[0149] For example, an image filter can be a dissimilarity filter based on grayscale occurrence. This material filter is based on the grayscale occurrence matrix defined above.

[0150] The material characteristics of the phase filter based on the appearance of gray levels can be given by the following formula:

[0151]

[0152] For example, an image filter could be a law-based energy filter. This material filter could be based on the law vectors L5 = [1,4,6,4,1] and E5 = [-1,-2,0,-2,-1], and the matrix L5(E5). T And E5 (L5) T .

[0153] Image f k Convolution with these matrices:

[0154]

[0155] as well as

[0156]

[0157]

[0158]

[0159] The material characteristics of the energy filter according to the law can be determined by the following formula:

[0160]

[0161] For example, a material-related image filter can be a threshold region filter. The material feature can be associated with two regions in the image plane. The first region Ω1 can be a region where the function f is greater than a multiple of the maximum value f by α. The second region Ω2 can be a region where the function f is less than a multiple of the maximum value f by α but greater than a threshold ε of the maximum value f. Preferably, α can be 0.5, and ε can be 0.05. Due to speckle or noise, these regions may not only correspond to the inner and outer circles around the center of the speckle. As an example, Ω1 may include speckle or unconnected regions in the outer circle. The material feature can be determined by the following formula:

[0162]

[0163] Where Ω1={x|f(x)>α·max(f(x))} and Ω2={x|ε·max(f(x))} <f(x)<α·max(f(x))}。

[0164] The evaluation device can be configured to use material characteristics The material properties of the surface with the reflective features are determined by at least one predetermined relationship between the material properties and the surface with the reflective features. This predetermined relationship can be one or more of empirical, semi-empirical, and analytically derived relationships. The evaluation apparatus may include at least one data storage device, such as a lookup list or lookup table, for storing the predetermined relationships.

[0165] The evaluation device is configured to identify a reflective feature as being generated by irradiated biological tissue if the material properties corresponding to the reflective feature meet at least one predetermined or predefined criterion. If the material properties indicate "biological tissue," the reflective feature can be identified as being generated by biological tissue. If the material properties are below or equal to at least one threshold or range, the reflective feature can be identified as being generated by biological tissue, wherein the reflective feature is identified as being generated by biological tissue and / or the detection of biological tissue is confirmed if a determined deviation is below and / or equal to the threshold. At least one threshold and / or range can be stored in a table or lookup table and can be determined empirically, for example, and, as an example, can be stored in at least one data storage device of the detector. Otherwise, the evaluation device is configured to identify the reflective feature as background. Therefore, the evaluation device can be configured to assign depth information and material properties, such as whether skin is present or not, to each projection spot.

[0166] After determining the vertical coordinate z, subsequent evaluation can be performed. To determine material properties so that information about the longitudinal coordinate z can be considered for evaluation.

[0167] The evaluation apparatus is configured to segment an image of the region using at least one segmentation algorithm. As used herein, the term "fragment" can refer to a portion of an image and / or a sub-part of an image and / or a region of an image. A fragment can include a set of pixels. As used herein, the term "segmentation" can refer to the process of dividing an image into multiple fragments. Segmentation can include assigning at least one label to each pixel of the image such that pixels with the same label share at least one feature. Labels can be assigned under predefined targets. Segmentation can be binary segmentation. Binary segmentation can include labeling pixels of the image as "skin pixels" and "background pixels." All non-skin pixels can be considered background. Skin pixels can be considered foreground seeds, also represented as seed points, and used as input to image-based segmentation algorithms. For more information on image segmentation, see https: / / en.wikipedia.org / wiki / Image_segmentation. Segmentation algorithms can include at least one region-growing image segmentation method.

[0168] Segmentation algorithms can be configured to fuse and / or consider prior knowledge. Specifically, segmentation algorithms can be configured to fuse and / or consider material information determined from bundle contour analysis. This allows for the acquisition or filtering of positional data based solely on the movement of visible skin. Conversely, in conventional settings, identification and separation, such as what constitutes a finger / arm and what does not, must be performed first. In addition to using material information, the evaluation device can be configured to consider depth information to segment objects from the background based on depth maps, particularly hand regions in images.

[0169] Segmentation algorithms can be based on energy or cost functions, such as one or more of graph cutting, level sets, fast walk, and Markov random field methods. Image segmentation can be driven by color homogeneity and edge indices, where seed points constitute edge and color homogeneity criteria. As used herein, the term "seed point" can refer to the starting point of the segmentation algorithm. Reflectance features identified as produced by irradiating biological tissue are used as seed points, and reflectance features identified as background are used as background seed points for the segmentation algorithm. These seed points can constitute edge and color homogeneity criteria, thus providing an appropriate initialization for target segmentation: "What color and / or reflectance and / or appearance does the skin in the image have?".

[0170] For example, graph slicing segmentation can be used. Graph slicing segmentation can be configured to incorporate prior knowledge because it also provides real-time capabilities when combined with pre-clustering algorithms such as superpixels, see R.Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,” IEEE TPAMI, vol.34, no.11, 2012. For example, modified lazy-capture graph slicing can be used. Regarding the modified lazy snapping graph cutting, refer to "Lazy Snapping" by Li, Yin and Sun, Jian and Tang, Chi-Keung and Shum, and Heung-Yeung, ACM Trans. Graph, vol. 23, no. 3, 2004. The Gibbs energy term can include additional terms to incorporate depth information used in the graph cutting segmentation routine. Instead of "lazy snapping" as proposed by Li, Yin and Sun, Jian and Tang, Chi-Keung and Shum, and Heung-Yeung, the watershed-based pre-cutting proposed in ACM Trans. can use superpixel clustering to ensure real-time capabilities.

[0171] The evaluation apparatus is configured to consider segmented images and depth maps to determine the spatial position and / or orientation of an object. As used herein, the term "segmented image" may refer to an image following a segmentation process and / or the result of a segmentation algorithm. As used herein, the term "position" refers to at least one piece of information regarding the spatial position and / or orientation of an object and / or at least a portion of an object. This position may be a longitudinal coordinate or a longitudinal coordinate that may help determine the object point. Additionally or alternatively, one or more other pieces of information regarding the position and / or orientation of an object and / or at least a portion of an object may be determined. As an example, additionally, at least one lateral coordinate of an object and / or at least a portion of an object may be determined. Thus, the position of an object may imply at least one longitudinal coordinate of an object and / or at least a portion of an object. Additionally or alternatively, the position of an object may imply at least one lateral coordinate of an object and / or at least a portion of an object. Additionally or alternatively, the position of an object may imply at least one orientation information of the object, indicating the object's orientation in space.

[0172] The evaluation device can be configured to determine at least one hand pose or gesture based on the object's position and / or orientation in space. The evaluation device can be configured to recognize poses from segmented images, particularly from portions of the segmented image labeled as objects. The evaluation device can be configured to identify image coordinates of the palm and fingers in the segmented image. Palm and finger recognition can be obtained from segmentation. The evaluation device can be configured to extract features, such as color and / or brightness and / or gradient values, from portions of the segmented image labeled as objects. For example, standard OpenCV routines can be used to detect the palm and fingertips.

[0173] The evaluation device can be configured to determine at least one three-dimensional finger vector by considering image coordinates of the palm and fingertips, as well as a depth map. As used herein, the term "three-dimensional finger vector" can refer to a 3D vector spanned by points at the fingertips and the palm. A 3D finger vector can specify the position and orientation of the finger in space. The evaluation device can be configured to determine 3D finger vectors from images and depth maps. 3D finger vectors can provide a basis for hand pose estimation and scene interpretation. For gesture recognition, the evaluation device can be configured to use at least one classifier, such as a Hidden Markov Model (HMM), Support Vector Machine (SVM), Conditional Random Field (CRF), etc. The classifier can be configured to distinguish gestures.

[0174] Using the beam profile analysis sensor fusion concept according to the present invention has the advantages of providing robust detection results, not requiring complex model assumptions, a single sensor concept, easy integration, and real-time capability.

[0175] In another aspect, the present invention discloses a method for pose detection, wherein a detector according to the invention is used. The method includes the following steps:

[0176] a) Projecting at least one illumination pattern comprising multiple illumination features onto at least one area comprising at least one object, wherein the object at least partially comprises at least one human hand;

[0177] b) Determine at least one image of the region using at least one optical sensor having at least one photosensitive region, wherein the image includes a plurality of reflection features generated by the region in response to illumination by illumination features;

[0178] c) Determine at least one depth map of the area by using at least one evaluation device to determine at least one depth information for each reflection feature;

[0179] d) Locating an object by using an evaluation device to identify reflective features produced by irradiating biological tissue, wherein at least one reflective beam profile of each reflective feature is determined, wherein a reflective feature is identified as produced by irradiating biological tissue if the reflective beam profile of the reflective feature satisfies at least one predetermined or predefined criterion, wherein the reflective feature is otherwise identified as background.

[0180] e) The image of the region is segmented using an evaluation device and at least one segmentation algorithm, wherein reflective features identified as produced by irradiation of biological tissue are used as seed points and reflective features identified as background are used as background seed points for the segmentation algorithm.

[0181] f) Determine the location and / or orientation of an object in space by using an evaluation device that takes into account segmented images and depth maps.

[0182] The method steps can be performed in a given order or in a different order. Furthermore, one or more additional method steps not listed may be present. Additionally, one, more, or even all method steps may be performed repeatedly. For details, options, and definitions, refer to the detector discussed above. Therefore, specifically, as described above, the method may include the use of a detector according to the invention (such as according to one or more embodiments given above or in more detail below).

[0183] The at least one evaluation device may be configured to execute at least one computer program, such as being configured to perform or support one or more, or even all, of the method steps according to the method of the invention. As an example, one or more algorithms capable of detecting poses may be implemented.

[0184] In a further aspect of the invention, uses of the detectors according to the invention are proposed, such as uses of one or more detectors according to the embodiments given above or given in more detail below, selected for the purpose of use from the group consisting of: location measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photographic applications; imaging applications or camera applications; map building applications for generating maps of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotic applications; quality control applications; manufacturing applications.

[0185] For further uses of the detector of the present invention, see WO 2018 / 091649 A1, WO 2018 / 091638 A1 and WO 2018 / 091640 A1, the contents of which are incorporated herein by reference.

[0186] In general, the following embodiments are considered preferred in the context of this invention:

[0187] Example 1: A detector for pose detection, including

[0188] - At least one illumination source configured to project at least one illumination pattern comprising a plurality of illumination features onto at least one area comprising at least one object, wherein the object at least partially comprises at least one human hand;

[0189] - At least one optical sensor having at least one photosensitive area, wherein the optical sensor is configured to determine at least one image of the area, wherein the image includes a plurality of reflection features generated by the area in response to illumination by the illumination feature;

[0190] - At least one evaluation device, wherein the evaluation device is configured to determine at least one depth map of the region by determining at least one depth piece of information for each reflection feature.

[0191] The evaluation device is configured to locate an object by identifying the reflectance features produced by irradiating biological tissue. The evaluation device is configured to determine at least one reflectance beam profile for each reflectance feature. The evaluation device is configured to identify the reflectance feature as produced by irradiating biological tissue if the reflectance beam profile of the reflectance feature meets at least one predetermined or predefined criterion. Otherwise, the evaluation device is configured to identify the reflectance feature as background.

[0192] The evaluation device is configured to segment the image of the region using at least one segmentation algorithm, wherein the reflectance features identified as being produced by irradiation of biological tissue are used as seed points, and the reflectance features identified as background are used as background seed points for the segmentation algorithm.

[0193] The evaluation device is configured to take into account the segmented image and the depth map to determine the position and / or orientation of the object in space.

[0194] Example 2: According to the detector described in the previous example, the evaluation device is configured to identify the image coordinates of the palm and fingers in the segmented image, wherein the evaluation device is configured to determine at least one three-dimensional finger vector by considering the image coordinates of the palm and fingers and the depth map.

[0195] Example 3: The detector according to any one of the foregoing embodiments, wherein the evaluation device is configured to determine at least one hand gesture or gesture based on the object's position in space and / or its orientation.

[0196] Example 4: The detector according to any one of the foregoing embodiments, wherein the segmentation algorithm is based on an energy or cost function, such as one or more of graph cutting, level set, fast travel and Markov random field methods.

[0197] Example 5: The detector according to any one of the preceding embodiments, wherein the segmentation of the image is driven by color homogeneity and edge indices, wherein the seed points constitute the edge and color homogeneity criteria.

[0198] Example 6: A detector according to any one of the foregoing embodiments, wherein the evaluation device is configured to compare the reflection beam profile of each of the reflection features with at least one predetermined and / or pre-recorded and / or predefined beam profile.

[0199] Example 7: A detector according to a previous embodiment, wherein the comparison includes overlapping the reflected beam profile with the predetermined and / or pre-recorded and / or predefined beam profile such that their intensity centers match, wherein the comparison includes determining a deviation between the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile, wherein the evaluation device is configured to compare the determined deviation with at least one threshold, wherein if the determined deviation is less than and / or equal to the threshold, the reflected feature is indicated as biological tissue.

[0200] Example 8: The detector according to any one of the foregoing embodiments, wherein the evaluation device is configured to determine the depth information of each of the reflection features by one or more of the following techniques: photon ratio depth sounding, structured light, beam profile analysis, time of flight, shape recovery from motion, focused depth sounding, triangulation, defocused depth sounding, and stereo sensor.

[0201] Example 9: A detector according to any one of the foregoing embodiments, wherein the evaluation device is configured to determine the depth information of each of the reflection features by using photon ratio depth sounding, wherein the evaluation device is configured to determine at least one first region and at least one second region of the beam profile of at least one of the reflection features, wherein the evaluation device is configured to integrate the first region and the second region, wherein the evaluation device is configured to derive the quotient Q by one or more of the following: dividing the integrated first region and the integrated second region, dividing the integrated first region and the integrated second region by a multiple thereof, and dividing the integrated first region and the integrated second region by a linear combination thereof.

[0202] Example 10: The detector according to the previous embodiment, wherein the first region of the reflected beam profile includes substantially edge information of the reflected beam profile, and the second region of the reflected beam profile includes substantially center information of the reflected beam profile, and / or wherein the first region of the reflected beam profile includes information substantially about the left side of the reflected beam profile, and the second region of the reflected beam profile includes information substantially about the right side of the reflected beam profile.

[0203] Example 11: A detector according to any one of the foregoing two embodiments, wherein the evaluation device is configured to derive the quotient Q by formula

[0204]

[0205] Where x and y are the horizontal coordinates, A1 and A2 are the first and second regions of the reflected beam profile, respectively, and E(x, y) represents the reflected beam profile.

[0206] Example 12: The detector according to any one of the preceding embodiments, wherein the illumination source is configured to generate the at least one illumination pattern in the near-infrared region (NIR).

[0207] Example 13: The detector according to any one of the foregoing embodiments, wherein the optical sensor includes at least one CMOS sensor.

[0208] Example 14: A method for pose detection, wherein at least one detector according to the foregoing embodiments is used, the method comprising the following steps:

[0209] a) Projecting at least one illumination pattern comprising multiple illumination features onto at least one area comprising at least one object, wherein the object at least partially comprises at least one human hand;

[0210] b) Determine at least one image of the region using at least one optical sensor having at least one photosensitive region, wherein the image includes a plurality of reflection features generated by the region in response to illumination by illumination features;

[0211] c) Determine at least one depth map of the region by using at least one evaluation device to determine at least one depth information for each of the reflection features;

[0212] d) The object is located by using the evaluation device and by identifying the reflection features produced by irradiating biological tissue, wherein at least one reflection beam profile of each reflection feature is determined, wherein the reflection feature is identified as produced by irradiating biological tissue if the reflection beam profile of the reflection feature meets at least one predetermined or predefined criterion, wherein the reflection feature is otherwise identified as background.

[0213] e) The image of the region is segmented by using the evaluation device and by using at least one segmentation algorithm, wherein the reflection features identified as generated by irradiation of biological tissue are used as seed points, and the reflection features identified as background are used as background seed points for the segmentation algorithm;

[0214] f) Determine the position and / or orientation of the object in space by using the evaluation device, taking into account the segmented image and the depth map.

[0215] Example 15: An application of a detector according to any one of the foregoing embodiments involving the detector, for the purpose of use, selected from the group consisting of: driver monitoring; in-vehicle monitoring; posture tracking; security applications; safety applications; human-machine interface applications; information technology applications; agricultural applications; crop protection applications; medical applications; maintenance applications; cosmetic applications. Attached Figure Description

[0216] Other optional details and features of the invention will become apparent from the following description of preferred exemplary embodiments in conjunction with the dependent claims. In such cases, a particular feature may be implemented separately or in combination with other features. The invention is not limited to exemplary embodiments. Exemplary embodiments are schematically illustrated in the accompanying drawings. The same reference numerals in the various drawings refer to the same elements or elements having the same function, or elements that correspond to each other in terms of their function.

[0217] Specifically, in the diagram:

[0218] Figure 1 An embodiment of the detector according to the invention is shown; and

[0219] Figures 2A to 2C The posture detection according to the present invention is illustrated; and Detailed Implementation

[0220] Figure 1 An embodiment of a detector 110 for pose detection is shown schematically. A pose can include body postures, such as the posture of at least a part of the body, particularly gestures. A pose can be static or dynamic. A pose can include movement of at least a part of the hand (such as fingers), movement of one or both hands, the face, or other parts of the body. Pose detection can include determining the presence or absence of a pose and / or pose recognition. Pose recognition can include interpreting human poses using mathematical algorithms.

[0221] The detector 110 includes at least one illumination source 112 configured to project at least one illumination pattern comprising a plurality of illumination features onto at least one region 114 comprising at least one object 116. The object 116 at least partially comprises at least one human hand.

[0222] Illumination source 112 can be configured to provide an illumination pattern for illuminating region 114. Illumination source 112 can be adapted to directly or indirectly irradiate region 114, wherein the illumination pattern is reflected or scattered by the surface of region 114, and thus at least partially directed to at least one optical sensor 118. Illumination source 112 can be configured to irradiate region 114, for example, by directing a light beam toward region 114, which reflects the light beam. Illumination source 112 can be configured to generate an illumination beam for irradiating region 114.

[0223] The area may include at least one object 116 and its surrounding environment. For example, the object may be at least one object selected from the group consisting of: a scene, a person such as a human, wood, carpet, foam, an animal such as a cow, a plant, a block of tissue, metal, a toy, a metallic object, a beverage, food such as fruit, meat, or fish, a plate, cosmetics, applied cosmetics, clothing, fur, hair, skincare products, plants, a body, a part of a body, organic materials, inorganic materials, reflective materials, a screen, a display, a wall, a piece of paper, or a photograph. Object 116 may include at least one surface on which an illumination pattern is projected. The surface may be adapted to at least partially reflect the illumination pattern back to detector 110. Object 116 may specifically be a human body or at least a part of a human body, such as at least one arm, at least one hand, at least one finger, or a face. Object 116 at least partially includes at least one human hand. Object 116 may be a human hand and / or at least one finger and / or at least a part of the palm.

[0224] Illumination source 112 may include at least one light source. Illumination source 112 may include multiple light sources. Illumination source 112 may include artificial illumination sources, particularly at least one laser source and / or at least one incandescent lamp and / or at least one semiconductor light source, such as at least one light-emitting diode, particularly organic and / or inorganic light-emitting diodes. As an example, the light emitted by the illumination source may have a wavelength of 300 to 1000 nm (especially 500 to 1000 nm). Additionally or alternatively, light in the infrared spectral range may be used, such as in the range of 780 nm to 3.0 μm. Specifically, light in a portion of the near-infrared region may be used, in which silicon photodiodes are specifically suitable for the range of 700 nm to 1100 nm. Illumination source 114 may be configured to generate at least one illumination pattern in the infrared region. Using light in the near-infrared region allows the light to be undetectable by the human eye or only weakly detected, and can still be detected by silicon sensors, particularly standard silicon sensors.

[0225] Illumination source 112 can be configured to emit light at a single wavelength. Specifically, the wavelength can be in the near-infrared region. In other embodiments, illumination source 112 can be configured to emit light with multiple wavelengths, thereby allowing additional measurements in other wavelength channels.

[0226] Irradiation source 112 may be or may include at least one multi-beam light source. For example, irradiation source 112 may include at least one laser source and one or more diffractive optical elements (DOEs). Specifically, irradiation source 112 may include at least one laser and / or laser source. Various types of lasers can be used, such as semiconductor lasers, dual heterostructure lasers, external cavity lasers, split-confined heterostructure lasers, quantum cascade lasers, distributed Bragg reflector lasers, polaron lasers, hybrid silicon lasers, extended cavity diode lasers, quantum dot lasers, volume Bragg grating lasers, indium arsenide lasers, transistor lasers, diode-pumped lasers, distributed feedback lasers, quantum well lasers, interband cascade lasers, gallium arsenide lasers, semiconductor ring lasers, extended cavity diode lasers, or vertical cavity surface-emitting lasers. Additionally or alternatively, non-laser light sources, such as LEDs and / or bulbs, may be used. Irradiation source 112 may include one or more diffractive optical elements (DOEs) adapted to generate an irradiation pattern. For example, the illumination source 112 may be adapted to generate and / or project a point cloud. For instance, the illumination source may include one or more of the following: at least one digital light processing projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one light-emitting diode array; at least one laser source array. Using at least one laser source as the illumination source is particularly preferred due to its generally defined beam profile and other characteristics of operability. The illumination source 112 may be integrated into the housing 120 of the detector 110.

[0227] One or more light beams generated by the illumination source 112 can typically propagate parallel to or tilted relative to the optical axis 122, for example, including an angle with the optical axis. The detector 110 can be configured such that one or more light beams propagate from the detector 110 toward the region 114 along the optical axis of the detector 110. For this purpose, the detector 110 may include at least one reflective element, preferably at least one prism, for deflecting the illumination beam onto the optical axis 122. As an example, one or more light beams, such as a laser beam, and the optical axis 122 may include an angle of less than 10°, preferably less than 5°, or even less than 2°. However, other embodiments are also possible. Furthermore, one or more light beams can be on or off the optical axis. As an example, one or more light beams can be parallel to the optical axis 122, with a distance of less than 10 mm from the optical axis, preferably less than 5 mm, or even less than 1 mm, or even coincide with the optical axis.

[0228] The illumination pattern may include at least one illumination feature, which includes at least a portion suitable for illuminating the illuminating area 114. The illumination pattern may include a single illumination feature. The illumination pattern may include multiple illumination features. The illumination pattern may be selected from the group consisting of: at least one dot pattern; at least one line pattern; at least one stripe pattern; at least one checkerboard pattern; at least one pattern including an arrangement of periodic or aperiodic features. The illumination pattern may include regular and / or constant and / or periodic patterns, such as triangular patterns, rectangular patterns, hexagonal patterns, or patterns including further convex tiles. The illumination pattern may exhibit at least one illumination feature selected from the group consisting of: at least one dot; at least one line; at least two lines, such as parallel lines or intersecting lines; at least one dot and one line; at least one arrangement of periodic or aperiodic features; at least one arbitrarily shaped feature. The illumination pattern may include at least one pattern selected from the group consisting of: at least one point pattern, particularly a pseudo-random point pattern; a random point pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one predictable feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including convex uniform tiles; at least one line pattern including at least one line; at least one line pattern including at least two lines, such as parallel or intersecting lines. For example, illumination source 112 may be adapted to generate and / or project a point cloud. Illumination source 112 may include at least one light projector adapted to generate a point cloud such that the illumination pattern may include multiple point patterns. Illumination source 112 may include at least one mask adapted to generate an illumination pattern from at least one light beam generated by illumination source 112.

[0229] The distance between two features of the illumination pattern and / or the area of ​​at least one illumination feature may depend on the blurring circle in the image. As outlined above, illumination source 112 may include at least one light source configured to generate at least one illumination pattern. Specifically, the illumination source includes at least one laser source and / or at least one laser diode designated for generating laser radiation. Illumination source 112 may include at least one diffractive optical element (DOE). Detector 110 may include at least one point projector, such as at least one laser source and DOE, adapted to project at least one periodic dot pattern.

[0230] Illumination source 112 can illuminate at least one object 116 with an illumination pattern. The illumination pattern may include multiple points. These points are shown as a beam of light 124 emitted from illumination source 112.

[0231] Detector 110 includes at least one optical sensor 118 having at least one photosensitive area 126. The optical sensor 118 is configured to determine at least one image 128 of the area 114. Examples of images are described in... Figure 2A As shown in the figure. Image 128 includes a plurality of reflective features 130 generated by region 114 in response to illumination by the illumination features. Detector 110 may include a plurality of optical sensors 118, each optical sensor 118 having a photosensitive region 126. Preferably, the photosensitive region 126 may be oriented substantially perpendicular to the optical axis 122 of detector 110.

[0232] The optical sensor 118 may specifically be or may include at least one photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, and most preferably a silicon photodetector. Specifically, the optical sensor 118 may be sensitive in the infrared spectral range. The optical sensor 118 may include at least one sensor element comprising a pixel matrix. All pixels in the matrix or at least one group of optical sensors in the matrix may specifically be identical. Specifically, groups of identical pixels in the matrix may be provided for different spectral ranges, or all pixels may have the same spectral sensitivity. Furthermore, the pixels may have the same size and / or be identical in their electronic or optoelectronic properties. Specifically, the optical sensor 118 may be or may include at least one inorganic photodiode that is sensitive in the infrared spectral range, preferably in the range of 700 nm to 3.0 micrometers. Specifically, the optical sensor 118 may be sensitive in a portion of the near-infrared region, in which the silicon photodiode is specifically suited for the range of 700 nm to 1100 nm. Infrared optical sensors that can be used for the optical sensor are commercially available infrared optical sensors, such as the TrinamiX from Ludwigshafen am Rhein (Germany) D-67056. TM The trademark name launched by GmbH is Hertzstueck TM Commercially available infrared optical sensors are available. Therefore, as an example, optical sensor 118 may include at least one optical sensor of an inherent photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, and HgCdTe photodiodes. Additionally or alternatively, optical sensor 118 may include at least one optical sensor of an inherent photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of: Ge:Au photodiodes, Ge:Hg photodiodes, Ge:Cu photodiodes, Ge:Zn photodiodes, Si:Ga photodiodes, and Si:As photodiodes. Additionally or alternatively, optical sensor 118 may include at least one photoconductivity sensor, such as a PbS or PbSe sensor, a radiative thermal meter, preferably selected from V0 radiative thermal meters and amorphous Si radiative thermal meters.

[0233] Optical sensor 118 can be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. Specifically, optical sensor 118 can be sensitive in the visible spectral range from 500 nm to 780 nm, most preferably in the range of 650 nm to 750 nm or 690 nm to 700 nm. Specifically, optical sensor 118 can be sensitive in the near-infrared region. Specifically, optical sensor 118 can be sensitive in a portion of the near-infrared region, in which the silicon photodiode is specifically suited for the range of 700 nm to 1000 nm. Optical sensor 118 can specifically be sensitive in the infrared spectral range, specifically in the range of 780 nm to 3.0 micrometers. For example, each optical sensor 118 can individually be or can include at least one element selected from the group consisting of: photodiode, photovoltaic cell, photoconductor, phototransistor, or any combination thereof. For example, optical sensor 118 can be or can include at least one element selected from the group consisting of: CCD sensor element, CMOS sensor element, photodiode, photovoltaic cell, photoconductor, phototransistor, or any combination thereof. Any other type of photosensitive element can be used. Photosensitive elements can typically be made entirely or partially of inorganic materials and / or entirely or partially of organic materials. Most commonly, one or more photodiodes, such as commercially available photodiodes, for example, inorganic semiconductor photodiodes, can be used.

[0234] Optical sensor 118 may include at least one sensor element comprising a pixel matrix. Therefore, by way of example, optical sensor 118 may be part of or constitute a pixelated optics device. For example, optical sensor 118 may be and / or may include at least one CCD and / or CMOS device. By way of example, optical sensor 118 may be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, with each pixel forming a photosensitive area. Sensor elements may be formed as a single device or a combination of multiple devices. Sensor elements include an optical sensor matrix. Sensor elements may include at least one CMOS sensor. The matrix may consist of individual pixels, such as independent optical sensors. Therefore, a matrix of inorganic photodiodes may be included. However, alternatively, commercially available matrices, such as one or more of CCD detectors (such as CCD detector chips) and / or CMOS detectors (such as CMOS detector chips), may be used. Therefore, generally, sensor elements may be and / or may include at least one CCD and / or CMOS device and / or optical sensors may form a sensor array or may be part of a sensor array, such as the matrix described above. Therefore, as an example, the sensor element may include a pixel array, such as a rectangular array with m rows and n columns, where m and n are independently positive integers. Preferably, more than one column and more than one row are given, i.e., n>1, m>1. Therefore, as an example, n can be 2 to 16 or higher, and m can be 2 to 16 or higher. Preferably, the ratio of the number of rows to the number of columns is close to 1. As an example, n and m can be chosen such that 0.3≤m / n≤3, for example by choosing m / n = 1:1, 4:3, 16:9, or similar. As an example, the array may be a square array with an equal number of rows and columns, such as by choosing m=2, n=2 or m=3, n=3, etc.

[0235] The matrix may include individual pixels, such as individual pixels of an independent optical sensor. Therefore, it may include a matrix of inorganic photodiodes. However, alternatively, commercially available matrices may be used, such as one or more of CCD detectors (such as CCD detector chips) and / or CMOS detectors (such as CMOS detector chips). Therefore, typically, the optical sensor may be and / or may include at least one CCD and / or CMOS device and / or detector. Optical sensors may form a sensor array or may be part of a sensor array, such as the matrix mentioned above.

[0236] The matrix can specifically be a rectangular matrix having at least one row, preferably multiple rows, and multiple columns. As an example, the rows and columns can be substantially vertically oriented. As used herein, the term "substantially vertical" refers to a vertical orientation with a tolerance of, for example, ±20° or less, preferably ±10° or less, more preferably ±5° or less. Similarly, the term "substantially parallel" refers to a parallel orientation with a tolerance of, for example, ±20° or less, preferably ±10° or less, more preferably ±5° or less. Thus, as an example, tolerances less than 20°, specifically less than 10°, or even less than 5° are acceptable. To provide a wide field of view, the matrix can specifically have at least 10 rows, preferably at least 500 rows, more preferably at least 1000 rows. Similarly, the matrix can have at least 10 columns, preferably at least 500 columns, more preferably at least 1000 columns. The matrix can include at least 50 optical sensors, preferably at least 100,000 optical sensors, more preferably at least 5,000,000 optical sensors. The matrix may include multiple pixels spanning millions of pixels. However, other embodiments are feasible. Therefore, in a configuration where axial rotational symmetry is desired, a circular or concentric arrangement (also referred to as pixels) of the matrix's optical sensors may be preferred.

[0237] Therefore, as an example, the sensor element may be part of or constitute a pixelated optical device. For example, the sensor element may be and / or may include at least one CCD and / or CMOS device. As an example, the sensor element may be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, with each pixel forming a photosensitive area. The sensor element may employ a rolling shutter or global shutter method to read out the matrix of the optical sensor.

[0238] Detector 110 may also include at least one transmission device 132. Detector 110 may also include one or more additional elements, such as one or more additional optical elements. Detector 110 may include at least one optical element selected from the group consisting of: transmission devices, such as at least one lens and / or at least one lens system, and at least one diffractive optical element. Transmission device 132 may be adapted to guide a light beam onto an optical sensor. Transmission device 132 may specifically include one or more of the following: at least one lens, for example, at least one lens selected from the group consisting of at least one focusing lens, at least one aspherical lens, at least one spherical lens, and at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one reflector; at least one beam splitting element, preferably at least one of a beam splitter cube or a beam splitter mirror; and at least one multi-lens system. Focal length constitutes a measure of the ability of transmission device 132 to converge an incident light beam. Therefore, transmission device 132 may include one or more imaging elements that can have a converging lens effect. By way of example, transmission device 132 may have one or more lenses, particularly one or more refractive lenses, and / or one or more convex mirrors. In this example, the focal length can be defined as the distance from the center of the thin refractive lens to the principal focal point of the thin lens. For converging thin refractive lenses, such as convex or biconvex thin lenses, the focal length can be considered positive and can provide a distance at which the collimated beam incident on the thin lens, which serves as a transmission device, can be focused into a single spot. Furthermore, the transmission device 132 may include at least one wavelength selection element, such as at least one filter. Additionally, the transmission device can be designed to apply a predetermined beam profile to electromagnetic radiation, for example, at a location within the sensor region, particularly at the sensor region. The above-described alternative embodiments of the transmission device 132 can, in principle, be implemented individually or in any desired combination.

[0239] The transmission device 132 may have an optical axis. Specifically, the detector 110 and the transmission device 132 share a common optical axis 122. The optical axis 122 may be a line of symmetry of the optical arrangement of the detector 110. The transmission device 132 may constitute a coordinate system 134, wherein the longitudinal coordinate is the coordinate along the optical axis 122, and wherein d is a spatial offset from the optical axis 122. The coordinate system 134 may be a polar coordinate system, wherein the optical axis of the transmission device 132 forms a z-axis, and the distance from the z-axis and the polar angle can be used as additional coordinates. Directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered longitudinal coordinates. Any direction perpendicular to the z-axis can be considered a transverse direction, and polar coordinates and / or polar angles can be considered transverse coordinates.

[0240] An optical sensor 118 is configured to determine at least one image 128 of region 114. Image 128 includes a plurality of reflection features 130 generated by the region in response to illumination by illumination features. Image 128 may be at least one two-dimensional image. Image 128 may be an RGB (red, green, blue) image.

[0241] The detector includes at least one evaluation device 136. Evaluation device 136 is configured to evaluate image 128. Evaluation device 136 may include at least one data processing device, and more preferably, by using at least one processor and / or at least one application-specific integrated circuit (ASIC). Thus, as an example, at least one evaluation device 136 may include at least one data processing device on which software code comprising a large number of computer commands is stored. Evaluation device 136 may provide one or more hardware elements for performing one or more specified operations, and / or may provide software to one or more processors to run thereon to perform one or more specified operations, including evaluating the image. Specifically, instructions for determining the beam profile and surface may be performed by at least one evaluation device 136. Thus, as an example, one or more instructions may be implemented in software and / or hardware. Thus, as an example, evaluation device 136 may include one or more programmable devices configured to perform the above evaluation, such as one or more computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs). However, additionally or alternatively, evaluation device 136 may also be embodied entirely or partially in hardware.

[0242] The evaluation device 136 and detector 110 can be fully or partially integrated into a single device. Therefore, typically, the evaluation device 136 can also form part of the detector 110. Alternatively, the evaluation device 136 and detector 110 can be implemented as separate devices, either fully or partially. Detector 110 may include further components.

[0243] Evaluation device 136 is configured to determine at least one depth map of region 114 by determining at least one depth information for each reflection feature 130. Evaluation device 136 may be configured to determine the depth information for each reflection feature 130 using one or more of the following techniques: photon ratio sounding, structured light, beam profilometry, time-of-flight, shape recovery from motion, focused sounding, triangulation, defocused sounding, and stereo sensor. Evaluation device 136 may be configured to take the depth information into account to segment object 116 from the background based on the depth map, particularly the hand region in image 128.

[0244] For example, evaluation device 136 can be configured to determine depth information for each reflection feature 130 using photon ratio depth sounding. Each reflection feature 130 may include at least one beam profile, also representing a reflection beam profile. The beam profile can be selected from a group consisting of: trapezoidal beam profiles; triangular beam profiles; conical beam profiles; and linear combinations of Gaussian beam profiles. Evaluation device 136 can be configured to determine depth information for each reflection feature 130 by analyzing their beam profiles.

[0245] Evaluation device 136 can be configured to determine at least one longitudinal coordinate z for each reflection feature 130 by analyzing their beam profiles. DPR For example, beam profile analysis may include at least one of histogram analysis steps, calculation of difference measures, application of neural networks, and application of machine learning algorithms. Evaluation device 136 may be configured to symmetrize and / or normalize and / or filter the beam profile, particularly to remove noise or asymmetry from recordings at large angles, recording edges, etc. Evaluation device 136 may filter the beam profile by removing high spatial frequencies, such as through spatial frequency analysis and / or median filtering. Summarization may be performed by averaging the intensity center of the spot and all intensities at the same distance from the center. Evaluation device 136 may be configured to normalize the beam profile to maximum intensity, particularly taking into account intensity differences attributable to recording distances. Evaluation device 136 may be configured to remove the effects of background light from the beam profile, for example, through imaging without illumination.

[0246] The evaluation device 136 can be configured to determine the longitudinal coordinate z of each reflection feature using photon ratio depth sounding. DPR Regarding photon ratio depth sounding (DPR) technology, see WO 2018 / 091649 A1, WO 2018 / 091638 A1 and WO 2018 / 091640 A1, the entire contents of which are incorporated herein by reference. The evaluation device 136 can be configured to perform at least one photon ratio depth sounding algorithm that calculates the distances of all reflective features 130 at zero order and higher.

[0247] The evaluation of image 128 includes identifying reflection features 130 of image 128. Evaluation device 136 can be configured to perform at least one image analysis and / or image processing to identify reflection features 130. Image analysis and / or image processing may use at least one feature detection algorithm. Image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image created by a sensor signal and at least one offset; inverting a sensor signal by inverting an image created by a sensor signal; forming a difference image between images created by a sensor signal at different times; background correction; decomposition into color channels; decomposition into hue, saturation, and luminance channels; frequency decomposition; singular value decomposition; applying a speckle detector; applying a corner detector; applying a Hessian filter determinant; applying a principal curvature-based region detector; applying a maximum stable extremum region detector; applying a generalized Hough transform; applying a ridge detector; applying an affine invariant feature detector; applying an affine-adapted interest point operator; applying a Harris affine region detector; applying a Hessian affine region detection. The algorithm employs the following methods: scale-invariant feature transform; scale-space extremum detector; local feature detector; accelerated robust feature algorithm; gradient location and orientation histogram algorithm; histogram of orientation gradient descriptors; Deriche edge detector; differential edge detector; spatiotemporal interest point detector; Moravec corner detector; Canny edge detector; Gaussian Laplacian filter; Gaussian difference filter; Sobel operator; Laplacian operator; Scharr operator; Prewitt operator; Roberts operator; Kirsch operator; high-pass filter; low-pass filter; Fourier transform; Radon transform; Hough transform; wavelet transform; thresholding; and creation of binary images. The region of interest can be determined manually by the user or automatically, such as by identifying features within the image 128 generated by the optical sensor 118.

[0248] For example, illumination source 112 can be configured to generate and / or project point clouds, such that multiple illumination regions are generated on optical sensor 118 (e.g., a CMOS detector). Additionally, interference may exist on optical sensor 118, such as interference due to speckle and / or incident light and / or multiple reflections. Evaluation device 136 can be adapted to determine at least one region of interest, for example, one or more pixels illuminated by the light beam for determining the longitudinal coordinates of an object. For example, evaluation device 136 can be adapted to perform filtering methods, such as speckle analysis and / or edge filtering and / or object recognition methods.

[0249] The evaluation device 136 can be configured to perform at least one image correction. Image correction may include at least one background subtraction. The evaluation device 136 may be adapted to remove the effects of background light from the beam profile, for example, by imaging without further illumination.

[0250] Evaluation device 136 is configured to locate object 116 by identifying reflective features 130 produced by irradiating biological tissue. Evaluation device 136 is configured to determine at least one reflective beam profile for each reflective feature 130. Evaluation device 136 is configured to identify a reflective feature 130 as produced by irradiating biological tissue if the reflective beam profile of the reflective feature meets at least one predetermined or predefined criterion. Otherwise, evaluation device 136 is configured to identify the reflective feature as background.

[0251] Detector 110 can be a device for detecting biological tissue (particularly human skin), especially optical detection. Identification generated by biological tissue can include determining and / or verifying whether a surface to be examined or tested is or includes biological tissue, particularly human skin, and / or distinguishing biological tissue, particularly human skin, from other tissues, particularly other surfaces, and / or distinguishing different types of biological tissue, such as distinguishing different types of human tissue, such as muscle, fat, organs, etc. For example, biological tissue can be or may include human tissue or a portion thereof, such as skin, hair, muscle, fat, organs, etc. For example, biological tissue can be or may include animal tissue or a portion thereof, such as skin, fur, muscle, fat, organs, etc. For example, biological tissue can be or may include plant tissue or a portion thereof. Detector 110 can be adapted to distinguish animal tissue or a portion thereof from one or more of inorganic tissue, metal surfaces, and plastic surfaces, such as those found on agricultural machinery or milking machines. Detector 110 can be adapted to distinguish plant tissue or a portion thereof from one or more of inorganic tissue, metal surfaces, and plastic surfaces, such as those found on agricultural machinery. Detector 110 can be adapted to distinguish food and / or beverages from plates and / or glasses. Detector 110 can be adapted to distinguish different types of food, such as fruit, meat, and fish. Detector 110 can be adapted to distinguish cosmetics and / or applied cosmetics from human skin. Detector 110 can be adapted to distinguish human skin from foam, paper, wood, displays, and screens. Detector 110 can be adapted to distinguish human skin from clothing. Detector 110 can be adapted to distinguish materials used in maintenance products and machine components, such as metal components. Detector 110 can be adapted to distinguish organic and inorganic materials. Detector 110 can be adapted to distinguish human biological tissue from the surface of artificial or inanimate objects. Detector 110 can be used specifically for non-therapeutic and non-diagnostic applications.

[0252] The predetermined or predefined criteria may be or may include at least one property and / or value suitable for distinguishing biological tissue, particularly human skin, from other materials. The predetermined or predefined criteria may be or may include at least one predetermined or predefined value and / or threshold and / or threshold range relating to material properties. If the reflected beam profile satisfies at least one predetermined or predefined criterion, the reflective feature 130 may be indicated as being generated through biological tissue. The indication may refer to any indication, such as an electronic signal and / or at least one visual or auditory indication.

[0253] Evaluation device 136 can be configured to determine at least one material property m of an object by evaluating the beam profile of its reflection characteristics. The material property can be or can include at least one arbitrary property of a material configured to characterize and / or identify and / or classify the material. For example, a material property can be a property selected from: roughness, depth of light penetration through the material, properties characterizing the material as biological or non-biological, reflectivity, specular reflectivity, diffuse reflectivity, surface properties, translucency measures, scattering, specifically backscattering behavior, etc. At least one material property can be a property selected from: scattering coefficient, translucency, transparency, deviation from Lambertian surface reflection, speckle, etc. Identifying at least one material property can include one or more of: determining the material property and assigning the material property to an object. Detector 110 can include at least one database comprising lists and / or tables of predefined and / or predetermined material properties, such as lookup lists or lookup tables. The list and / or table of material properties can be determined and / or generated by performing at least one test measurement using the detector according to the invention, for example, by performing material testing using a sample with known material properties. The list and / or table of material properties may be determined and / or generated at the manufacturer's site and / or by the user of detector 110. Material properties may be additionally assigned to a material classifier, such as one or more of the following: material name, material group, such as biological or non-biological material, translucent or non-translucent material, metallic or non-metallic, skin or non-skin, fur or non-fur, carpet or non-carpet, reflective or non-reflective, specular or non-specular, foam or non-foam, hair or non-hair, roughness group, etc. Detector 110 may include at least one database containing lists and / or tables of material properties and associated material names and / or material groups.

[0254] For example, to avoid being bound by this theory, human skin can have a reflective profile, also represented as a backscattering profile, comprising a portion produced by backscattering of light from the surface, represented as surface reflection; and a portion produced by very diffuse reflection of light penetrating the skin, represented as the diffuse portion of backscattering. For a description of the reflective profile of human skin, see “Lasertechnik in der Medizin: Grundlagen, Systeme, Anwendungen”, “Wirkung von Laserstrahlung auf Gewebe”, 1991, pp. 10 171-266, Jürgen Eichler, Theo Seiler, Springer Verlag, ISBN 0939-0979. The surface reflectance of skin increases with increasing wavelength towards the near-infrared. Furthermore, the penetration depth increases with increasing wavelength from visible light to the near-infrared. The diffuse portion of backscattering increases with the penetration depth of light. By analyzing the backscattering profile, these properties can be used to distinguish skin from other materials.

[0255] Specifically, the evaluation device 136 can be configured to compare the reflected beam profile with at least one predetermined and / or pre-recorded and / or predefined beam profile. The predetermined and / or pre-recorded and / or predefined beam profile can be stored in a table or lookup table and can be determined empirically, for example, and, as an example, can be stored in at least one data storage device of the detector. For example, the predetermined and / or pre-recorded and / or predefined beam profile can be determined during the initial startup of the mobile device including the detector. For example, the predetermined and / or pre-recorded and / or predefined beam profile can be stored in at least one data storage device of the mobile device, for example, via software, specifically via an application downloaded from an app store, etc. If the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile are identical, the reflection feature 130 can be identified as being generated by biological tissue. The comparison may include overlapping the reflected beam profile and the predetermined or predefined beam profile such that their intensity centers match. The comparison may include determining the deviation between the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile, such as the sum of squares of point-to-point distances. The evaluation device 136 may be adapted to compare a determined deviation with at least one threshold, wherein if the determined deviation is less than and / or equal to the threshold, the surface is indicated as biological tissue and / or the detection of biological tissue is confirmed. The threshold may be stored in a table or lookup table and may be determined empirically, for example, and, as an example, may be stored in at least one data storage device of the detector 110.

[0256] Additionally or alternatively, in order to identify whether the reflectance feature 130 is generated by biological tissue, the evaluation device 136 may be configured to apply at least one image filter to the image 128 of region 114. The evaluation device 136 may be configured to determine at least one material feature by applying at least one material-related image filter Ф2 to the image. The material characteristics may be, or may include, at least one piece of information about at least one material property of the surface of region 114 where the reflective characteristics 130 have been generated.

[0257] Material-related image filters can be at least one filter selected from the following: brightness filter; speckle shape filter; square norm gradient; standard deviation; smoothness filter, such as a Gaussian filter or median filter; contrast filter based on gray-level occurrence; energy filter based on gray-level occurrence; homogeneity filter based on gray-level occurrence; dissimilarity filter based on gray-level occurrence; law-based energy filter; threshold area filter; or linear combinations thereof; or further material-related image filters. 2other The further material-related image filter Ф 2other via |ρФ 2other，Фm |≥0.40 is related to one or more of the following, or linear combinations thereof: brightness filter, speckle shape filter, square norm gradient, standard deviation, smoothness filter, energy filter based on gray-level occurrence, homogeneous filter based on gray-level occurrence, dissimilar filter based on gray-level occurrence, law-based energy filter, or threshold region filter, where Ф m It is one of the following: a brightness filter, a speckle shape filter, a square norm gradient filter, a standard deviation filter, a smoothness filter, an energy filter based on gray-level occurrence, a homogeneity filter based on gray-level occurrence, a dissimilarity filter based on gray-level occurrence, a law-based energy filter, or a threshold region filter, or a linear combination thereof. This further describes the material-related image filter Ф. 2other It can be done through |ρ Ф2other,Фm |≥0.60, preferably through |ρ Ф2other,Фm |≥0.80 is correlated with one or more of the material-related image filters Фm.

[0258] The evaluation device 136 can be configured to use material characteristics The material properties of the surface with the generated reflective feature 130 are determined by at least one predetermined relationship between the material properties and the material properties of the surface. This predetermined relationship may be one or more of empirical, semi-empirical, and analytically derived relationships. The evaluation device 136 may include at least one data storage device, such as a lookup list or lookup table, for storing the predetermined relationship.

[0259] Evaluation device 136 is configured to identify reflective feature 130 as being generated by irradiation of biological tissue if the corresponding material properties of reflective feature 130 meet at least one predetermined or predefined criterion. If the material properties indicate "biological tissue," reflective feature 130 can be identified as being generated by biological tissue. If the material properties are below or equal to at least one threshold or range, reflective feature 130 can be identified as being generated by biological tissue, wherein if a determined deviation is below and / or equal to the threshold, the reflective feature is identified as being generated by biological tissue and / or the detection of biological tissue is confirmed. At least one threshold and / or range can be stored in a table or lookup table and can be determined empirically, for example, and, as an example, can be stored in at least one data storage device of the detector. Otherwise, evaluation device 136 is configured to identify reflective features as background. Therefore, evaluation device 136 can be configured to assign depth information and material properties, such as whether skin is present or not, to each projection spot.

[0260] After determining the vertical coordinate z, subsequent evaluation can be performed. To determine material properties so that information about the longitudinal coordinate z can be considered for evaluation.

[0261] Evaluation device 136 is configured to segment image 128 of region 114 using at least one segmentation algorithm. A segment may include a set of pixels. Segmentation may include the process of dividing image 128 into multiple segments. Segmentation may include assigning at least one label to each pixel of image 128 such that pixels with the same label share at least one feature. Labels may be assigned under predefined targets. Segmentation may be binary segmentation. Binary segmentation may include labeling pixels of image 128 as "skin pixels" 138 and "background pixels" 140. All non-skin pixels may be considered background. Skin pixels may be considered foreground seeds, also represented as seed points, and used as input to image-based segmentation algorithms. For more information on image segmentation, see https: / / en.wikipedia.org / wiki / Image_segmentation. Segmentation algorithms may include at least one region growing image segmentation method.

[0262] The segmentation algorithm can be configured to fuse and / or consider prior knowledge. Specifically, the segmentation algorithm can be configured to fuse and / or consider material information determined from bundle contour analysis. This allows for obtaining or filtering positional data only for the movement of visible skin. Conversely, in conventional settings, identification and separation, such as what is a finger / arm and what is not, must be performed first. In addition to using material information, the evaluation device 136 can be configured to consider depth information to segment objects from the background based on depth maps, particularly hand regions in the image.

[0263] Segmentation algorithms can be based on energy or cost functions, such as one or more of graph cutting, level sets, fast walk, and Markov random field methods. Image segmentation can be driven by color homogeneity and edge indices, where seed points constitute edge and color homogeneity criteria. Reflectance features identified as generated by irradiating biological tissue 130 are used as seed points, and reflection features identified as background are used as background seed points for the segmentation algorithm. These seed points can constitute edge and color homogeneity criteria, thus providing appropriate initialization for target segmentation: "What color and / or reflection and / or appearance does the skin in the image have?".

[0264] For example, graph slicing segmentation can be used. Graph slicing segmentation can be configured to incorporate prior knowledge because it also provides real-time capabilities when combined with pre-clustering algorithms such as superpixels, see R.Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,” IEEE TPAMI, vol.34, no.11, 2012. For example, modified lazy-capture graph slicing can be used. Regarding the modified lazy snapping graph cutting, refer to "Lazy Snapping" by Li, Yin and Sun, Jian and Tang, Chi-Keung and Shum, and Heung-Yeung, ACM Trans. Graph, vol. 23, no. 3, 2004. The Gibbs energy term can include additional terms to incorporate depth information used in the graph cutting segmentation routine. Instead of "lazy snapping" as proposed by Li, Yin and Sun, Jian and Tang, Chi-Keung and Shum, and Heung-Yeung, the watershed-based pre-cutting proposed in ACM Trans. can use superpixel clustering to ensure real-time capabilities.

[0265] Image 128 before segmentation Figure 2AAs shown. Each projection point is assigned depth and material properties (skin yes / no). To obtain... Figure 2B The binary segmentation shown treats all non-skin points as background and skin points as foreground seeds, serving as input to an image-based segmentation routine. In this example, a modified lazy-capture graph cut is used. The Gibbs energy term includes additional terms to incorporate depth information used in the graph cut segmentation routine. Furthermore, superpixel clustering is used to ensure real-time performance.

[0266] Evaluation device 136 is configured to consider the segmented image and depth map to determine the spatial position and / or orientation of object 116. Evaluation device 136 may be configured to determine at least one hand pose or gesture based on the spatial position and / or orientation of object 116. Evaluation device 136 may be configured to recognize poses from the segmented image, particularly from portions of the segmented image labeled as object 116. Evaluation device 136 may be configured to identify image coordinates of the palm and fingers in the segmented image. Palm and finger recognition can be obtained from the segmentation. Evaluation device 136 may be configured to extract features, such as color and / or brightness and / or gradient values, from the portions of the segmented image labeled as objects. For example, palms and fingertips can be detected using standard OpenCV routines.

[0267] The evaluation device 136 can be configured to determine at least one three-dimensional finger vector 141 by taking into account the image coordinates of the palm 142 (such as the center point 144 of the palm) and the fingertips 146 as well as the depth map. Figure 2C An embodiment of a 3D finger vector 140 is illustrated. The 3D finger vector 140 can specify the position and orientation of a finger in space. An evaluation device 136 can be configured to determine the 3D finger vector 140 from an image 128 and a depth map. The 3D finger vector 140 can provide a basis for hand pose estimation and scene interpretation. For gesture recognition, the evaluation device 136 can be configured to use at least one classifier, such as a Hidden Markov Model (HMM), Support Vector Machine (SVM), Conditional Random Field (CRF), etc. The classifier can be configured to distinguish gestures.

[0268] Using the beam profile analysis sensor fusion concept according to the present invention has the advantages of providing robust detection results, not requiring complex model assumptions, a single sensor concept, easy integration, and real-time capability.

[0269] Reference number list

[0270] 110 detector

[0271] 112 Irradiation Source

[0272] District 114

[0273] 116 objects

[0274] 118 Optical Sensors

[0275] 120 housing

[0276] 122 optical axes

[0277] 124 beams

[0278] 126 Photosensitive area

[0279] 128 images

[0280] 130 Reflection characteristics

[0281] 132 Conveying device

[0282] 134 coordinate system

[0283] 136 Evaluation Device

[0284] 138 skin pixels

[0285] 140 background pixels

[0286] 141 Three-dimensional finger vectors

[0287] 142 palms

[0288] 144 Center Point

[0289] 146 Fingertips

Claims

1. A detector (110) for gesture detection, comprising - at least one illumination source (112) configured for projecting at least one illumination pattern comprising a plurality of illumination features onto at least one area (114) comprising at least one object (116), wherein, - an object (116) comprising at least partially at least one human hand; - at least one optical sensor (118) having at least one light-sensitive area (126), wherein the optical sensor (118) is configured for determining at least one image (128) of the area, wherein the image (128) comprises a plurality of reflection features (130) resulting from the area (114) in response to an illumination by the illumination feature; - at least one evaluation device (136), wherein the evaluation device (136) is configured for determining at least one depth map of the area by determining at least one depth information for each reflection feature (130), wherein the evaluation device (136) is configured for finding the object (116) by identifying the reflection features (130) resulting from the illumination of biological tissue, wherein the evaluation device (136) is configured for determining at least one reflection beam profile for each reflection feature (130), wherein the evaluation device (136) is configured for identifying a reflection feature (130) as resulting from the illumination of biological tissue in case the reflection beam profile of the reflection feature (130) fulfills at least one predetermined or predefined criterion, wherein the predetermined or predefined criterion is or comprises at least one predetermined or predefined value and / or threshold and / or threshold range referring to a material property, wherein the evaluation device (136) is configured for determining at least one material property m of the object (116) by evaluating the reflection beam profile of the reflection feature (130), wherein the evaluation device (136) is configured for comparing the reflection beam profile of each reflection feature (130) to at least one predetermined and / or pre-recorded and / or predefined beam profile, wherein the evaluation device (136) is configured for otherwise identifying the reflection feature (130) as background, wherein the evaluation device (136) is configured for segmenting the image (128) of the area by using at least one segmentation algorithm, wherein the segmentation algorithm is configured for segmenting the image of the area based at least partially on the at least one material property and the at least one depth information for each reflection feature, wherein the reflection features (130) identified as resulting from the illumination of biological tissue are used as seed points and the reflection features (130) identified as background are used as background seed points for the segmentation algorithm, wherein the evaluation device (136) is configured for determining a position and / or orientation of the object (116) in space considering the segmented image and the depth map.

2. The detector (110) of claim 1, wherein The evaluation device (136) is configured for identifying image coordinates of a palm and fingers in the segmented image, wherein the evaluation device (136) is configured for determining at least one three-dimensional finger vector (141) considering the image coordinates of a palm and fingers and the depth map.

3. The detector (110) according to claim 1 or 2, wherein The evaluation device (136) is configured to determine at least one hand pose from the position and / or the orientation of the object (116) in space.

4. The detector (110) according to claim 1 or 2, wherein The evaluation device (136) is configured to determine at least one hand gesture from the position and / or the orientation of the object (116) in space.

5. The detector (110) according to claim 1 or 2, wherein The segmentation algorithm is based on an energy or cost function.

6. The detector (110) of claim 5, wherein The energy or cost function comprises one or more of graph cuts, level sets, fast marching and Markov random field methods.

7. The detector (110) according to claim 1 or 2, wherein The segmentation of the image (128) is driven by color homogeneity and edge indicators, wherein the seed points constitute edge and color homogeneity criteria.

8. The detector (110) of claim 7, wherein The comparison comprises superimposing the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile such that their intensity centers match, wherein the comparison comprises determining a deviation between the reflected beam profile and the predetermined and / or pre-recorded and / or predefined beam profile, wherein the evaluation device (136) is configured to compare the determined deviation to at least one threshold value, wherein in case the determined deviation is below and / or equal to the threshold value, the reflected feature (130) is indicated as biological tissue.

9. The detector (110) according to claim 1 or 2, wherein The evaluation device (136) is configured to determine the depth information of each of the reflected features by one or more of the following techniques: optical ratio depth from photon, structured light, beam profile analysis, time of flight, shape from motion, focus depth, triangulation, defocus depth, stereo sensor.

10. The detector (110) according to claim 1 or 2, wherein The evaluation device (136) is configured to determine the depth information of each of the reflected features (130) by using the optical ratio depth from photon technique, wherein the evaluation device is configured to determine at least one first region and at least one second region of a beam profile of at least one of the reflected features, wherein the evaluation device (136) is configured to integrate the first region and the second region, wherein the evaluation device (136) is configured to derive the quotient Q by one or more of the following: a division operation on the integrated first region and the integrated second region, a division operation on a multiple of the integrated first region and the integrated second region, a division operation on a linear combination of the integrated first region and the integrated second region.

11. The detector (110) according to claim 10, wherein the first region of the reflected beam profile comprises substantially edge information of the reflected beam profile and the second region of the reflected beam profile comprises substantially center information of the reflected beam profile, and / or wherein, The first region of the reflected beam profile comprises information substantially about a left side portion of the reflected beam profile and the second region of the reflected beam profile comprises information substantially about a right side portion of the reflected beam profile.

12. The detector (110) of claim 10, wherein, The evaluation device (136) is configured to derive the quotient Q by ， where x and y are lateral coordinates, A1 and A2 are the first region and the second region of the reflected beam profile, respectively, and E(x, y) represents the reflected beam profile.

13. The detector (110) according to claim 1 or 2, wherein The illumination source (112) is configured to generate the at least one illumination pattern in the near infrared region (NIR).

14. The detector (110) according to claim 1 or 2, wherein The optical sensor (118) comprises at least one CMOS sensor.

15. A method for gesture detection, wherein, The method uses at least one detector (110) according to any one of claims 1-14, wherein the method comprises the following steps: a) projecting at least one illumination pattern comprising a plurality of illumination features onto at least one region (114) comprising at least one object (116), wherein the object (116) at least partially comprises at least one human hand; b) determining at least one image (128) of the region (114) using at least one optical sensor (118) having at least one light-sensitive area (126), wherein the image (128) comprises a plurality of reflection features (130) resulting from the region (114) in response to illumination by illumination features; c) determining at least one depth map of the region (114) by determining at least one depth information of each of the reflection features (130) using at least one evaluation device (136); d) finding the object (116) by using the evaluation device (136) and by identifying the reflection features (130) resulting from illuminating biological tissue, wherein at least one reflection beam profile of each of the reflection features (130) is determined, wherein the reflection features (130) are identified as resulting from illuminating biological tissue in case the reflection beam profile of a reflection feature meets at least one predetermined or predefined criterion, wherein the predetermined or predefined criterion is or comprises at least one predetermined or predefined value and / or threshold and / or threshold range referring to a material property, wherein the evaluation device (136) is configured for determining at least one material property m of the object (116) by evaluating the reflection beam profile of the reflection features (130), wherein the evaluation device (136) is configured for comparing the reflection beam profile of each of the reflection features (130) to at least one predetermined and / or pre-recorded and / or predefined beam profile, wherein the reflection features (130) are otherwise identified as background; e) segmenting the image (128) of the region (114) by using the evaluation device (136) and by using at least one segmentation algorithm, wherein the segmentation algorithm is configured for segmenting the image of the region at least partially based on the at least one material property and the at least one depth information for each reflection feature, wherein the reflection features (130) identified as resulting from illuminating biological tissue are used as seed points and the reflection features (130) identified as background are used as background seed points for the segmentation algorithm; f) determining a position and / or orientation of the object (116) in space by using the evaluation device (136) considering the segmented image and the depth map.

16. Use of a detector (110) according to any one of claims 1 to 14 referring to a detector for a use selected from the group consisting of: driver monitoring; in-vehicle monitoring; gesture tracking; safety applications; human-machine interface applications; agricultural applications; crop protection applications; medical applications; maintenance applications; cosmetic applications.

Images