Detector for determining the position of at least one object

Abstract

A detector (110) for determining the position of at least one object (112) is proposed. The detector (110) comprises: - at least one sensor element (130) having a matrix (132) of optical sensors (134), each optical sensor (134) having a photosensitive region (136), wherein the sensor element (130) is configured to determine at least one reflected image (142); - at least one evaluation device (146), wherein the evaluation device (146) is configured to select at least one reflection feature of the reflected image (142) at at least one first image location (148) in the reflected image (142), wherein the evaluation device (146) is configured to optimize at least one blur function f a. To determine at least one ordinate z of the selected reflection feature, wherein the evaluation device (146) is configured to determine at least one reference feature in at least one reference image (168) at at least one second image position (154) in the reference image (168) corresponding to at least one reflection feature, wherein the reference image (168) and the reflection image (142) are determined in two different spatial configurations, wherein the difference in the spatial configurations lies in the relative spatial constellation, wherein the evaluation device (146) is configured to determine the relative spatial constellation based on the ordinate z, the first image position (148) and the second image position (154).

Description

Technical Field

[0001] This invention relates to a detector for determining the position of at least one object, a method for determining a relative spatial constellation using at least one detector for determining the position of at least one object, and a method for calibrating at least one detector. The invention also relates to various uses of human-machine interfaces, entertainment devices, tracking systems, cameras, scanning systems, and detector devices for exchanging at least one type of information between a user and a machine. The devices, methods, and uses according to the invention are specifically applicable to various fields, such as daily life, gaming, transportation technology, manufacturing technology, security technology, photography (e.g., digital photography or videography for artistic, documentary, or technical purposes), medical technology, or science. Furthermore, the invention can be specifically used for scanning one or more objects and / or for scanning scenes, such as for generating depth profiles of objects or scenes in fields such as architecture, surveying, archaeology, art, medicine, engineering, or manufacturing. However, other applications are also possible. Background Technology

[0002] In situations where environmental reflections cause multiple reflections, optical 3D sensing methods can often rely on biasing light sources or reflecting objects to arrive at unreliable results. Furthermore, imaging-enabled 3D sensing methods (such as triangulation using structured light or stereo cameras) typically require high computational power to solve these problems. This necessary computational power can lead to high costs due to processors or field-programmable gate arrays (FPGAs), difficult heat dissipation considering ventilation requirements or waterproof housings, power consumption (especially in mobile devices), and additional measurement uncertainties. The high power requirements may prevent the implementation of real-time applications, high frame rates, or even standard video frame rates of 25 frames per second.

[0003] Numerous optical devices are known from existing technologies that utilize triangulation imaging methods. For example, structured light methods or stereo methods are known. Examples include passive stereo methods using two cameras with fixed relative orientations, and active stereo techniques using an additional light projector. Another example is the structured light method, which uses a light projector and a camera with fixed relative orientations. To determine the depth image via triangulation, the correspondence problem must first be solved. Therefore, in passive stereo camera techniques, a sufficient number of corresponding feature points must be identified in the views of the two cameras. In structured light methods, the correspondence between a pre-stored pseudo-random light pattern and the projected pattern must be determined. Robust solutions to these correspondence problems require computational imaging algorithms, such as those that approximate quadratic scaling of the number of points in the projected point pattern. In structured light methods, for example, a stereo system comprising two detectors with a fixed relative distance is used, with the light source projecting a pattern such as a dotted, pseudo-random, random, non-periodic, or irregular dot pattern. Each detector generates an image of the reflected pattern, and the image analysis task is to identify corresponding features in the two images. Due to their fixed relative positions, corresponding feature points selected in one of two images lie on the epipolar line in the other image. However, solving the so-called correspondence problem can be difficult. In stereo and triangulation systems, all feature points must have a reasonable correspondence with each other along the epipolar line. Correspondence decisions cannot be made one after another. If one correspondence is incorrect, it will affect other feature points, such as causing them to become invisible. This often leads to nonlinear evaluation algorithms such as quadratic scaling.

[0004] For example, US 2008 / 0240502 A1 and US 2010 / 0118123 A1 describe an apparatus for mapping an object, including an illumination component comprising a single transparent body containing a fixed dot pattern. A light source illuminates the single transparent body by radiating light to project the pattern onto the object. An image capture component captures an image of the pattern projected onto the object using the single transparent body. A processor processes the image captured by the image capture component to reconstruct a three-dimensional map of the object.

[0005] Furthermore, 3D sensing methods are known to determine distances using so-called pose estimation, or structure from motion or shape from motion, see, for example, Ramalingam et al., “Pose Estimation using Both Points and Lines for Geo-Localization,” published in Robotics and Automation (ICRA), 2011 IEEE International Conference, Publisher: IEEE ISBN: 978-1-61284-385-8. The term “structure from motion” will be used as a synonym for both. In pose estimation algorithms, camera images are recorded, and the camera’s pose (such as viewing direction, distance from the object, and camera speed) is estimated. Similarly, in structure from motion algorithms, the 3D structure of the object is recorded by estimating the camera’s pose and the position of image features relative to each other. Unbound by this theory, both algorithms are based on the observation that in an image of a moving camera object, objects closer to the camera move faster than those farther away. The distance to the camera can be subtracted by tracking feature points from image to image.

[0006] Furthermore, WO 2017 / 012986 A1 describes a detector for determining the position of at least one object. The detector includes: - at least one optical sensor configured to detect at least one spot of light generated by at least one light beam propagating from the object toward the detector, the optical sensor having at least one pixel matrix, each pixel adapted to generate at least one pixel signal si,j in response to illumination by the light beam; - at least one nonlinearization device configured to convert the pixel signals si,j of all pixels i,j or at least a group of pixels into nonlinear pixel signals s'i,j, each of the nonlinear pixel signals s'i,j being a nonlinear function of the illumination power pij of the corresponding pixel; - at least one summing device configured to sum the nonlinear pixel signals s'ij of all pixels i,j or at least a group of pixels and generate at least one nonlinear sum signal S' = ∑ij s'ij; and - at least one evaluation device configured to determine at least one ordinate z of the object by evaluating the nonlinear sum signal S'.

[0007] Despite the advantages suggested by the aforementioned devices and detectors, some technical challenges remain.

[0008] Evaluation algorithms used in known 3D sensing methods require high computational power, which is a significant cost driver. Furthermore, the computational demands limit the use of such 3D sensor methods in outdoor and mobile applications due to the energy consumption and heat generated by the computational resources.

[0009] Furthermore, triangulation systems heavily rely on a fixed, unchanging mechanical connection between the illumination source and the sensor (e.g., in detectors using structured light or laser triangulation) or between two sensors (e.g., in stereo systems). Accurately determining the distance between the illumination source and the sensor, or between the two sensors, is fundamental to distance measurement. Variations in distance can introduce errors, such as offsets, linear, second-order, or higher-order errors in distance measurement. In known 3D sensing methods, achieving stable distance measurement requires significant mechanical effort and cost. Stability issues are particularly relevant to aging, temperature variations, and mechanical stress.

[0010] Furthermore, pose estimation algorithms cannot determine whether the camera is approaching and moving slowly or moving away and moving quickly. Therefore, these algorithms lack a scaling factor that allows for absolute measurement. In known 3D sensing methods using moving cameras, the camera's velocity is typically measured by the camera's inertial measurement unit (IMU), however, with limited accuracy. Summary of the Invention

[0011] The problem solved by this invention

[0012] Therefore, one object of the present invention is to provide an apparatus and method that addresses the aforementioned technical challenges posed by known devices and methods. Specifically, the object of the present invention is to provide an apparatus and method capable of reliably determining the position of an object in space, preferably with low technical effort and low requirements in terms of technical resources and cost. Summary of the Invention

[0014] The present invention, having the features of the independent claims, solves this problem. Advantageous developments of the invention, which can be implemented individually or in combination, are presented in the dependent claims and / or the following description and detailed embodiments.

[0015] As used below, the terms “have,” “contain,” or “include,” or any grammatical variation thereof, are used in a non-exclusive manner. Thus, these terms can refer to either the case where no other features exist in the entity described in the context besides those introduced by these terms, or the case where one or more other features exist besides those introduced by these terms. For example, the expressions “A has B,” “A contains B,” and “A includes B” can refer to the case where no other element exists in A besides B (i.e., where A consists only and uniquely of B), and to the case where entity A contains one or more other elements besides B, such as element C, elements C and D, or even other elements.

[0016] Furthermore, it should be noted that the terms "at least one," "one or more," or similar expressions indicating that a feature or element may exist once or more are typically used only once when introducing the corresponding feature or element. Below, in most cases, when referring to a corresponding feature or element, the expressions "at least one" or "one or more" will not be repeated, even though the corresponding feature or element may exist only once or more.

[0017] Furthermore, as used below, the terms “preferably,” “more preferably,” “particularly,” “more particularly,” “specifically,” “more specifically,” or similar terms are used in combination with optional features without limiting the possibility of substitution. Therefore, features introduced by these terms are optional features and are not intended to limit the scope of the claims in any way. As those skilled in the art will recognize, the invention can be practiced by using alternative features. Similarly, features introduced by “in embodiments of the invention” or similar expressions are intended to be optional features and do not limit alternative embodiments of the invention, the scope of the invention, or the possibility of combining features introduced in this way with other optional or non-optional features of the invention.

[0018] In a first aspect of the invention, a detector for determining the position of at least one object is disclosed. As used herein, the term "object" refers to a point or region emitting at least one light beam. The light beam may originate from the object, such as through the object and / or at least one illumination source integrated with or attached to the object emitting the light beam, or it may originate from different illumination sources, such as illumination sources that directly or indirectly illuminate the object, wherein the light beam is reflected or scattered by the object. As used herein, the term "position" refers to at least one piece of information regarding the location and / or orientation of at least a portion of the object in space. Thus, the at least one piece of information may imply at least one distance between at least one point of the object and the at least one detector. As will be further detailed below, the distance may be a ordinate, or a ordinate that may help determine the location of a point of the object. Additionally or alternatively, one or more other pieces of information regarding the location and / or orientation of at least a portion of the object may be determined. As an example, at least one abscissa of at least a portion of the object may also be determined. Thus, the position of the object may imply at least one ordinate of the object and / or at least a portion of the object. Additionally or alternatively, the position of the object may imply at least one abscissa of the object and / or at least a portion of the object. Alternatively or concurrently, the position of an object may imply at least one orientation information of the object, indicating its orientation in space.

[0019] The detectors include:

[0020] - At least one sensor element having a matrix of optical sensors, each optical sensor having a photosensitive region, wherein the sensor element is configured to determine at least one reflected image;

[0021] - At least one evaluation device, wherein the evaluation device is configured to select at least one reflection feature of the reflection image at at least one first image location in the reflection image, wherein the evaluation device is configured to optimize at least one blur function f a To determine at least one ordinate z of a selected reflection feature, wherein the evaluation device is configured to determine at least one reference feature in at least one reference image at at least one second image position corresponding to at least one reflection feature in the reference image, wherein the reference image and the reflection image are determined in two different spatial configurations, wherein the difference between the spatial configurations lies in the relative spatial constellation, wherein the evaluation device is configured to determine the relative spatial constellation based on the ordinate z, the first image position and the second image position.

[0022] As used herein, the term "sensor element" generally refers to a device or combination of devices configured to sense at least one parameter. In the present context, the parameter may specifically be an optical parameter, and the sensor element may specifically be an optical sensor element. A sensor element may be formed as a single device or as a combination of multiple devices. As used herein, "optical sensor" generally refers to a photosensitizing device for detecting a light beam (such as for detecting illumination and / or a light spot generated by at least one light beam).

[0023] As used further herein, the term "matrix" generally refers to an arrangement of multiple elements in a predetermined geometric order. As further detailed below, the matrix can specifically be or may include a rectangular matrix having one or more rows and one or more columns. The rows and columns can specifically be arranged in a rectangular manner. However, it should be noted that other arrangements are also possible, such as triangular, circular, hexagonal, or other non-rectangular arrangements. As an example, a circular arrangement is also possible, where the elements are arranged around a central point in concentric circles or ellipses. For example, the matrix can be a single row of pixels. Other arrangements are possible. The optical sensors of the matrix can specifically be equal in size, sensitivity, and one or more other optical, electrical, and mechanical properties. Specifically, the photosensitive areas of all the optical sensors of the matrix can be located in a common plane, which preferably faces the object, such that a beam of light propagating from the object to the detector can generate a spot on the common plane.

[0024] As used further herein, "photosensitive region" generally refers to an area of ​​an optical sensor that can be illuminated from the outside by at least one beam of light, which generates at least one sensor signal in response to illumination. The photosensitive region may specifically be located on the surface of the respective optical sensor. However, other embodiments are feasible. As used herein, the term "each optical sensor having at least one photosensitive region" refers to a configuration with multiple individual optical sensors, each having one photosensitive region, and also refers to a configuration with a combined optical sensor having multiple photosensitive regions. Thus, the term "optical sensor" further refers to a photosensitive device configured to generate one output signal, while here, a photosensitive device (e.g., at least one CCD and / or CMOS device) configured to generate two or more output signals is referred to as two or more optical sensors. As will be further detailed below, each optical sensor may be embodied such that exactly one photosensitive region is present in the respective optical sensor, such that by providing exactly one illuminated photosensitive region, a uniform sensor signal is created precisely for the entire optical sensor in response to illumination of that photosensitive region. Thus, each optical sensor may be a single-region optical sensor. However, the use of single-region optical sensors makes detector setup particularly simple and efficient. Therefore, as an example, commercially available optical sensors, such as commercially available silicon photodiodes, can be used in the setup, each silicon photodiode having exactly one photosensitive area. However, other embodiments are feasible. Thus, as an example, optical devices comprising two, three, four, or more photosensitive areas can be used, which are considered, in the context of this invention, as two, three, four, or more optical sensors. As summarized above, sensor elements comprise a matrix of optical sensors. Thus, as an example, an optical sensor can be part of or constitute a pixelated optical device. As an example, an optical sensor can be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, each pixel forming a photosensitive area.

[0025] The optical sensor may specifically be or may include a photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, and most preferably a silicon photodetector. Specifically, the optical sensor may be sensitive in the infrared spectral range. All optical sensors in the matrix, or at least a group of optical sensors in the matrix, may specifically be identical. Groups of identical optical sensors in the matrix may be specifically configured for different spectral ranges, or all optical sensors may be identical in terms of spectral sensitivity. Furthermore, the optical sensors may be identical in size and / or with respect to their electronic or photoelectric properties.

[0026] Specifically, the optical sensor can be, or can include, an inorganic photodiode sensitive in the infrared spectral range (preferably in the range of 780 nm to 3.0 micrometers). Specifically, the optical sensor is sensitive in a portion of the near-infrared region, wherein silicon photodiodes are specifically suitable for the range of 700 nm to 1000 nm. The infrared optical sensor that can be used can be a commercially available infrared optical sensor, such as the infrared optical sensor commercially available under the trademark Hertz-stueck™ from trinamiX GmbH, D-67056, Ludwigshafen am Rhein, Germany. Thus, by way of example, the optical sensor can include at least one intrinsic photovoltaic type optical sensor, more preferably, at least one semiconductor photodiode selected from the group consisting of: Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, and HgCdTe photodiodes. Alternatively or alternatively, the optical sensor may include at least one extrinsic photovoltaic type optical sensor, more preferably, at least one semiconductor photodiode selected from the group consisting of: Ge:Au photodiode, Ge:Hg photodiode, Ge:Cu photodiode, Ge:Zn photodiode, Si:Ga photodiode, and Si:As photodiode. Alternatively or alternatively, the optical sensor may include at least one calorimeter, preferably a calorimeter selected from the group consisting of V0 calorimeters and amorphous Si calorimeters.

[0027] The matrix may include individual optical sensors. Therefore, it may include a matrix of inorganic photodiodes. However, alternatively, one or more commercially available matrices may be used, such as CCD detectors (such as CCD detector chips) and / or CMOS detectors (such as CMOS detector chips).

[0028] Therefore, the optical sensors of the detector can typically form a sensor array or be part of a sensor array, such as the matrix described above. Thus, as an example, the detector can include an optical sensor array such as a rectangular array with m rows and n columns, where m and n are independently positive integers. Preferably, more than one column and more than one row are given, i.e., n > 1, m > 1. Thus, as an example, n can be 2 to 16 or higher, and m can be 2 to 16 or higher. Preferably, the ratio of the number of rows to the number of columns is close to 1. As an example, n and m can be chosen such that 0.3 ≤ m / n ≤ 3, such as by choosing m / n = 1:1, 4:3, 16:9, or similar values. As an example, the array can be a square array with an equal number of rows and columns, such as by choosing m = 2, n = 2, or m = 3, n = 3, etc.

[0029] Specifically, the matrix can be a rectangular matrix having at least one row (preferably multiple rows) and multiple columns. As an example, the rows and columns can be substantially vertically oriented. As used herein, the term "substantially vertical" refers to the condition of vertical orientation, such as a tolerance of ±20° or less, preferably ±10° or less, more preferably ±5° or less. Therefore, as an example, tolerances less than 20°, particularly less than 10°, or even less than 5° are acceptable. To provide a wide field of view, the matrix can particularly have at least 10 rows, preferably at least 50 rows, more preferably at least 100 rows. Similarly, the matrix can have at least 10 columns, preferably at least 50 columns, more preferably at least 100 columns. The matrix can include at least 50 optical sensors, preferably at least 100 optical sensors, more preferably at least 500 optical sensors. The matrix can include multiple pixels in the millions of pixels range. However, other embodiments are feasible. Therefore, in a configuration where axial rotational symmetry is desired, a circular or concentric arrangement of the matrix's optical sensors (also referred to as pixels) may be preferred.

[0030] Preferably, the sensor element can be oriented substantially perpendicular to the optical axis of the detector. Again, regarding the term "substantially perpendicular," refer to the definitions and tolerances given above. The optical axis can be a straight optical axis, or it can be bent or even split, such as by using one or more deflecting elements and / or by using one or more beam splitters, wherein in the latter case, the substantially perpendicular orientation can refer to the local optical axis in a corresponding branch or beam path of the optical setup.

[0031] As used herein, the term "beam propagating from an object" refers to at least one beam of light that may originate from an object or from an illumination source (such as an illumination source that directly or indirectly illuminates the object), wherein the beam is reflected or scattered by the object and thus guided at least partially toward the detector. A beam propagating from an object may also be referred to below as a "reflected beam." The detector can be used in active and / or passive illumination scenarios. For example, at least one illumination source may be adapted to illuminate an object, for example, by guiding the beam toward the object, which reflects the beam. The illumination source may be or may include at least one multi-beam light source. For example, the illumination source may include at least one laser source and one or more diffractive optical elements (DOEs). Additionally or alternatively, the detector may use radiation already present in the scene, such as radiation from at least one ambient light source.

[0032] Specifically, the light beam propagating from the object to the detector can completely illuminate at least one optical sensor in the optical sensors, such that at least one optical sensor is completely within the beam, and the width of the beam is larger than the photosensitive area of ​​at least one optical sensor that generates the sensor signal. Conversely, preferably, the reflected beam can specifically produce a spot smaller than the matrix over the entire matrix, such that the spot is completely within the matrix. Those skilled in the art of optics can readily adjust this by selecting one or more suitable lenses or elements that have a focusing or defocusing effect on the beam, such as by using suitable transfer devices, which will be further detailed below. As used further herein, "spot" generally refers to the visible or detectable circular or non-circular illumination of an article, area, or object by the light beam.

[0033] As used further herein, "sensor signal" generally refers to a signal generated by an optical sensor in response to illumination by a light beam. Specifically, a sensor signal can be or may include at least one electrical signal, such as at least one analog electrical signal and / or at least one digital electrical signal. More specifically, a sensor signal can be or may include at least one voltage signal and / or at least one current signal. More specifically, a sensor signal may include at least one photocurrent. Furthermore, an auxiliary sensor signal can be generated using the original sensor signal, or a detector, optical sensor, or any other element adapted to process or preprocess the sensor signal, which can also be used as a sensor signal, such as through preprocessing by filtering, etc.

[0034] Specifically, the photosensitive region can be oriented toward the object. As used herein, the term "oriented toward the object" generally refers to the situation where the corresponding surface of the photosensitive region is fully or partially visible from the object. Specifically, at least one interconnecting line between at least a point on the object and at least a point on the corresponding photosensitive region can form an angle of not 0° with the surface element of the photosensitive region, such as an angle in the range of 20° to 90°, preferably an angle in the range of 80° to 90°, such as an angle of 90°. Therefore, when the object is located on or near the optical axis, the light beam propagating from the object toward the detector can be substantially parallel to the optical axis. As used herein, the term "substantially perpendicular" refers to the condition of perpendicular orientation, such as a tolerance of ±20° or less, preferably ±10° or less, more preferably ±5° or less. Similarly, the term "substantially parallel" refers to the condition of parallel orientation, such as a tolerance of ±20° or less, preferably ±10° or less, more preferably ±5° or less.

[0035] As used herein, the term "ray" generally refers to a line perpendicular to the wavefront of light, pointing in the direction of energy flow. As used herein, the term "beam" generally refers to a collection of rays. Hereinafter, the terms "ray" and "beam" will be used synonymously. As further used herein, the term "beam" generally refers to a quantity of light, particularly a quantity of light traveling substantially in the same direction, including the possibility that the beam has a spreading angle or a widening angle. A beam may have spatial extension. Specifically, a beam may have a non-Gaussian beam profile. The beam profile may be selected from the group consisting of: trapezoidal beam profile; triangular beam profile; and conical beam profile. A trapezoidal beam profile may have a plateau region and at least one edge region. As will be outlined in more detail below, a beam may specifically be a Gaussian beam or a linear combination of Gaussian beams. However, other embodiments are possible. The delivery device may be configured to adjust, define, and determine one or more of the beam profiles (particularly the shape of the beam profile).

[0036] Optical sensors can be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. Specifically, optical sensors can be sensitive in the visible spectral range from 390 nm to 780 nm, most preferably from 650 nm to 750 nm, or from 690 nm to 700 nm. Specifically, optical sensors can be sensitive in the near-infrared region. Specifically, optical sensors can be sensitive in a portion of the near-infrared region, with silicon photodiodes being particularly suitable in the 700 nm to 1000 nm range. Specifically, optical sensors can be sensitive in the infrared spectral range, specifically from 780 nm to 3.0 micrometers. For example, each optical sensor can independently be or can include at least one element selected from the group consisting of photodiodes, phototubes, photoconductors, phototransistors, or any combination thereof. For example, an optical sensor can be or can include at least one element selected from the group consisting of CCD sensor elements, CMOS sensor elements, photodiodes, phototubes, photoconductors, phototransistors, or any combination thereof. Any other type of photosensitive element can be used. As will be further detailed below, photosensitive elements can typically be made entirely or partially of inorganic materials and / or entirely or partially of organic materials. Most commonly, as will be further detailed below, one or more photodiodes, such as commercially available photodiodes, such as inorganic semiconductor photodiodes, can be used.

[0037] The sensor element is configured to determine at least one reflected image. As used herein, the term "reflected image" means an image determined by the sensor element that includes at least one reflective feature. As used herein, the term "reflective feature" means a feature in an image plane generated in response to illumination of an object, such as at least one illumination feature. As used herein, the term "illumination feature" means at least one feature of arbitrary shape generated by at least one ambient light source or at least one illumination source suitable for illuminating the object. As used herein, the term "determine at least one reflected image" means one or more of the imaging, recording, and generation of a reflected image.

[0038] A reflective image may include at least one reflective pattern. As used herein, the term "reflective pattern" refers to a response pattern generated by the reflection or scattering of light on the surface of an object (particularly generated by the object in response to illumination of an illumination pattern). The illumination pattern may include at least one feature suitable for illuminating the object. The illumination feature may be generated by ambient light or by at least one illumination source. The reflective pattern may include at least one feature corresponding to at least one feature of the illumination pattern. Compared to the illumination pattern, the reflective pattern may include at least one distortion pattern, wherein the distortion depends on the distance to the object, such as the surface characteristics of the object.

[0039] The detector may further include an illumination source. As an example, the illumination source may be configured to generate an illumination beam for illuminating an object. The detector may be configured such that the illumination beam propagates from the detector along the optical axis of the detector toward the object. For this purpose, the detector may include at least one reflective element, preferably at least one prism, for deflecting the illumination beam onto the optical axis.

[0040] The illumination source can be adapted to generate at least one illumination pattern for illuminating an object. Alternatively or additionally, the illumination pattern can be generated by at least one ambient light source. The detector can be configured such that the illumination pattern propagates from the detector, particularly from at least one opening in the housing, along and / or parallel to the optical axis of the detector toward the object. For this purpose, the detector may include at least one reflective element, preferably at least one prism, for deflecting the illumination pattern such that it propagates along or parallel to the optical axis. Specifically, the illumination source may include at least one laser and / or laser source. Various types of lasers can be employed, such as semiconductor lasers. Alternatively or additionally, non-laser light sources, such as LEDs and / or bulbs, can be used. As used herein, the term "pattern" refers to any known or predetermined arrangement comprising at least one feature of arbitrary shape. A pattern may include at least one feature, such as a dot or symbol. A pattern may include multiple features. A pattern may include an arrangement of periodic or non-periodic features. As used herein, the term "illumination pattern" refers to a pattern illuminating an object. The illumination pattern can be generated by ambient light (such as by at least one ambient light source or by at least one illumination source). The illumination pattern may include at least one pattern selected from the group consisting of: at least one point pattern, particularly a pseudo-random point pattern; a random point pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one known feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including a convex uniform tiling pattern; at least one line pattern including at least one line; at least one line pattern including at least two lines such as parallel lines or intersecting lines. For example, the illumination source may be adapted to generate and / or project point clouds. The illumination pattern may include regular and / or constant and / or periodic patterns, such as triangular patterns, rectangular patterns, hexagonal patterns, or patterns including other convex tiling patterns. The illumination pattern may include as many features as possible in each region, such that a hexagonal pattern may be preferred. The distance between two features of the illumination pattern and / or the area of ​​at least one illumination feature may depend on the circle of confusion in the image.

[0041] The illumination source may include one or more of the following: at least one optical projector; at least one digital light processing (DLP) projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one light-emitting diode array; at least one laser source array. The illumination source may include at least one light source adapted to directly generate an illumination pattern. For example, the illumination source may include at least one laser source. For example, the illumination source may include at least one line laser. The line laser may be adapted to send laser lines, such as horizontal or vertical laser lines, to an object. The illumination source may include multiple line lasers. For example, the illumination source may include at least two line lasers, which may be arranged such that the illumination pattern includes at least two parallel or intersecting lines. The illumination source may include at least one optical projector adapted to generate a point cloud, such that the illumination pattern may include multiple point patterns. The illumination source may include at least one mask adapted to generate an illumination pattern based on at least one beam generated by the illumination source. The illumination source may be attached to or integrated into a mobile device such as a smartphone. The illumination source may be used for other functions that can be used to determine an image, such as autofocus. Irradiation devices can be attached to mobile devices, for example, by using connectors such as USB connectors or telephone connectors such as headphone jacks.

[0042] Specifically, the illumination source can be configured to emit light within the infrared spectral range. However, it should be noted that other spectral ranges are also feasible, either additionally or alternatively. Furthermore, the illumination source can specifically be configured to emit modulated or unmodulated light. In the case of using multiple illumination sources, the different illumination sources can have different modulation frequencies, as further detailed below, which can later be used to distinguish the beams. The detector can be configured to evaluate a single beam or multiple beams. In the case of multiple beams propagating from an object to the detector, means for distinguishing the beams can be provided. Thus, the beams can have different spectral characteristics, and the detector can include one or more wavelength selection elements for distinguishing different beams. Each beam can then be evaluated independently. As an example, the wavelength selection element can be or can include one or more filters, one or more prisms, one or more gratings, one or more dichroic mirrors, or any combination thereof. Furthermore, either additionally or alternatively, the beams can be modulated in a specific manner to distinguish two or more beams. Thus, as an example, the beams can be frequency modulated, and the sensor signals can be demodulated to partially distinguish sensor signals originating from different beams based on their demodulation frequencies. These techniques are generally known to those skilled in the art of high-frequency electronics. Typically, evaluation equipment can be configured to distinguish between different beams with different modulations.

[0043] Specifically, the illumination source and the optical sensor can be arranged in a common plane or in different planes. The illumination source and the optical sensor can have different spatial orientations. In particular, the illumination source and sensor elements can be arranged in a twisted arrangement.

[0044] The illumination source can be adapted to generate and / or project point clouds, such that multiple illumination regions are generated on the matrix of an optical sensor (e.g., a CMOS detector). Additionally, interference may exist on the matrix of the optical sensor, such as interference caused by speckles and / or external light and / or multiple reflections. The evaluation device can be adapted to determine at least one region of interest, such as one or more pixels illuminated by a beam, which is used to determine the ordinate of an object. For example, the evaluation device can be adapted to perform filtering methods, such as speckle analysis and / or edge filtering and / or object recognition methods.

[0045] As used further herein, the term "evaluation device" generally refers to any device suitable for performing specified operations, preferably by using at least one data processing device, and more preferably by using at least one processor and / or at least one application-specific integrated circuit (ASIC). Thus, by way of example, at least one evaluation device may include at least one data processing device having software code stored thereon, the software code comprising a plurality of computer commands. The evaluation device may provide one or more hardware elements for performing one or more specified operations, and / or may provide software running thereon to one or more processors for performing one or more specified operations. The evaluation device may include one or more programmable devices, such as one or more computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs), configured to perform image analysis, such as the selection of reference features and the determination of the ordinate z. However, alternatively or additionally, the evaluation device may also be entirely or partially embodied in hardware.

[0046] The evaluation device is configured to select at least one reflection feature of the reflected image at at least one first image location in the reflected image. As used herein, the term "select at least one reflection feature" means identifying, determining, and selecting one or more of at least one reflection feature of the reflected image. The evaluation device may be adapted to perform at least one image analysis and / or image processing to identify the reflection feature. The image analysis and / or image processing may use at least one feature detection algorithm. Image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image generated by a sensor signal and at least one offset; inverting a sensor signal by inverting an image generated by a sensor signal; forming a difference image between images generated by a sensor signal at different times; background correction; decomposition into color channels; decomposition into hue, saturation, and luminance channels; frequency decomposition; singular value decomposition; applying a Canny edge detector; applying a Laplacian operator to a Gaussian filter; applying a difference Gaussian filter; applying a Sobel operator; applying a Laplacian operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high-pass filter; applying a low-pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; creating a binary image. The region of interest may be determined manually by the user or automatically, such as by identifying objects within an image generated by an optical sensor.

[0047] The term "image position in a reflective image" refers to any location of a reflective feature within the reflective image. For example, in the case of a point-like reflective feature, the image position could be the x and y coordinates of the reflective feature. Similarly, in the case of an extended reflective feature, the image position could be the x and y coordinates of a center point of the reflective feature. The ordinate z could correspond to the first image position of the reflective feature. The terms "first," "second," "third," etc., image positions are used only as names and do not indicate the order of the image positions.

[0048] The evaluation device is configured to optimize at least one fuzzy function f aThe evaluation device is configured to determine at least one ordinate z of the selected reflective feature. Specifically, the evaluation device can be configured to determine at least one distance estimate. As used herein, the term "distance estimate" refers to at least one estimate of the ordinate, specifically, at least one uncertainty interval defined by the ordinate z and a determined measurement uncertainty ± ε for the ordinate. The error interval ε may depend on the measurement uncertainty of the optical sensor and / or other parameters such as temperature, motion, etc. The measurement uncertainty of the optical sensor may be predetermined and / or estimated and / or may be stored in at least one data storage unit of the evaluation device. For example, the error interval may be ±10%, preferably ±5%, more preferably ±1%. The determination of the distance estimate can produce a distance estimate with an error bar, which is typically larger than the error bar of the triangulation method. The ordinate z can be determined by using at least one convolution-based algorithm (such as a defocus depth algorithm). To obtain the distance to the reflective feature, the defocus depth algorithm estimates the defocus of the object. For this estimate, a blur function is assumed. As used herein, the term "blur function fa" (also called a blur kernel or point spread function) refers to the detector's response function to illumination from the object. Specifically, the blur function models the blur of a defocused object. At least one blur function fa can be a function or composite function consisting of at least one function from the group consisting of: Gaussian function, sine function, pillbox function, square function, Lorentz function, radial function, polynomial, Hermite polynomial, Zernike polynomial, Legendre polynomial.

[0049] The sensor element can be adapted to determine at least one reflection pattern. The evaluation device can be adapted to select at least one feature of the reflection pattern and determine the ordinate z of the selected feature of the reflection pattern by optimizing at least one fuzzy function fa.

[0050] The blur function can be optimized by changing at least one parameter of the blur function. The reflected image can be a blurred image i. b The evaluation device can be configured to evaluate based on the blurred image i b and fuzzy function f a Reconstruct the ordinate z. The blurred image ib and the blurred function f can be minimized by changing the parameter σ of the blur function. a With at least one other image i' b The difference between the convolutions (*) determines the ordinate z.

[0051] min||(i′ b *f a (σ(z))-i b )||,

[0052] σ(z) is a set of distance-dependent blur parameters. Another image may be blurry or sharp. As used herein, the term "sharp" or "sharp image" refers to the blurry image with maximum contrast. Blurry images i can be obtained by convolving them with a known blur function. b At least one other image is generated. Therefore, a defocus depth algorithm can be used to obtain a distance estimate of the reflection features. This distance estimate can be used to efficiently select the region in which the epipolar line is chosen, which will be outlined in more detail below. The distance can then be calculated using triangulation and the selected epipolar line. Unlike most triangulation methods, the determination of the distance estimate can be applied to a single feature of the reflection image. Therefore, the determination of the distance estimate can be used to accelerate triangulation methods by generating a smaller region that solves the correspondence problem.

[0053] The evaluation device is configured to determine at least one reference feature in at least one reference image at at least one second image location corresponding to at least one reflection feature in the reference image. Specifically, the evaluation device (e.g., at least one image processing device of the evaluation device) may be configured to perform at least one image analysis to determine the reference feature corresponding to the reflection feature in the reference image. As used herein, the term "reference image" refers to an image different from the reflection image. The reference image can be determined by one or more of the following: recording at least one reference feature, imaging at least one reference feature, and calculating the reference image. The reference image and the reflection image are determined in two different spatial configurations. For example, one of the reference image and the reflection image may be determined by a first optical sensor in a first spatial position and / or spatial orientation, wherein the other of the reference image and the reflection image may be determined by a second optical sensor in a second spatial position and / or spatial orientation different from the first spatial position and / or orientation. For example, the reference image and the reflection image may be determined by the same optical sensor at different time points and / or in different spatial positions and / or in different spatial orientations. The evaluation device may include at least one storage device in which at least one reference image may be stored. The reference image may be a predetermined and / or known reference image. For example, when using structured light, the evaluation device can be adapted to select at least one pattern depending on the structured light and / or determine a corresponding reference image.

[0054] As used herein, the term "spatial configuration" refers to the arrangement of a reference image and a reflected image in space, such as orientation, particularly spatial angles and / or torsion angles, and / or position. Specifically, spatial configuration can refer to the arrangement of at least two sensor elements and / or at least one sensor element and at least one illumination source in space, such as orientation, particularly spatial angles and / or torsion angles, and / or position. As used herein, the term "relative spatial constellation" refers to the relative alignment of a reference image and a reflected image in space. A relative spatial constellation can be at least one constellation selected from the group consisting of: relative spatial orientation; relative angular position; relative distance; relative displacement; relative movement. For example, a relative spatial constellation can be the relative spatial orientation and / or relative angular position and / or relative distance and / or relative displacement and / or relative movement of at least two sensor elements and / or at least one sensor element and at least one illumination source. A relative spatial constellation can be a baseline. As used herein, the term "baseline" refers to a relative spatial orientation and / or relative angular position, such as a relative angle or relative torsion angle and / or relative distance and / or relative displacement. The evaluation device can be adapted to store determined values ​​of the relative spatial constellation.

[0055] In one embodiment, the detector may include at least two sensor elements separated by a relative spatial constellation, particularly a baseline. At least one first sensor element may be adapted to record a reference image and at least one second sensor element may be adapted to record a reflected image. Specifically, the first sensor element may be configured to image the reference image, and at least one second sensor element may be configured to image the reflected image. A first optical sensor may be arranged such that its sensor element receives at least one image of the reference feature. A second optical sensor may be arranged such that its sensor element receives at least one image of the reflected feature. The first and second sensor elements may be separated by a mechanical connector. The mechanical connector may be adjustable and / or non-permanent. The detector may include at least one stereo camera.

[0056] In one embodiment, the detector may be adapted to record reflected images and reference images using the same optical sensor matrix at different times. For example, the sensor elements may be adapted to move from a first spatial configuration or be moved to a second spatial configuration, for example, at a constant or variable speed. Specifically, the detector may be configured to position the optical sensor matrix at a first location, wherein the optical sensor matrix may be configured to image the reference image at the first location. The detector may be configured to position the optical sensor matrix at a second location different from the first location, wherein the optical sensor matrix may be configured to image the reflected image at the second location.

[0057] In one embodiment, the detector may include at least one illumination source. The illumination source and at least one sensor element can be separated by a baseline (e.g., via a mechanical connector). The mechanical connector may be adjustable and / or non-permanent.

[0058] As used herein, the term "determining at least one reference feature in at least one reference image at at least one second image location corresponding to at least one reflection feature" means selecting a reference feature in the reference image that corresponds to the reflection feature. As summarized above, the evaluation device may be adapted to perform image analysis and identify features of the reference image. The evaluation device may be adapted to identify at least one reference feature in the reference image having a ordinate that is substantially the same as the selected reflection feature. The term "substantially the same" means identical within 10%, preferably 5%, and most preferably 1%. The reference feature corresponding to the reflection feature can be determined using epipolar geometry. For a description of epipolar geometry, see, for example, Chapter 2 of X. Jiang, H. Bunke: "Dreidimensionales Computersehen," Springer, Berlin Heidelberg, 1997. Epipolar geometry may assume that the reference image and the reflection image are images of objects determined, for example, at different spatial locations and / or spatial orientations with a fixed distance during the recording of the reference image and the reflection image. This distance may be a relative distance, also represented as a baseline. The evaluation device may be adapted to determine epipolar lines in the reference image. The relative positions of the reference image and the reflection image can be known. For example, the relative positions of the reference image and the reflected image can be stored in at least one storage unit of the evaluation device. The evaluation device can be adapted to determine a straight line extending from a selected reflective feature of the reflected image to a real-world feature from which it originates. Thus, the straight line can include possible object features corresponding to the selected reflective feature. The straight line and the baseline cross the epipolar plane. Since the reference image is determined at a different relative constellation than the reflected image, the corresponding possible object features can be imaged on a straight line in the reference image called the epipolar line. The epipolar line can be the intersection of the epipolar plane and the reference image. Therefore, the feature of the reference image corresponding to the selected feature of the reflected image lies on the epipolar line. Due to image distortion or variations in system parameters, such as due to aging, temperature changes, mechanical stress, etc., the epipolar lines may intersect or be very close to each other and / or the correspondence between the reference feature and the reflective feature may be unclear. Furthermore, every known location or object in the real world can be projected onto the reference image, and vice versa. The projection may be known due to detector calibration, which is equivalent to teaching the epipolar geometry of a particular camera.

[0059] Depending on the distance to the object, a reference feature corresponding to the second image position of the reflection feature is displaced within the reference image compared to the first image position. The reference image may include at least one displaced region where the reference feature corresponding to the selected reflection feature can be imaged. The displaced region may include only one reference feature. The displaced region may also include more than one reference feature. The evaluation device may be configured to determine at least one longitudinal region, wherein the longitudinal region is given by a ordinate z and an error interval ±ε. The evaluation device may be configured to determine at least one displaced region in the reference image corresponding to the longitudinal region. The displaced region may include an epipolar line or a portion of an epipolar line. The displaced region may include more than one epipolar line or multiple portions of more than one epipolar line. The displaced region may extend along an epipolar line, orthogonally to an epipolar line, or both. The evaluation device may be configured to determine at least one epipolar line in the reference image. The evaluation device may be adapted to determine the reference feature corresponding to the ordinate z along the epipolar line and to determine the extent of the displaced region corresponding to or orthogonal to the error interval ±ε along the epipolar line. Measurement uncertainty in distance estimation can lead to non-circular displacement regions because the measurement uncertainty may differ in different directions. Specifically, the measurement uncertainty along one or more epipolar lines may be greater than the measurement uncertainty in directions orthogonal to one or more epipolar lines. The displacement region can include extensions in directions orthogonal to one or more epipolar lines. The evaluation device can determine the displacement region around the image location of the reflection feature. The evaluation device can be adapted to determine the ordinate and the displacement region corresponding to z±ε along the epipolar line. The evaluation device can determine the displacement region around a second image location of the reflection feature.

[0060] The evaluation device may be adapted to match selected reflection features with at least one reference feature within a displacement region. As used herein, the term "match" refers to determining and / or evaluating corresponding reference and reflection features. The evaluation device may be adapted to match selected features of a reflection image with reference features within a displacement region using at least one evaluation algorithm that takes into account the determined ordinate z. The evaluation algorithm may be a linear scaling algorithm. The evaluation device may be adapted to determine the epipolar line closest to and / or within the displacement region. The evaluation device may be adapted to determine the epipolar line at the second image location closest to the reflection feature. The extent of the displacement region along the epipolar line may be greater than the extent of the displacement region orthogonal to the epipolar line. The evaluation device may be adapted to determine the epipolar line before determining the corresponding reference feature. The evaluation device may determine the displacement region around the second image location of each reflection feature. The evaluation device may be adapted to assign an epipolar line to each displacement region of each second image location of the reflection feature, such as by assigning the epipolar line closest to and / or within the displacement region and / or along a direction orthogonal to the epipolar line. The evaluation device can be adapted to determine a reference feature corresponding to a second image position of a reflection feature by identifying a reference feature that is closest to the assigned displacement region and / or along the assigned epipolar line within the assigned displacement region and / or along the assigned epipolar line within the assigned displacement region.

[0061] Alternatively or concurrently, the evaluation device may be configured to perform the following steps:

[0062] - Determine the displacement region for the second image position of each reflection feature;

[0063] - For example, by assigning the epipolar line to the displacement region of each reflection feature by assigning the epipolar line closest to the displacement region and / or the epipolar line closest to the displacement region in a direction orthogonal to the epipolar line;

[0064] - Assign and / or determine at least one reference feature to each reflection feature, such as by assigning a reference feature that is closest to the assigned displacement region and / or along the assigned epipolar line within the assigned displacement region.

[0065] Alternatively or concurrently, the evaluation device may be adapted to make a decision among more than one epipolar and / or reference features to be assigned to a reflection feature, such as by comparing the distances between reflection features and / or epipolar lines in a second image and / or by comparing error-weighted distances (such as the ε-weighted distances between reflection features and / or epipolar lines in a second image), and assigning epipolar and / or reference features with shorter distances and / or ε-weighted distances to reference features and / or reflection features.

[0066] The evaluation device can be adapted to determine the displacements of a reference feature and a reflection feature. The evaluation device can be adapted to determine the displacements of a matched reference feature and a selected reflection feature. The evaluation device (e.g., at least one data processing device of the evaluation device) can be configured to determine the displacements of the reference feature and the reflection feature specifically by comparing the corresponding image positions of the reference image and the reflection image. As used herein, the term "displacement" refers to the difference between the image position in the reference image and the image position in the reflection image. The evaluation device can be adapted to determine longitudinal information of the matched feature using a predetermined relationship between the ordinate and the displacement. As used herein, the term "longitudinal information" refers to information related to the ordinate z-axis. triang Relevant information. For example, longitudinal information could be distance values. The evaluation device can be adapted to determine predetermined relationships using triangulation methods. Given that the positions of selected reflection features and matching reference features in the reflection image and / or the relative displacements of the selected reflection features and matching reference features are known, the ordinate of the corresponding object feature can be determined by triangulation. Therefore, the evaluation device can be adapted to select, for example, subsequent and / or column-by-column reflection features and use triangulation to determine the corresponding distance values ​​for each potential position of the reference feature. The displacements and corresponding distance values ​​can be stored in at least one storage device of the evaluation device. As an example, the evaluation device may include at least one data processing device, such as at least one processor, at least one DSP, at least one FPGA, and / or at least one ASIC. Furthermore, to store at least one predetermined or determinable relationship between the ordinate and the displacement, at least one data storage device can be provided, such as for providing one or more lookup tables for storing predetermined relationships. The evaluation device can be adapted to store parameters for intrinsic and / or extrinsic calibration of the camera and / or detector. The evaluation device can be adapted to generate parameters for intrinsic and / or extrinsic calibration of the camera and / or detector, such as by performing Tsai camera calibration. Evaluation equipment can be used to calculate and / or estimate parameters such as the focal length of the transmission device, the radial lens distortion coefficient, the coordinates of the radial lens distortion center, the scaling factor used to resolve any uncertainties caused by imperfections in the hardware timing of scanning and digitization, the rotation angle of the transformation between world coordinates and camera coordinates, the translation component of the transformation between world coordinates and camera coordinates, the aperture angle, the image sensor format, the principal point, the tilt coefficient, the camera center, the camera heading, the baseline, the rotation or translation parameters between the camera and / or the illumination source, the aperture, the focal length, etc.

[0067] The evaluation device is configured to determine the relative spatial constellation based on the ordinate z and the positions of the first and second images. As outlined above, epipolar geometry may require a good understanding of the relative spatial constellations of the reflected and reference images, particularly the baseline. However, the relative spatial constellations of detector components (such as the illumination source and sensor elements) or at least two sensor elements may be unknown and / or may change during measurement time, for example, due to thermal effects or mechanical stress. The determined ordinate z can be used to recalibrate the triangulation system. As outlined above, the evaluation device can be adapted to determine the displacement of the reference and reflected features. The evaluation device can be adapted to determine the ordinate z by triangulating the object. triang A predefined relationship between the object and its displacement is used to determine at least one triangulation ordinate z of the object. triang Triangulation of the ordinate z triang The relationship can be determined using epipolar geometry (assuming a fixed spatial relative constellation and employing predefined and / or predetermined values ​​of the spatial relative constellation) based on the positions of the first and second images. As used herein, the term "predefined relationship" refers to an assumed relationship and / or a predetermined relationship and / or a preset relationship. In particular, the predefined relationship can depend on the relative spatial constellation. The evaluation device can be adapted to store the predefined relationship. The evaluation device can be adapted to compare the ordinate z determined by using a defocus depth algorithm with the triangulated ordinate z. triang The evaluation device can be adapted to consider the determined relative spatial constellation to determine the actual relationship between the ordinate z and the displacement. The evaluation device (e.g., at least one data processing device of the evaluation device) can be configured to consider the determined relative spatial constellation to determine the actual relationship between the ordinate z and the displacement. The term "actual relationship" refers to the relationship between the triangulated ordinate and the displacement caused by changes in the relative spatial constellation, particularly the baseline, such as due to movement or environmental influences (e.g., temperature). The evaluation device can be adapted to rely on the actual relationship to adjust for predefined relationships. The evaluation device can be adapted to replace predefined relationships with actual relationships, particularly stored predefined relationships, and / or the evaluation can be adapted to determine a moving average and replace the predefined relationship with the moving average. The evaluation device can be adapted to determine the relationship between the ordinate z and the triangulated ordinate z. triang The difference between them. The evaluation device may be adapted to compare the determined difference with at least one threshold and adjust a predefined relationship if the determined difference is higher than or equal to the threshold. The evaluation device may be adapted to determine a relative spatial constellation based on the actual relationship and the ordinate z. For example, the detector may include at least one system comprising at least one illumination source and a sensor element, b is the baseline, d is the displacement on the sensor, f is the focal length of the detector's transfer device, and β is the angle between the illumination source and the baseline. For β = 90°, it is

[0068] as well as,

[0069]

[0070] Therefore, given that the absolute distance to the object (i.e., the ordinate z determined from the defocus depth algorithm) is known, z triang The baseline b can be replaced by z, and the corrected baseline can be used instead. cor It can be calculated using the following formula

[0071]

[0072] For β less than 90°, it is Therefore, given that the absolute distance z to the object is known, the corrected baseline b cor It can be calculated using the following formula

[0073]

[0074] And the angle β can be determined from the following formula.

[0075]

[0076] Since β and b may vary simultaneously, these two values ​​can be determined using subsequent measurements. Therefore, measuring the ordinate z of a feature point can be used to correct predefined relationships; furthermore, triangulation of the ordinate z... triang The distance from the sensor element to the object (i.e., determined by triangulation) is known. The evaluation device can be adapted to use the ordinate z and the triangulated ordinate z. triang To determine and / or correct and / or calibrate the values ​​of relative space constellations (such as baseline values).

[0077] The evaluation equipment can be adapted to determine the corrected estimates of relative spatial relationships using a mathematical model, which includes parameters such as various sensor signals and / or position and / or image position and / or system characteristics and / or ordinate, displacement d on the sensor, focal length f of the transmission device, temperature, z-axis. triang The parameters include the baseline b, the angle β between the illumination source and the baseline, and the ordinate z. The mathematical model can be selected from: Kalman filter, linear quadratic estimator, Kalman-Busch filter, Stratonovich-Kalman-Busch filter, Kalman-Busch-Stratonovich filter, minimum variance estimator, Bayesian estimator, optimal linear unbiased estimator, invariant estimator, Wiener filter, etc., to account for the measurement errors and inaccuracies experienced by each sensor signal. The fusion of these sensor signals in one of the aforementioned mathematical models (such as the Kalman filter) can be achieved, for example, by measuring the relative spatial constellation and / or the ordinate z and / or triangulating the ordinate z. triang and / or corrected baseline b corThis produces improved estimates and further contributes to improved error compensation.

[0078] The ordinate z can be determined for multiple feature points, and the ordinate z can be triangulated. triang This is particularly important for obtaining statistical confirmation values ​​of calibration relationships and / or calibration relative space constellations. Since relative space constellations do not change abruptly, such statistical assessments may be very suitable.

[0079] The evaluation device can be adapted to repeatedly determine and / or recalibrate the relative space constellation. Sensor elements can be adapted to determine at least one second reflection image. The evaluation device can be adapted to select at least one second reflection feature of the second reflection image at at least one third image location within the second reflection image, and to optimize at least one blur function f. a The evaluation device determines at least one second ordinate z of the second reflection feature. The evaluation device may be adapted to determine at least one second reference feature corresponding to the at least one second reflection feature in at least one second reference image at at least one fourth image position in the second reference image. The second reference image and the second reflection image may be determined with two different second spatial configurations. The spatial configuration may differ depending on the actual relative spatial constellation. The evaluation device may be adapted to determine the actual relative spatial constellation based on the second ordinate z and the third and fourth image positions. The evaluation device may be adapted to compare a first relative spatial constellation with the actual relative spatial constellation. The evaluation device may be adapted to adjust the first relative spatial constellation based on the actual relative spatial constellation. The evaluation device may be adapted to replace the first relative spatial constellation with the actual relative constellation and / or the evaluation may be adapted to determine a moving average and replace the first relative spatial constellation with the moving average. The evaluation device may be adapted to determine the difference between the first relative spatial constellation and the actual relative spatial constellation. The evaluation device may be adapted to compare the determined difference with at least one threshold and adjust the first relative spatial constellation if the determined difference is higher than or equal to the threshold.

[0080] Baseline drift may occur, particularly due to temperature variations and mechanical damage. The detector may include at least one temperature determination unit. This unit is adapted to determine at least one temperature value for the detector. The evaluation device is adapted to consider temperature values ​​when determining the relative spatial constellation and / or to adjust the relative spatial constellation based on those values. The evaluation device is adapted to monitor the evaluation and / or changes in the relative spatial constellation. Monitoring the temperature of the system, particularly the temperature of the mechanical connections forming the baseline, can improve the monitoring of the relative spatial constellation.

[0081] The evaluation device is suitable for setting up a system including at least one sensor element and at least one irradiation source. The at least one sensor element and irradiation source can be located in a fixed position, but without predetermined and / or direct and / or even stable mechanical connections. The mechanical connectors can be adjustable and / or non-permanent. The relative spatial constellation of the at least one sensor and irradiation source can be adjusted manually or automatically (e.g., automatically using a motor or manually by the user or during manufacturing steps). The relative spatial constellation can even be changed or adjusted during use. The evaluation device can be adapted to optimize at least one fuzzy function f. a The ordinate z of at least one reflective feature in the reflective image determined by the sensor element is determined. As outlined above, the evaluation device can be adapted to use the ordinate z to determine the relative spatial constellation of the illumination source and the sensor element, and thus calibrate the system.

[0082] The evaluation device can be adapted to set up a moving stereo system with a flexible relative constellation, particularly a flexible baseline, including a first sensor element and a second sensor element. The first and second sensor elements can be positioned in fixed locations, but without predetermined and / or direct and / or even stable mechanical connections. The mechanical connection can be via a mechanical connector. The mechanical connection can be adjustable and / or non-permanent.

[0083] The relative spatial constellation of at least the first and second sensor elements can be adjusted manually or automatically (e.g., automatically using a motor, manually by the user, or during manufacturing steps). The relative spatial constellation can even be changed or adjusted during use. The evaluation device can be adapted to optimize at least one fuzzy function f. a The ordinate z of at least one reflective feature in the reflective image determined by one of the sensor elements is determined. As outlined above, the evaluation device can be adapted to use the ordinate z to determine the relative spatial constellation of the first and second sensor elements, and thus calibrate the system.

[0084] Using the defocus depth method allows for the estimation of distances, such as the ordinate z within an error interval ε. By determining the displacement region corresponding to the estimated ordinate and the corresponding error interval, the number of possible solutions along the epipolar line can be significantly reduced. The number of possible solutions can even be reduced to a single one. The determination of the ordinate z and the error interval can be performed during pre-evaluation before matching the selected reflection feature with a reference feature. This allows for a reduction in computational requirements, resulting in a significant cost reduction and enabling use in mobile or outdoor devices.

[0085] Furthermore, in typical triangulation systems, a large baseline is required to detect large distances. Pre-evaluating the ordinate z and error interval using distance estimation and subsequent matching of selected reflection features with reference features allows for the use of a short baseline, enabling the provision of a compact device. Moreover, the defocus depth result is independent of the baseline or the location of the light source. Furthermore, pre-evaluating the ordinate z and error interval using defocus depth and subsequent matching of selected reflection features with reference features can improve accuracy and / or speed and / or reduce computational requirements compared to traditional triangulation systems.

[0086] Defocus depth methods allow for the estimation of distances to individual features in an image, such as projection points. Furthermore, the number of illumination features, such as the number of illumination points in an illumination pattern, can be reduced to increase the light intensity at each illumination point, such as while competing with ambient light in compliance with eye safety regulations. The reduction in the number of illumination features in traditional triangulation systems can increase the difficulty of matching reflection features with reference features. Conversely, the number of illumination features, such as the number of illumination points in an illumination pattern, can be increased to improve the resolution of distance measurements, for example, in mobile applications, to increase the resolution of the obtained depth map without increasing the processing power of the evaluation device.

[0087] Specifically, a CMOS-based defocus depth system estimates the distance to a monocular image by integrating regions within the image and forming the quotient of these regions. Integration can be performed along circular or edge shapes. Before solving the correspondence problem, the number of possible solutions can be significantly reduced when applying defocus depth-based distance estimation. Distance estimation can be performed on selected feature points. The distance estimate and its error bar can correspond to the line X. i This refers to a section on the epipolar line. Quadratic scaling algorithms for solving correspondence problems, such as those for structured light, can be simplified to linear scaling algorithms. In correspondence problems, computational power is a significant cost driver that can be substantially reduced.

[0088] Depth of focus can be used to calibrate and / or recalibrate a triangulation system. If a distance value, such as the ordinate z, is determined for a feature point, where the triangulation distance is also determined using a depth of focus algorithm, the depth of focus distance value can be used to correct the baseline value. Depth of focus distance values ​​can be determined for a large number of triangulation measurements to obtain statistical confirmation values ​​for recalibrating the baseline. This type of statistical evaluation is well-suited because the baseline does not change abruptly. Baseline drift can occur, especially due to temperature variations and mechanical damage. Baseline monitoring can be improved by monitoring the temperature of the system (particularly the temperature of the mechanical connections forming the baseline). Furthermore, depth of focus calibration can be used to set up mobile stereo systems with flexible baselines: two cameras or one camera and an illumination source can be set in fixed positions without predetermined direct or even stable mechanical connections. The depth of focus distance value can then be used to first determine the baseline and thus calibrate the system. The calibration process can be the same as determining or re-determining the position of the illumination source.

[0089] For example, the reference image may be an image of the illumination pattern at the location of the illumination source on an image plane. The evaluation device may be adapted to determine a displacement region in the reference image corresponding to a longitudinal region of a selected feature of the reflection pattern. The evaluation device may be adapted to match the selected feature of the reflection pattern with at least one feature of the reference pattern within the displacement region.

[0090] For example, the detector may include at least two sensor elements, each having an optical sensor matrix. At least one first sensor element and at least one second sensor element may be located at different spatial positions and / or orientations. The at least one first sensor element may be adapted to determine at least one first reflection pattern, particularly at least one first reflection feature, and the at least one second sensor element may be adapted to determine at least one second reflection pattern, particularly at least one second reflection feature. The evaluation device may be adapted to select at least one image determined by the first or second sensor element as a reflection image and select at least one image determined by the other of the first or second sensor elements as a reference image. The evaluation device may be adapted to select at least one reflection feature in the reflection pattern and optimize at least one blur function f. a The ordinate is determined. The evaluation device can be adapted to determine a displacement region in the reference image corresponding to a selected longitudinal region of the reflection pattern, wherein the longitudinal region is given by the ordinate z and the error interval ±ε. The evaluation device can be adapted to match the selected feature of the reflection pattern with at least one feature of the reference pattern within the displacement region.

[0091] In one embodiment, the detector may be adapted to record a reflected image and a reference image using the same optical sensor matrix, wherein sensor elements move or are moved at a constant or variable speed from a first spatial configuration or to at least one second spatial configuration. The detector may include at least one illumination source that can be moved or fixed. The detector may be adapted, in particular, to subsequently determine multiple images, wherein one of the images can be selected as the reflected image and another image can be selected as the reference image. Evaluation equipment can be adapted to perform 3D sensing methods, such as structure recovery from motion or pose estimation, for example, as described by Ramalingam et al., “Pose Estimation using Both Points and Lines for Geo-Localization,” published in Robotics and Automation (ICRA), 2011 IEEE International Conference on Robotics and Automation, Publisher: IEEE ISBN: 978-1-61284-385-8. The term “structure recovery from motion” will be used as a synonym for both structure recovery from motion and shape recovery from motion. The lack of a fixed relative spatial constellation (such as a baseline) for reference and reflected images can lead to so-called scaling drift and loss of distance determination accuracy, or may prevent absolute distance measurements from being performed without additional information. Structure recovery from motion and pose estimation algorithms cannot reconstruct from image information determined solely by the moving sensor elements if the sensor elements are close together and moving slowly, or if the sensor elements are far apart and moving rapidly. Specifically, structure recovery from motion and pose estimation algorithms can determine longitudinal and lateral information, such as the size, dimensions, distance, and / or orientation of the object, only equivalent to (up) The scaling factor scales arbitrary distance units within the evaluation device to an absolute real-world distance scale. Specifically, motion recovery structure and pose estimation algorithms require additional image reconstruction information to scale the image information to an absolute distance scale. The evaluation device may be adapted to determine at least one scaling factor for a relative spatial constellation. The evaluation device may be configured to determine the scaling factor, for example, by using at least one processor. The evaluation device may be configured to perform a measurement of the ordinate z and determine the scaling factor based on the ordinate z. As used herein, the term "scaling factor" (also referred to as scaling factor) refers to a transformation, specifically a factor, that scales the distance units of the evaluation device to an absolute scale. For example, in a structured light system consisting of at least one image sensor and at least one illumination source, the baseline may elongate due to a rise in system temperature, resulting in an increased distance between at least one image sensor and at least one illumination source, while the focal length of the transmission element (such as at least one lens) and the distance from the lens to the sensor remain constant.In this example, when comparing two objects at the same position in a reflected image, where the first object can be measured with the original baseline in a first measurement and the second object can be measured with an extended baseline in a second measurement, the second object measured with the extended baseline appears to be farther than the object measured with the original baseline. The angle between the baseline and the straight line connecting the feature point in the reflected image to the corresponding feature point on the object itself can be the same for both objects, allowing the principle of similar triangles to be used to compare the two measurements. The distance to the object is measured along the straight line. In this example, according to the principle of similar triangles, the quotient of the object's distance to the lens and the baseline may be the same for both measurements using the original baseline and measurements using the extended baseline. Therefore, the scaling factor for scaling the original baseline to the extended baseline can be the same as the scaling factor for scaling the original object distance to the increased object distance. Therefore, according to the principle of similar triangles, the scaling factor of the baseline also scales the distance z. The determination of the scaling factor can be further refined by using sensor data from the inertial measurement unit.

[0092] The evaluation device can be adapted to optimize the fuzzy function f a A scaling factor is determined by identifying the ordinate z for at least one feature point in at least one image recorded by a sensor element. The scaling factor can be maintained for the remaining measurements and / or as long as at least one feature point can be traced from one image to another and / or can be recalculated during measurement. For example, the scaling factor can be determined in each image recorded by the sensor element. This ensures statistical validation and consistent measurement of the scaling factor. The scaling factor can be determined from a single measurement point of the image and / or from multiple measurements. In particular, the evaluation device can be adapted to determine a medium scaling factor.

[0093] The evaluation device can be adapted to infer at least one ordinate of at least one adjacent feature of the reflected image and / or the reference image from at least one determined ordinate of at least one feature of the reflected image and / or the reference image, respectively. The evaluation device can also be adapted to infer at least one ordinate z of at least one other feature i at at least one other image location in the reflected image from the ordinate z of a first image location. i The term "inference" can refer to deriving or determining the ordinate z. i .

[0094] To determine the scaling factor in the case of translational motion between the illumination source and / or sensor elements, or between two sensor elements, the detector may be adapted to determine at least two images, each including at least four feature points. To determine the scaling factor in the case of, for example, rotation of a component of the sensor element, the detector may be adapted to determine at least two images, each including at least six feature points. Preferably, the number of feature points in each image is much greater than six, such as more than 60 or more than 600 feature points. Preferably, the feature points in the images correspond to each other.

[0095] As summarized above, the detector is suitable for optimizing the ambiguity function f. a To determine at least one ordinate of the object. In one embodiment, as outlined above, the detector can be designed as a stereo system comprising two optical sensors separated by a relative spatial constellation. In this embodiment, the detector can be adapted to determine the distance from the object to the detector and / or the relative spatial constellation using stereo triangulation.

[0096] As an addition to or alternative to using solid triangulation, the detector can be designed as a structured light system comprising at least one optical sensor and at least one illumination source. In this embodiment, as outlined above, the detector can be adapted to determine the distance and / or relative spatial constellation from the object to the detector using structured light. As an addition to or alternative to using structured light and / or solid triangulation, as outlined above, the detector can be used to determine the distance and / or relative spatial constellation from the object to the detector by using methods for reconstructing structure from motion or reconstructing shape from motion. In an alternative embodiment where the detector is designed as a single optical sensor instead of a solid system, as outlined above, the detector can be adapted to determine the distance and / or relative spatial constellation from the object to the detector using structured light. As an addition to or alternative to using structured light, as outlined above, the detector can determine the distance and / or relative spatial constellation from the object to the detector by using methods for reconstructing structure from motion or reconstructing shape from motion.

[0097] The detector may further include one or more additional elements, such as one or more additional optical elements. Furthermore, the detector may be fully or partially integrated into at least one housing.

[0098] The detector may include at least one optical element selected from the group consisting of: a transfer device, such as at least one lens and / or at least one lens system, and at least one diffractive optical element. The term "transfer device" (also referred to as "transfer system") generally refers to one or more optical elements adapted to modify a beam, such as by modifying one or more of the beam parameters, beam width, or beam direction. The transfer device may be adapted to guide the beam onto an optical sensor. Specifically, the transfer device may include one or more of the following: at least one lens, such as at least one lens selected from the group consisting of: at least one focusing adjustable lens, at least one aspherical lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one reflector; at least one beam splitting element, preferably at least one of a beam splitter cube or beam splitter mirror; and at least one multi-lens system. As used herein, the term "focal length" of the transfer device refers to the distance at which an incident collimated ray that may strike the transfer device enters the "focal point," which may also be referred to as the "focal point." Thus, the focal length constitutes a measure of the transfer device's ability to converge the striking beam. Therefore, the transfer device may include one or more imaging elements that can function as a converging lens. For example, the transfer device may have one or more lenses, particularly one or more refractive lenses, and / or one or more convex mirrors. In this example, the focal length may be defined as the distance from the center of the thin refractive lens to the principal focal point of the thin lens. For converging thin refractive lenses (such as convex or biconvex thin lenses), the focal length may be considered a positive value and may provide a distance at which the collimated beam striking the thin lens acting as the transfer device can be focused into a single spot. Additionally, the transfer device may include at least one wavelength selection element, such as at least one filter. Furthermore, the transfer device may be designed (e.g., at a location in the sensor region, and particularly in the sensor region) to apply a predetermined beam profile to the electromagnetic radiation. In principle, the above-described alternative embodiments of the transfer device may be implemented individually or in any desired combination.

[0099] The transfer device may have an optical axis. Specifically, the detector and the transfer device share a common optical axis. As used herein, the term "optical axis of the transfer device" generally refers to the axis of mirror symmetry or rotational symmetry of a lens or lens system. The optical axis of the detector may be the line of symmetry of the detector's optical arrangement. The detector includes at least one transfer device, preferably at least one transfer system having at least one lens. As an example, the transfer system may include at least one beam path, wherein the elements of the transfer system in the beam path are positioned in a rotationally symmetric manner about the optical axis. Again, as will be outlined in more detail below, one or more optical elements located within the beam path may also be eccentric or tilted about the optical axis. However, in this case, the optical axis may be defined sequentially, such as through the center of the optical elements in the interconnected beam paths, for example, through the center of the interconnected lenses, wherein, in this case, the optical sensor is not considered an optical element. The optical axis may generally represent a beam path. The detector may have a single beam path along which a beam travels from an object to an optical sensor, or it may have multiple beam paths. As an example, a single beam path may be given, or the beam path may be divided into two or more partial beam paths. In the latter case, each partial beam path may have its own optical axis. Optical sensors can be located in one and the same beam path or part of the beam path. However, alternatively, optical sensors can also be located in different part of the beam path.

[0100] The transmission device can form a coordinate system where the ordinate *l* is the coordinate along the optical axis, and where *d* is the spatial offset from the optical axis. The coordinate system can also be a polar coordinate system, where the optical axis of the transmission device forms the z-axis, and where the distance from the z-axis and the polar angle can be used as additional coordinates. Directions parallel to or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered ordinate *l*. Any direction perpendicular to the z-axis can be considered transverse directions, and polar coordinates and / or polar angles can be considered transverse coordinates.

[0101] The transfer device can be arranged and / or configured to maximize focus variation within the measurement range. For devices using the depth-of-focus method, maximizing focus variation within the measurement range may be advantageous, particularly reducing the depth of focus within the measurement range.

[0102] As outlined above, a detector can be enabled to determine at least one ordinate of an object, including the option to determine the ordinate of the entire object or one or more portions thereof. However, other coordinates of the object, including one or more abscissas and / or rotational coordinates, can also be determined by the detector, specifically by the evaluation device. Thus, as an example, one or more lateral sensors can be used to determine at least one abscissa of the object. As outlined above, the position of at least one optical sensor (from which a center signal is generated) can provide information about at least one abscissa of the object, wherein, as an example, an optical transformation can be performed using a simple lens equation to derive the abscissa. Alternatively or additionally, one or more additional lateral sensors can be used and may be included by the detector. Various lateral sensors are generally known in the art, such as the lateral sensors and / or other position-sensitive devices (PSDs) disclosed in WO 2014 / 097181 A1, such as quadrant diodes, CCDs, or CMOS chips. Alternatively or alternatively, as an example, the detector according to the invention may include one or more PSDs disclosed in RAStreet (Ed.): Technology and Applications of Amorphous Silicon, Springer-Verlag Heidelberg, 2010, pp. 346-349. Other embodiments are feasible. These devices can also generally be implemented in the detector according to the invention. As an example, a portion of the light beam may be split within the detector by at least one beam-splitting element. As an example, the split portion may be directed to a lateral sensor (such as a CCD or CMOS chip or camera sensor), and the lateral position of the light spot generated by the split portion on the lateral sensor may be determined, thereby determining at least one lateral coordinate of the object. Thus, the detector according to the invention may be a one-dimensional detector, such as a simple distance measuring device, or may be embodied as a two-dimensional detector or even a three-dimensional detector. Furthermore, as outlined above or further detailed below, a three-dimensional image may also be created by scanning a scene or environment in a one-dimensional manner. Therefore, the detector according to the invention can specifically be one of a one-dimensional detector, a two-dimensional detector, or a three-dimensional detector. The evaluation device can be further configured to determine at least one abscissa x, y of the object. The evaluation device can be adapted to combine information from the ordinate and abscissa to determine the object's position in space.

[0103] The use of an optical sensor matrix offers several advantages and benefits. Therefore, the center of the light spot generated by the light beam on the sensor elements (such as on the common plane of the photosensitive areas of the optical sensors in the matrix of sensor elements) can vary with the lateral position of the object. Thus, the use of an optical sensor matrix provides significant flexibility in terms of the object's position (specifically, in terms of the object's lateral position). The lateral position of the light spot on the optical sensor matrix (such as the lateral position of at least one optical sensor that generates the sensor signal) can be used as additional information from which at least one piece of information about the object's lateral position can be derived, as disclosed, for example, in WO 2014 / 198629 A1. Additionally or alternatively, the detector according to the invention may include at least one additional lateral detector for detecting at least one lateral coordinate of the object in addition to at least one ordinate.

[0104] In another aspect of the invention, a detector system for determining the location of at least one object is disclosed. The detector system includes at least one detector according to the invention (such as according to one or more embodiments disclosed above or according to one or more embodiments disclosed in further detail below). The detector system further includes at least one beacon device adapted to direct at least one light beam toward the detector, wherein the beacon device is at least one that is attachable to an object, can be held by the object, and can be integrated into the object. Further details regarding the beacon device, including potential embodiments thereof, will be given below. Thus, the at least one beacon device may be or may include at least one active beacon device comprising one or more illumination sources, such as one or more light sources, such as lasers, LEDs, bulbs, etc. As an example, the light emitted by the illumination source may have a wavelength of 390 to 780 nm. Alternatively, as outlined above, an infrared spectral range, such as in the range of 780 nm to 3.0 μm, may be used. Specifically, the near-infrared region of a silicon photodiode, in which the range of 700 nm to 1000 nm is particularly applicable, may be used. As outlined above, the light emitted by one or more beacon devices can be unmodulated or modulated to distinguish two or more beams. Additionally or alternatively, at least one beacon device can be adapted to reflect one or more beams toward a detector, such as by including one or more reflective elements. Furthermore, the at least one beacon device can be or may include one or more scattering elements adapted to scatter the beam. Elastic or inelastic scattering can be used. Where at least one beacon device is adapted to reflect and / or scatter the main beam toward the detector, the beacon device can be adapted to ensure that the spectral characteristics of the beam are not affected, or alternatively, can be adapted to change the spectral characteristics of the beam, such as by modifying the wavelength of the beam.

[0105] In another aspect of the invention, a human-machine interface (HMI) for exchanging at least one piece of information between a user and a machine is disclosed. The HMI includes at least one detector system according to one or more embodiments disclosed above and / or according to the embodiments further detailed below. At least one beacon device is adapted to be attached to or held by a user, either directly or indirectly. The HMI is designed to determine at least one location of the user by means of the detector system, wherein the HMI is designed to assign at least one piece of information to that location.

[0106] In another aspect of the invention, an entertainment device for performing at least one entertainment function is disclosed. The entertainment device includes at least one human-machine interface according to one or more embodiments disclosed above and / or according to the embodiments further detailed below. The entertainment device is configured to allow a player to input at least one piece of information via the human-machine interface. The entertainment device is further configured to change the entertainment function based on the information. The entertainment device may be or may include at least one device selected from the group consisting of: a television, a smartphone, a game console, a video recorder, a DVD player, a personal computer, a laptop computer, a tablet computer, at least one virtual reality device, or a combination thereof.

[0107] In another aspect of the invention, a tracking system for tracking the position of at least one movable object is disclosed. The tracking system includes at least one detector system according to one or more embodiments relating to detector systems as disclosed above and / or further detailed below. The tracking system further includes at least one tracking controller. The tracking controller is adapted to track a series of positions of the object at specific points in time.

[0108] In another aspect of the invention, a camera for imaging at least one object is disclosed. The camera includes at least one detector according to any embodiment involving detectors disclosed above or further detailed below.

[0109] In another aspect of the invention, a scanning system for determining the depth profile of a scene is provided, which may also imply determining at least one location of at least one object. The scanning system includes at least one detector according to the invention, such as those disclosed in one or more embodiments listed above and / or those disclosed in one or more embodiments below. The scanning system further includes at least one illumination source adapted to scan the scene with at least one beam, which may also be referred to as an illumination beam or a scanning beam. As used herein, the term "scene" generally refers to a two-dimensional or three-dimensional range visible to the detector, such that at least one geometric or spatial characteristic of the two-dimensional or three-dimensional range can be evaluated by the detector. As further used herein, the term "scan" generally refers to successive measurements in different regions. Thus, a scan may specifically imply at least one first measurement in which the illumination beam is oriented or directed in a first manner, and at least one second measurement in which the illumination beam is oriented or directed in a second manner different from the first. The scan may be a continuous scan or a stepwise scan. Thus, in a continuous or stepwise manner, the illumination beam can be directed to different regions of the scene, and the detector can be detected to generate at least one piece of information for each region, such as at least one ordinate. As an example, to scan an object, one or more illumination beams can continuously or progressively generate light spots on the surface of the object, where the ordinate of the generated light spots is [not specified]. Alternatively, however, light patterns can be used for scanning. Scanning can be point scanning or line scanning, or even scanning with more complex light patterns. The illumination source of the scanning system can be different from the optional illumination source of the detector. Alternatively, the illumination source of the scanning system can also be wholly or partially the same as or integrated into at least one optional illumination source of the detector.

[0110] Therefore, a scanning system may include at least one illumination source adapted to emit at least one light beam configured to illuminate at least one point located at at least one surface of at least one object. As used herein, the term "point" refers to an area on a portion of the object's surface, specifically a small area, which can be selected, for example, by a user of the scanning system, to be illuminated by the illumination source. Preferably, on the one hand, the point may exhibit the smallest possible size so as to allow the scanning system to determine a value of the distance between the illumination source included in the scanning system and the portion of the object's surface on which the point can be positioned as precisely as possible, and on the other hand, it may be as large as possible so as to allow the user of the scanning system or the scanning system itself, particularly by an automated procedure, to detect the presence of the point on the relevant portion of the object's surface.

[0111] For this purpose, the illumination source may include an artificial illumination source, particularly at least one laser source and / or at least one incandescent lamp and / or at least one semiconductor light source, such as at least one light-emitting diode, particularly organic and / or inorganic light-emitting diodes. For example, the light emitted by the illumination source may have a wavelength of 390 to 780 nm. Alternatively or additionally, light in the infrared spectral range (such as the range of 780 nm to 3.0 μm) may be used. Specifically, light in a portion of the near-infrared region may be used, in which silicon photodiodes are particularly suitable in the range of 700 nm to 1000 nm. Considering the beam profile and other operability characteristics they typically define, it is particularly preferred to use at least one laser source as the illumination source. Here, the use of a single laser source may be preferred, especially where it may be important to provide a compact scanning system that can be easily stored and transported by the user. Therefore, the illumination source may preferably be an integral part of the detector and thus may be particularly integrated into the detector, such as into the housing of the detector. In a preferred embodiment, in particular, the housing of the scanning system may include at least one display configured to provide distance-related information to the user in an easily readable manner. In another preferred embodiment, the housing of the scanning system may additionally include at least one button configured to operate at least one function associated with the scanning system, such as setting one or more operating modes. In another preferred embodiment, the housing of the scanning system may additionally include at least one fastening unit configured to secure the scanning system to another surface, such as a rubber foot, base plate, or wall retainer, such as a substrate or retainer comprising magnetic material, particularly for improving the accuracy of distance measurements and / or the user's operability of the scanning system.

[0112] Specifically, the illumination source of the scanning system can therefore emit a single laser beam, which can be configured to illuminate a single point located on the surface of the object. By using at least one detector according to the invention, at least one piece of information regarding the distance between at least one point and the scanning system can therefore be generated. Thus, preferably, the distance between the illumination system included by the scanning system and the single point generated by the illumination source can be determined, such as by employing an evaluation device included by at least one detector. However, the scanning system may further include an additional evaluation system that may be particularly suitable for this purpose. Alternatively or additionally, the size of the scanning system (especially the housing of the scanning system) can be taken into account, and thus the distance between a specific point on the housing of the scanning system (such as the front or rear edge of the housing) and the single point can be alternatively determined. The illumination source can be adapted to generate and / or project a point cloud, for example, the illumination source may include one or more of the following: at least one digital light processing (DLP) projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one light-emitting diode array; at least one laser source array.

[0113] Alternatively, the illumination source of the scanning system can emit two separate laser beams, which can be configured to provide a corresponding angle, such as a right angle, between the beam's emission direction, thereby illuminating two corresponding points located on the same object surface or on two different surfaces of two different objects. However, other values ​​for the corresponding angle between the two separate laser beams are also feasible. This feature can be particularly useful for indirect measurement functions, such as for deriving indirect distances, where direct access may be impossible due to one or more obstacles between the scanning system and the point, or otherwise, difficulty in reaching it. For example, it may be feasible to determine the object's height by measuring two separate distances and deriving the height using the Pythagorean theorem. In particular, to maintain a predetermined level relative to the object, the scanning system can further include at least one leveling unit, particularly an integrated bubble bottle, which can be used to maintain a user-defined level.

[0114] As a further alternative, the illumination source of the scanning system can emit multiple individual laser beams, such as a laser beam array, which can exhibit a corresponding pitch relative to each other, particularly a regular pitch, and can be arranged in a manner that generates a point array on at least one surface of at least one object. For this purpose, particularly suitable optical elements, such as beam splitters and mirrors, can be provided that allow for the generation of the laser beam array. In particular, the illumination source can be guided to scan an area or volume by using one or more movable mirrors to redirect the beam in a periodic or non-periodic manner.

[0115] Therefore, a scanning system can provide a static arrangement of one or more points on one or more surfaces of one or more objects. Alternatively, the illumination source of the scanning system, particularly one or more laser beams, such as the laser beam array described above, can be configured to provide one or more beams that may exhibit varying intensity over time and / or may undergo alternation of emission direction over time, particularly by moving one or more mirrors, such as micromirrors included in the micromirror array. As a result, the illumination source can be configured to scan a portion of at least one surface of at least one object as an image using one or more beams having alternating characteristics as generated by at least one illumination source of the scanning device. In particular, the scanning system can therefore use at least one row scan and / or line scan, such as sequentially or simultaneously scanning one or more surfaces of one or more objects. Thus, the scanning system can be adapted to measure angles by measuring three or more points, or the scanning system can be adapted to measure corners or narrow areas, such as the gable of a roof, which are difficult to access using conventional rulers. As a non-limiting example, the scanning system can be used in safety laser scanners (e.g., in production environments), and / or in 3D scanning devices for determining the shape of objects (such as in conjunction with 3D printing, body scanning, quality control), in construction applications (e.g., as a rangefinder), in logistics applications (e.g., for determining the size or volume of a package), in home applications (e.g., in robotic vacuum cleaners or lawnmowers), or in other types of applications that may include a scanning step. As a non-limiting example, the scanning system can be used in industrial safety curtain applications. As a non-limiting example, the scanning system can be used to perform sweeping, vacuuming, mopping, or waxing functions, or patio or garden care functions, such as mowing or raking. As a non-limiting example, the scanning system can employ an LED illumination source with collimating optics and can be adapted to shift the frequency of the illumination source to different frequencies to obtain more accurate results and / or employ filters to attenuate certain frequencies while transmitting other frequencies. As a non-limiting example, the scanning system and / or illumination source can rotate as a whole or use a dedicated motor to rotate only specific optical components, such as mirrors, beam splitters, etc., so that the scanning system can have a complete 360-degree view during operation, or can even be moved and / or rotated out of plane to further increase the scanning area. Furthermore, the illumination source can be actively aimed in a predetermined direction. Additionally, to allow rotation of the wired electrical system, slip rings, optical data transmission, or inductive coupling can be employed.

[0116] As a non-limiting example, the scanning system can be attached to a tripod and pointed at an object or area with multiple corners and surfaces. One or more flexible, movable laser sources are attached to the scanning system. The one or more laser sources are moved so that they illuminate points of interest. When a designated button on the scanning system is pressed, the position of the illuminated point relative to the scanning system is measured, and the position information is transmitted to a mobile phone via a wireless interface. The position information is stored in a mobile phone application. The laser sources are moved to illuminate other points of interest, the position of which is measured and transmitted to the mobile phone application. The mobile phone application can transform the set of points into a 3D model by connecting adjacent points to planar surfaces. The 3D model can be stored and further processed. The distances and / or angles between the measured points or surfaces can be displayed directly on a display attached to the scanning system or on the mobile phone to which the position information is transmitted.

[0117] As a non-limiting example, the scanning system may include two or more flexible, movable laser sources for projection points and other movable laser sources for projection lines. Lines may be used to arrange two or more laser spots along a line, and the display of the scanning device may show the distance between the two or more laser spots that can be arranged along a line (such as equidistantly). In the case of two laser spots, a single laser source may be used, while one or more beam splitters or prisms are used to modify the distance of the projection point, wherein the beam splitter or prism can be moved so that the projected laser spots are separated or brought closer together. Furthermore, the scanning system may be adapted to project other patterns, such as right angles, circles, squares, triangles, etc., along which measurements can be performed by projecting laser spots and measuring their positions.

[0118] As a non-limiting example, the scanning system can be adapted to be supported by tools (such as woodworking or metalworking tools, such as saws, drills, etc.). Therefore, the scanning system can be adapted to measure distances in two opposite directions and display the two measured distances or the sum of the distances on a display. Furthermore, the scanning system can be adapted to measure distances to the edges of a surface, such that when the scanning system is placed on the surface, the laser point automatically moves away from the scanning system along the surface until the distance measurement shows an abrupt change due to a corner or edge of the surface. This allows for measuring the distance to the end of a plank when the scanning device is placed on the plank but away from the end of the plank. Furthermore, the scanning system can measure the distance to the end of the plank in one direction and project a line, circle, or point within a specified distance in the opposite direction. The scanning system can be adapted to project a line, circle, or point within a certain distance, depending on the distance measured in the opposite direction, such as depending on a predetermined total distance. This allows work to be done with a tool (such as a saw or drill) at the projection location, while the scanning system is placed at a safe distance from the tool, and the process is performed simultaneously using the tool at a predetermined distance from the edge of the plank. Furthermore, the scanning system can be adapted to project points or lines, etc., in two opposite directions at a predetermined distance. When the total distance changes, only one of the projected distances changes.

[0119] As a non-limiting example, the scanning system may be adapted to be placed on a surface, such as a surface on which tasks such as cutting, sawing, drilling, etc., are performed, and to project lines onto the surface at a predetermined distance, which may be adjusted, for example, by using a button on the scanning device.

[0120] As a non-limiting example, the scanning system can be used in security laser scanners (e.g., in production environments), and / or in 3D scanning devices used to determine the shape of objects (such as in conjunction with 3D printing, body scanning, quality control), in construction applications (e.g., as a rangefinder), in logistics applications (e.g., for determining the size or volume of a package), in home applications (e.g., in robotic vacuum cleaners or lawnmowers), or in other types of applications that may include a scanning step.

[0121] As outlined above, the transmission device can be designed to preferably sequentially feed light propagating from the object to the detector to the optical sensor. As outlined above, this feeding can be optionally achieved by means of imaging or additionally by means of the non-imaging characteristics of the transmission device. In particular, the transmission device can also be designed to collect electromagnetic radiation before it is subsequently fed to the optical sensor. The transmission device can also be wholly or partially part of at least one optional illumination source, for example, by designing the illumination source to provide a beam with defined optical characteristics, such as having a defined or precisely known beam profile, such as at least one linear combination of Gaussian beams, particularly at least one laser beam with a known beam profile.

[0122] For potential embodiments with optional illumination sources, reference can be made to WO 2012 / 110924 A1. Nevertheless, other embodiments are also possible. Light emitted from the object may originate from the object itself, but may also optionally have a different origin, propagating from that origin to the object and subsequently toward the lateral and / or longitudinal optical sensors. The latter can be achieved, for example, by using at least one illumination source. This illumination source may be, for example, or include an ambient illumination source and / or may be or may include an artificial illumination source. For example, the detector itself may include at least one illumination source, such as at least one laser and / or at least one incandescent lamp and / or at least one semiconductor illumination source, such as at least one light-emitting diode, particularly organic and / or inorganic light-emitting diodes. Given their typically defined beam profiles and other operability characteristics, the use of one or more lasers as an illumination source or part thereof is particularly preferred. The illumination source itself may be a component of the detector or formed separately from the detector. The illumination source may be particularly integrated into the detector, for example, integrated into the detector housing. Alternatively or additionally, at least one illumination source may also be integrated into at least one beacon device or into one or more beacon devices and / or integrated into an object or connected to or spatially coupled to an object.

[0123] Light emitted from one or more optional beacon devices can be emitted from and / or excited by an illumination source, correspondingly, alternatively, or additionally, in the option where the light originates from the respective beacon device itself. For example, electromagnetic light emitted from a beacon device can be emitted by and / or reflected by the beacon device itself and / or scattered by the beacon device before being fed to a detector. In this case, the emission and / or scattering of electromagnetic radiation can be achieved without or with the spectral influence of electromagnetic radiation. Thus, for example, according to Stokes or Raman, wavelength shifts may also occur during scattering. Furthermore, light emission can be excited, for example, by a primary illumination source (e.g., by an object or a portion of an object excited to produce luminescence (partially phosphorescence and / or fluorescence). In principle, other emission processes are also possible. If reflection occurs, the object can have, for example, at least one reflective area, particularly at least one reflective surface. The reflective surface can be part of the object itself, but can also be a reflector connected or spatially coupled to the object, such as a reflector plate connected to the object. If at least one reflector is used, it can then be considered part of the detector, which is connected to the object, for example, independently of the other components of the detector.

[0124] For information on the design and characteristics of beacon equipment, refer to the description of the irradiation source above or below for a more detailed description.

[0125] Feeding the light beam to the optical sensor can be achieved by generating a light spot on a selectable sensor area of ​​the optical sensor, for example, the light spot having a circular, elliptical, or differently configured cross-section. For example, the detector can have a field of view, particularly a solid angle range and / or a spatial range, within which an object can be detected. Preferably, the delivery device can be designed such that the light spot is completely positioned on and / or on the sensor area of ​​the optical sensor, for example, when the object is positioned within the detector's field of view. For example, a sensor area of ​​a corresponding size can be selected to ensure this condition.

[0126] In another aspect, the present invention discloses an inertial measurement unit (IMU) for use in an electronic device. As used herein, the term "inertial measurement unit" refers to a system comprising at least two detector units and configured to determine linear and angular motion. The electronic device may be a mobile electronic device. The electronic device may be a camera. The electronic device may be a mobile phone. The IMU is adapted to receive data determined by at least one detector according to the present invention (such as according to one or more embodiments involving detectors as disclosed above or further detailed below). As used herein, the term "data determined by at least one detector" refers to at least one piece of information regarding a relative spatial constellation and / or at least one ordinate z. The IMU is further adapted to receive data determined by at least one other sensor selected from the group consisting of: wheel speed sensor, turning rate sensor, tilt sensor, orientation sensor, motion sensor, magnetohydrodynamic sensor, force sensor, angle sensor, angular rate sensor, magnetic field sensor, magnetometer, accelerometer, gyroscope. As used herein, the term "data determined by at least one other sensor" refers to at least one piece of information selected from the group consisting of: angle information; speed information; information regarding turning rate; information regarding tilt. An inertial measurement unit (IMU) is adapted to determine at least one characteristic of an electronic device by evaluating data from a detector and at least one other sensor, the characteristic being selected from the group consisting of: position in space, relative or absolute motion in space, rotation, acceleration, orientation, angular position, tilt, turning rate, and velocity. The IMU may include at least one processor. The processor may be adapted to evaluate data recorded by the other sensors. In particular, the processor may be adapted to determine and / or calculate one or more of spatial position, spatial orientation, motion, and velocity. The IMU may include multiple other sensors. The IMU may be adapted to fuse information determined from at least two of the other sensors. The IMU may be adapted to fuse information from at least two other sensors using at least one mathematical model, the mathematical model being selectable from Kalman filters, linear quadratic estimators, Kalman-Busch filters, Stratonovich-Kalman-Busch filters, Kalman-Busch-Stratonovich filters, minimum variance estimators, Bayesian estimators, optimal linear unbiased estimators, invariant estimators, Wiener filters, etc. As summarized above, the detector may be adapted to provide an absolute measurement of the ordinate z. Furthermore, as outlined above, the detector can be adapted to determine and / or calibrate and / or recalibrate the relative spatial constellation between at least two sensor elements. The processor (e.g., the evaluation device described above) can be adapted to fuse information from at least two other sensors and / or recalibrate the relative spatial constellation, taking into account the ordinate z-coordinate and / or the determined relative spatial constellation.Various sensor signals can be used within the mathematical model, which can be selected from Kalman filters, linear quadratic estimators, Kalman-Busch filters, Stratonovich-Kalman-Busch filters, Kalman-Busch-Stratonovich filters, minimum variance estimators, Bayesian estimators, optimal linear unbiased estimators, invariant estimators, Wiener filters, etc., to account for the measurement errors and inaccuracies that each sensor signal will experience. For example, for measurements of relative spatial constellations and / or ordinates, the fusion of these sensor signals within one of the aforementioned mathematical models (such as a Kalman filter) can produce improved estimates.

[0127] On the other hand, the present invention discloses a method for determining a relative space constellation using a detector (such as one or more detectors according to the present invention, such as those relating to detectors disclosed above or further detailed below). Other types of detectors may also be used. The method includes method steps, wherein the method steps may be performed in a given order or in a different order. Furthermore, one or more additional method steps not listed may be present. Additionally, one, more than one, or even all of the method steps may be repeated.

[0128] This method includes the following steps:

[0129] - Determine at least one reflected image of an object by using at least one sensor element having an optical sensor matrix, each optical sensor having a photosensitive area;

[0130] - Select at least one reflection feature of the reflected image at at least one first image location in the reflected image, and optimize at least one blur function f a To determine at least one ordinate z of the selected reflection feature;

[0131] - Provide at least one reference image, wherein the reference image and the reflection image are determined by two different spatial configurations, wherein the spatial configurations differ in that they are relative to a spatial constellation;

[0132] - Determine at least one reference feature in the reference image at at least one second image location corresponding to the ordinate z in the reference image;

[0133] - Determine the relative spatial constellation based on the vertical coordinate z and the positions of the first and second images.

[0134] For details, options, and definitions, please refer to the detectors discussed above. Therefore, specifically, as outlined above, the method may include the use of a detector according to the invention (such as according to one or more embodiments given above or further detailed below).

[0135] As used herein, the term "provide at least one reference image" means, for example, determining at least one reference image from a plurality of stored reference images, recording at least one reference image, selecting one or more of at least one reference image.

[0136] This method may include monitoring relative spatial constellations. The relative spatial constellations can be repeatedly determined.

[0137] The method may include at least one temperature determination step. In the temperature determination step, at least one temperature value of the detector may be determined. The temperature value may be taken into account to determine the relative spatial constellation, and / or the relative spatial constellation may be adapted based on the temperature value.

[0138] The detector may include at least one illumination source. This method can be used to determine the relative positions of the sensor element and the illumination source.

[0139] The detector may include at least one first sensor element and at least one second sensor element. The first sensor element and at least one second sensor element may be in different spatial configurations. The method may include selecting at least one image determined by the first sensor element or the second sensor element as a reflected image and selecting at least one image determined by the other of the first sensor element or the second sensor element as a reference image. The method may be used to determine the relative spatial constellation of the first sensor element and the second sensor element.

[0140] As outlined above, the method may include determining at least one ordinate z by considering the ordinate z and the relative spatial constellation. triang The steps.

[0141] On the other hand, the present invention discloses a method for calibrating at least one detector (such as a detector according to the present invention, such as according to one or more embodiments involving detectors as disclosed above or further detailed below). Other types of detectors may still be used. The method includes method steps, wherein the method steps may be performed in a given order or may be performed in a different order. In addition, there may be one or more additional method steps not listed. Furthermore, one, more than one, or even all of the method steps may be repeated.

[0142] This method includes the following steps:

[0143] - Determine at least one reflected image of an object using at least one sensor element of a matrix having optical sensors, each optical sensor having a photosensitive area;

[0144] - Select at least one reflection feature of the reflected image at at least one first image location in the reflected image, and optimize at least one blur function fa To determine at least one ordinate z of the selected reflection feature;

[0145] - Provide at least one reference image, wherein the reference image and the reflection image are determined by two different spatial configurations, wherein the spatial configurations differ in that they are relative to a spatial constellation;

[0146] - Determine at least one reference feature in the reference image at at least one second image location corresponding to the ordinate z in the reference image;

[0147] - Determine the relative spatial constellation based on the vertical coordinate z and the positions of the first and second images.

[0148] - The relative space constellation is stored as a calibration value in at least one data storage device of at least one evaluation unit.

[0149] This method may include determining the displacements of a reference feature and a reflection feature, and determining the relationship between the object's ordinate and the displacement. As outlined above, this method may include determining at least one ordinate z considering the ordinate z. triang The steps.

[0150] The method may further include storing the relative space constellation in at least one storage unit, such as an evaluation device.

[0151] The method may include a recalibration step, which includes the following steps:

[0152] - Determine at least one other reflected image of the object by using at least one sensor element;

[0153] - Select at least one other reflection feature of another reflection image at at least one third image location in the reflection image, and optimize at least one blur function f a To determine at least one second ordinate z of other reflection features;

[0154] - Provide at least one other reference image, wherein the other reference image and the other reflection image are determined by two different spatial configurations, wherein the difference in spatial configurations lies in the actual relative spatial constellations;

[0155] - Determine at least one other reference feature in the other reference image at at least one fourth image position corresponding to the second ordinate z;

[0156] - Determine the actual relative spatial constellation based on the second vertical coordinate z and the positions in the third and fourth images.

[0157] - Compare the first relative space constellation and the actual relative space constellation, and adjust the first relative space constellation according to the actual relative space constellation.

[0158] The method may further include at least one temperature determination step, wherein at least one temperature value of the detector is determined. The temperature value may be considered in determining the first relative space constellation and / or the actual relative space constellation, and / or the first relative space constellation and / or the actual relative space constellation may be adjusted based on the temperature value.

[0159] In another aspect of the invention, for purposes of use, uses of the detector according to the invention (such as according to one or more embodiments given above or further detailed below) are proposed, the uses being selected from the group consisting of: position measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photographic applications; imaging applications or camera applications; mapping applications for generating maps of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotic applications; quality control applications; and manufacturing applications.

[0160] The object can typically be a living or inanimate object. The detector or detector system may even include at least one object that forms part of the detector system. However, preferably, the object can move independently of the detector in at least one spatial dimension. The object can typically be any object. In one embodiment, the object can be a rigid object. Other embodiments are feasible, such as those in which the object is a non-rigid object or an object that can change its shape.

[0161] As will be summarized in more detail below, the present invention can be specifically used for tracking the location and / or movement of a person, such as for controlling machines, games, or sports simulations. In this or other embodiments, specifically, the object may be selected from the group consisting of: sports equipment articles, preferably selected from articles including rackets, cues, and clubs; clothing; hats; and shoes.

[0162] Therefore, the device (such as a detector) according to the invention can generally be applied to a wide range of applications. Specifically, the detector can be applied to purposes selected from the group consisting of: location measurement in transportation technology; entertainment applications; security applications; human-machine interface applications; tracking applications; photography applications; mapping applications for generating maps of at least one space (such as at least one space selected from the group consisting of rooms, buildings, and streets); mobile applications; webcams; audio equipment; Dolby surround sound systems; computer peripherals; gaming applications; camera or video applications; security applications; surveillance applications; automotive applications; transportation applications; medical applications; sports applications; machine vision applications; vehicle applications; aircraft applications; marine applications; spacecraft applications; construction applications; mapping applications; and manufacturing applications. Additionally or alternatively, applications in local and / or global positioning systems, particularly landmark-based positioning and / or navigation, can be specified, specifically for automobiles or other vehicles (such as trains, motorcycles, bicycles, trucks for freight transport), robots, or for pedestrians. Furthermore, indoor positioning systems can be specified as potential applications, such as for home applications and / or for robots used in manufacturing, logistics, monitoring, or maintenance technologies.

[0163] The device according to the invention can be used in mobile phones, tablet computers, laptop computers, smart panels, or other fixed or mobile or wearable computer or communication applications. Therefore, the device according to the invention can be combined with at least one active light source, such as a light source emitting light in the visible or infrared spectral range, to improve performance. Thus, as an example, the device according to the invention can be used as a camera and / or sensor, for example, in combination with mobile software for scanning and / or detecting the environment, objects, and organisms. The device according to the invention can even be combined with a 2D camera (such as a conventional camera) to enhance imaging effects. The device according to the invention can be further used for monitoring and / or recording purposes or as an input device for controlling mobile devices, particularly in combination with voice and / or gesture recognition. Therefore, specifically, the device according to the invention used as a human-machine interface (also called an input device) can be used in mobile applications, such as for controlling other electronic devices or components via mobile devices (such as mobile phones). As an example, a mobile application including at least one device according to the invention can be used to control a television, game console, music player, or other music device or entertainment device.

[0164] Furthermore, the device according to the invention can be used with webcams or other peripheral devices for computing applications. Thus, as an example, the device according to the invention can be used in conjunction with software for imaging, recording, monitoring, scanning, or motion detection. As outlined in the context of human-computer interfaces and / or entertainment devices, the device according to the invention is particularly useful for giving commands through facial expressions and / or body expressions. The device according to the invention can be combined with other input generation devices (e.g., mouse, keyboard, touchpad, microphone, etc.). Furthermore, the device according to the invention can be used in gaming applications, such as by using a webcam. Furthermore, the device according to the invention can be used in virtual training applications and / or video conferencing. Furthermore, the device according to the invention can be used to identify or track hands, arms, or objects used in virtual or augmented reality applications, particularly when wearing a head-mounted display.

[0165] Furthermore, as explained in part above, the device according to the invention can be used in mobile audio devices, television devices, and gaming devices. Specifically, the device according to the invention can be used as a controller or control device for electronic devices, entertainment devices, etc. Additionally, the device according to the invention can be used for eye detection or eye tracking, such as in 2D and 3D display technologies (particularly with transparent displays for augmented reality applications), and / or for identifying whether a display is being viewed and / or from what angle the display is being viewed. Furthermore, the device according to the invention can be used in conjunction with virtual or augmented reality applications to explore rooms, boundaries, and obstacles, particularly when wearing a head-mounted display.

[0166] Furthermore, the device according to the invention can be used or used as a digital camera (such as a DSC camera) and / or used or used as a reflective camera (such as an SLR camera). For these applications, as described above, reference can be made to the use of the device according to the invention in mobile applications such as mobile phones.

[0167] Furthermore, the device according to the invention can be used in security or surveillance applications. Thus, as an example, at least one device according to the invention can be combined with one or more digital and / or analog electronic devices that will signal if an object is within or outside a predetermined area (e.g., for surveillance applications in banks or museums). Specifically, the device according to the invention can be used for optical encryption. Detection using at least one device according to the invention can be combined with other detection devices (such as with IR, X-ray, UV-VIS, radar, or ultrasonic detectors) to supplement the wavelength. The device according to the invention can be further combined with an active infrared light source to allow detection in low-light environments. The device according to the invention is generally advantageous compared to active detector systems, especially because it avoids actively transmitting signals that could be detected by third parties, as is the case, for example, in radar applications, ultrasonic applications, LIDAR, or similar active detector devices. Therefore, the device according to the invention can generally be used for tracking moving objects that are unidentifiable and undetectable. Furthermore, the device according to the invention is generally less susceptible to manipulation and stimulation compared to conventional devices.

[0168] Furthermore, considering the simplicity and accuracy of 3D detection using the device according to the invention, the device according to the invention can generally be used for facial, body, and human identification and identification. The device according to the invention can be combined with other detection methods (such as passwords, fingerprints, iris detection, voice recognition, or other methods) for identification or personalization purposes. Therefore, the device according to the invention is generally used in security devices and other personalized applications.

[0169] Furthermore, the device according to the invention can be used as a 3D barcode reader for product identification.

[0170] In addition to the aforementioned security and surveillance applications, the device according to the invention can generally be used for the monitoring and surveillance of spaces and areas. Therefore, the device according to the invention can be used to monitor and surveillance spaces and areas, and, as an example, to trigger or execute alarms in the event of a violation of a prohibited area. Thus, the device according to the invention can generally be used for surveillance purposes in buildings or museums, optionally in conjunction with other types of sensors, such as motion or thermal sensors, image intensifiers or image enhancement devices, and / or photomultiplier tubes. Furthermore, the device according to the invention can be used in public or crowded spaces to detect potentially dangerous activities, such as criminal acts, theft of objects such as those in parking lots or unattended objects (such as unattended luggage in airports).

[0171] Furthermore, the device according to the invention can be advantageously applied to camera applications such as video and camcorder applications. Therefore, the device according to the invention can be used for motion capture and 3D movie recording. The device according to the invention generally offers numerous advantages over conventional optical devices. Therefore, the device according to the invention generally requires lower complexity regarding optical components. Thus, as an example, the number of lenses can be reduced compared to conventional optical devices, such as by providing a device according to the invention with only one lens. Due to the reduced complexity, very compact devices are possible, such as for mobile use. Conventional optical systems with two or more high-quality lenses are typically bulky due to the widespread need for numerous beam splitters. Furthermore, the device according to the invention can generally be used in focusing / autofocusing devices, such as autofocus cameras. Additionally, the device according to the invention can also be used in optical microscopes, particularly confocal microscopes.

[0172] Furthermore, the device according to the invention is generally applicable to the fields of automotive technology and transportation technology. Thus, as an example, the device according to the invention can be used as a distance and monitoring sensor, such as for adaptive cruise control, emergency braking assist, lane departure warning, surround view, blind spot detection, traffic sign detection, traffic sign recognition, lane recognition, rear cross-traffic alert, light source recognition that adjusts headlight intensity and range based on approaching traffic or vehicles ahead, adaptive headlight systems, automatic high beam control, adaptive cut-off lights in headlight systems, glare-free high beam headlight systems, marking animals, obstacles, etc., with headlights, rear cross-traffic alert, and other driver assistance systems, such as advanced driver assistance systems or other automotive and transportation applications. Furthermore, the device according to the invention can be used in driver assistance systems that predict driver maneuvers in advance to avoid collisions, etc. Additionally, the device according to the invention can also be used for speed and / or acceleration measurement, such as by analyzing the first and second time derivatives of position information obtained using the detector according to the invention. This feature is generally applicable to automotive technology, transportation technology, or general transportation technology. Applications in other technical fields are also possible. Specific applications of indoor positioning systems include detecting the location of passengers during transport, and more specifically, electronically controlling the use of safety systems such as airbags. If a passenger is in a position where the use of an airbag would cause serious injury, its use may be prevented. Furthermore, in vehicles such as cars, trains, and airplanes, particularly in autonomous vehicles, the device according to the invention can be used to determine whether the driver is paying attention to traffic or is distracted, asleep, fatigued, or unable to drive due to factors such as alcohol consumption.

[0173] In these or other applications, the device according to the invention can generally be used as a standalone device or in combination with other sensor devices (such as radar and / or ultrasonic devices). Specifically, the device according to the invention can be used for autonomous driving and safety issues. Furthermore, in these applications, the device according to the invention can be used in conjunction with infrared sensors, radar sensors, or other types of sensors such as acoustic sensors, two-dimensional cameras, or other sensors. In these applications, the generally passive nature of the device according to the invention is advantageous. Therefore, since the device according to the invention generally does not need to transmit signals, the risk of interference between active sensor signals and other signal sources can be avoided. Specifically, the device according to the invention can be used in conjunction with recognition software (such as standard image recognition software). Therefore, the signals and data provided by the device according to the invention are generally easy to process and therefore generally require less computational power than a conventional 3D measurement system. Considering low space requirements, the device according to the invention (such as a camera) can be placed virtually anywhere in the vehicle, such as on or behind the window screen, on the hood, on the bumper, on the lights, on the rearview mirror, or other places. Various detectors according to the invention can be combined, such as one or more detectors based on the effects disclosed within the invention, for example, to allow autonomous vehicles or to improve the performance of active safety concepts. Therefore, the various devices according to the invention can be combined with one or more other devices according to the invention and / or conventional sensors (such as in a rear window, side window or front window, on a bumper or on a light).

[0174] Combinations of at least one device according to the invention (such as at least one detector according to the invention) with one or more rain detection sensors are also possible. This is due to the fact that devices according to the invention are generally superior to conventional sensor technologies (such as radar), especially during heavy rain. Combining at least one device according to the invention with at least one conventional sensing technology (such as radar) allows software to select the correct signal combination based on weather conditions.

[0175] Furthermore, the device according to the invention can generally be used as a braking assist and / or parking assist and / or for speed measurement. Speed ​​measurement can be integrated into the vehicle or used externally, such as for measuring the speed of other vehicles in traffic control. Additionally, the device according to the invention can be used to detect vacant parking spaces in a parking lot.

[0176] Furthermore, the device according to the invention can be used in medical systems and sports. Therefore, in the field of medical technology, surgical robots (e.g., for endoscopes) can be specified because, as mentioned above, the device according to the invention may only require a small size and can be integrated into other devices. Specifically, the device according to the invention, having at most one lens, can be used to capture 3D information in medical devices such as endoscopes. Furthermore, the device according to the invention can be combined with appropriate monitoring software to enable the tracking and analysis of motion. This allows for the instantaneous overlay of the position of medical devices (such as endoscopes or scalpels) with the results of medical imaging (such as those obtained from magnetic resonance imaging, X-ray imaging, or ultrasound imaging). These applications are particularly valuable, for example, in medical treatments where precise location information is important (such as neurosurgery, remote diagnostics, and telemedicine). Additionally, the device according to the invention can be used for 3D body scanning. Body scanning can be applied in medical contexts such as dental surgery, orthopedic surgery, weight loss surgery, or cosmetic surgery, or in medical diagnostic contexts such as diagnosing myofascial pain syndromes, cancer, body dysmorphic disorders, or further diseases. Body scanning can be further applied in the field of sports to evaluate the ergonomic use or fit of sports equipment. Furthermore, the device according to the invention can be used in wearable robots, such as exoskeletons or prostheses.

[0177] Body scanning can also be used in the context of clothing, such as determining the proper size and fit of garments. This technology can be used in the context of custom-made clothing, or in the context of ordering clothes or shoes online, or in the context of self-service shopping devices such as mini-kiosks or customer concierge systems. Body scanning in the context of clothing is particularly important for scanning well-dressed customers.

[0178] Furthermore, the device according to the invention can be used in the context of people counting systems, such as counting the number of people in elevators, trains, buses, cars, or airplanes, or counting the number of people passing through corridors, doors, passageways, retail stores, stadiums, entertainment venues, museums, libraries, public places, cinemas, theaters, etc. In addition, the 3D functionality in the people counting system can be used to obtain or estimate further information about the people being counted, such as height, weight, age, and physical health. This information can be used for business intelligence metrics and / or to further optimize locations where people can be counted to make them more attractive or safer. In a retail environment, the device according to the invention, in the context of people counting, can be used to identify repeat customers or cross-shoppers, assess shopping behavior, evaluate the percentage of visitors making purchases, optimize employee shifts, or monitor shopping mall costs per visitor. Furthermore, the people counting system can be used for anthropometry surveys. Furthermore, the device according to the invention can be used in public transportation systems for automatically charging passengers based on transport length. Furthermore, the device according to the invention can be used in children's playgrounds to identify injured children or children engaged in dangerous activities, allowing additional interaction with playground toys to ensure safe use of playground toys, etc.

[0179] Furthermore, the device according to the invention can be used in construction tools, such as rangefinders for determining distances to objects or walls, for assessing whether a surface is planar, for aligning or placing objects in an orderly manner, or for use in inspection cameras in built environments, etc.

[0180] Furthermore, the device according to the invention can be applied to the fields of sports and exercise, such as for training, remote instruction, or competition purposes. Specifically, the device according to the invention can be applied to dance, aerobics, football, English football, basketball, baseball, cricket, hockey, athletics, swimming, polo, handball, volleyball, rugby, sumo, judo, fencing, boxing, golf, racing, laser tagging, battlefield simulation, etc. The device according to the invention can be used to detect the position of a ball, bat, sword, movement, etc., whether in sports or competitions, such as monitoring the game, supporting referees or making judgments, and especially automatically judging specific situations in sports, such as determining whether a score or goal has truly been scored.

[0181] Furthermore, the device according to the present invention can be used in fields such as racing car or automobile driver training or automobile safety training to determine the position of the car or the trajectory of the car, or the deviation from the previous trajectory or the ideal trajectory, etc.

[0182] The device according to the invention can be further used to support the practice of musical instruments, particularly distance learning courses, such as lessons on stringed instruments (e.g., fiddle, violin, viola, cello, bass, harp, guitar, banjo, or ukulele), keyboard instruments (e.g., piano, organ, keyboard, harpsichord, organ, or accordion), and / or percussion instruments (e.g., drums, timpani, marimba, xylophone, vibraphone, bongo, conga, timpani, African drum, or hand drum).

[0183] The device according to the invention can also be used in rehabilitation and physical therapy to encourage training and / or to investigate and correct movement. Furthermore, the device according to the invention can also be applied to distance diagnosis.

[0184] Furthermore, the device according to the invention can be applied to the field of machine vision. Therefore, one or more devices according to the invention can be used, for example, as passive control units for autonomous driving and / or robot operation. Combined with mobile robots, the device according to the invention can allow autonomous movement and / or autonomous detection of faults in parts. The device according to the invention can also be used for manufacturing and safety monitoring, such as to avoid accidents, including but not limited to collisions between robots, production parts, and living organisms. In robotics, safe and direct interaction between humans and robots is often a problem because robots can seriously injure humans when undetected. The device according to the invention can help robots locate objects and humans better and faster, and allow safe interaction. Considering the passive nature of the device according to the invention, it may be superior to active devices and / or can be used to complement existing solutions such as radar, ultrasound, 2D cameras, infrared detection, etc. A particular advantage of the device according to the invention is the low probability of signal interference. Therefore, multiple sensors can operate simultaneously in the same environment without the risk of signal interference. Therefore, the device according to the invention is generally suitable for highly automated production environments, such as, but not limited to, automotive, mining, steel, etc. The device according to the invention can also be used for quality control in production, for example, in combination with other sensors such as 2D imaging, radar, ultrasound, infrared, etc., for quality control or other purposes. Furthermore, the device according to the invention can be used to assess surface quality, such as for monitoring the surface flatness of products or compliance with specific dimensions (from micrometers to meters). Other quality control applications are possible. In manufacturing environments, the device according to the invention is particularly suitable for processing natural products, such as food or wood, with complex three-dimensional structures to avoid large amounts of waste. Additionally, the device according to the invention can be used to monitor the filling levels of cans, silos, etc. Furthermore, the device according to the invention can be used to inspect missing parts, incomplete parts, loose parts, low-quality parts, etc., in complex products, such as in automated optical inspections of printed circuit boards, components or sub-components, verification of engineered components, engine parts inspection, wood quality inspection, label inspection, medical device inspection, product orientation inspection, packaging inspection, food packaging inspection, etc.

[0185] Furthermore, the device according to the invention can be used in vehicles, trains, airplanes, ships, spacecraft, and other transportation applications. Therefore, in addition to the applications mentioned above in the context of transportation applications, passive tracking systems for aircraft, vehicles, etc., can also be specified. The use of at least one device according to the invention (such as at least one detector according to the invention) for monitoring the speed and / or direction of moving objects is feasible. Specifically, tracking of fast-moving objects on land, sea, and in the air (including space) can be specified. At least one device according to the invention (such as at least one detector according to the invention) can be specifically mounted on stationary and / or moving devices. The output signal of at least one device according to the invention can be combined, for example, with a guidance mechanism for autonomous or guided motion of another object. Therefore, applications for avoiding collisions between tracked and manipulated objects or for causing collisions between tracked and manipulated objects are feasible. The device according to the invention is generally useful and advantageous due to the low computational power required, instantaneous response, and the passive nature of detection systems that are generally more difficult to detect and interfere with compared to active systems (such as radar). The device according to the invention is particularly useful, but not limited to, speed control and air traffic control devices. Furthermore, the device according to the invention can be used in automatic toll collection systems for road toll collection.

[0186] The device according to the invention can generally be used in passive applications. Passive applications include guidance for ships in ports or hazardous areas, and for aircraft during landing or startup. Fixed, known active targets can be used for precise guidance. This is equally applicable to vehicles traveling on dangerous but well-defined routes, such as mining vehicles. Furthermore, the device according to the invention can be used to detect rapidly approaching objects, such as cars, trains, flying objects, animals, etc. Additionally, the device according to the invention can be used to detect the velocity or acceleration of an object, or to predict the motion of an object by tracking one or more of its position, velocity, and / or acceleration over time.

[0187] Furthermore, as outlined above, the device according to the invention can be used in the gaming field. Therefore, the device according to the invention can be passive and used with multiple objects having the same or different sizes, colors, shapes, etc., such as in conjunction with software that incorporates motion into its content for motion detection. In particular, application in realizing bit-graphic output of motion is feasible. Furthermore, applications of the device according to the invention for issuing commands are feasible, such as by using one or more devices according to the invention for gesture or facial recognition. The device according to the invention can be combined with active systems to operate, for example, in low-light conditions or in other situations where enhanced ambient conditions are required. Additionally or alternatively, combinations of one or more devices according to the invention with one or more IR or VIS light sources are possible. Combinations of detectors according to the invention with special devices are also possible, which can be easily distinguished by the system and its software, for example, but not limited to, special colors, shapes, relative positions to other devices, movement speed, light, frequency used to modulate the light source on the device, surface characteristics, materials used, reflectivity, transparency, absorption characteristics, etc. Among other possibilities, the device can resemble a stick, racket, cue stick, gun, knife, wheel, ring, steering wheel, bottle, ball, glass, vase, spoon, fork, cube, dice, doll, puppet, teddy bear, beaker, pedal, switch, glove, jewelry, musical instrument, or an accessory for playing a musical instrument, such as a pick or drumstick. Other options are also possible.

[0188] Furthermore, the device according to the invention can be used to detect and / or track objects that emit light themselves, such as due to high temperatures or further light-emitting processes. The light-emitting part can be an exhaust stream, etc. Additionally, the device according to the invention can be used to track reflective objects and analyze the rotation or orientation of these objects.

[0189] Furthermore, the devices according to the invention are generally applicable to the fields of architecture, construction, and cartography. Therefore, one or more devices according to the invention can typically be used to measure and / or monitor environmental areas, such as rural areas or buildings. These devices can be combined with other methods and devices or used alone to monitor the progress and accuracy of construction projects, altered objects, houses, etc. The devices according to the invention can be used to generate three-dimensional models of scanned environments to construct maps of rooms, streets, houses, communities, or landscapes from the ground or air. Potential applications include architecture, cartography, real estate management, land surveying, etc. As an example, the devices according to the invention can be used with drones or multi-rotor aircraft to monitor buildings, production sites, chimneys, agricultural production environments such as fields, factories, or landscapes to support rescue operations, support work in hazardous environments, support fire brigades at indoor or outdoor fire sites, or locate or monitor one or more people or animals, or for recreational purposes, such as drones following and recording activities performed by one or more people (e.g., skiing or cycling), which can be achieved through following helmets, markers, beacon devices, etc. The devices according to the invention can be used to identify obstacles, follow predetermined routes, follow edges, pipes, buildings, etc., or record global or local maps of the environment. Furthermore, the device according to the invention can be used for indoor or outdoor positioning and location determination of drones, for stabilizing the altitude of drones indoors where barometric pressure sensors are not accurate enough, or for interaction of multiple drones, such as for coordinated movement of several drones or for charging or refueling in the air.

[0190] Furthermore, the device according to the invention can be used in interconnected networks of home appliances, such as CHAIN ​​(Cedec Home Appliance Interoperability Network), to interconnect, automate, and control essential appliance-related services in the home, such as energy or load management, remote diagnostics, pet-related appliances, child-related appliances, child monitoring, appliance-related monitoring, support or services for the elderly or sick, home security and / or monitoring, remote control of appliance operation, and automated maintenance support. Additionally, the device according to the invention can be used in heating or cooling systems, such as air conditioning systems, to determine which part of a room should be set to a certain temperature or humidity, particularly depending on the location of one or more people. Furthermore, the device according to the invention can be used in home robots, such as service robots for household chores or autonomous robots. The device according to the invention can be used for many different purposes, such as to avoid collisions or mapping the environment, to identify users, to personalize the robot's performance for a given user, for safety purposes, or for gesture or facial recognition. As examples, the device according to the invention can be used in robotic vacuum cleaners, floor cleaning robots, dry cleaning robots, ironing robots for ironing clothes, animal waste robots such as dog or cat waste robots, charging robots for electric vehicles, security robots for detecting intruders, robotic lawnmowers, automatic pool cleaners, gutter cleaning robots, robotic shopping carts, luggage handling robots, line-following robots, laundry robots, ironing robots, window-cleaning robots, toy robots, patient monitoring robots, baby monitoring robots, elderly monitoring robots, child monitoring robots, transport robots, telepresence robots, professional service robots, programmable toy robots, pathfinding robots, social robots providing companionship for people with limited mobility, following robots, smart card following robots, psychotherapy robots, or robots that translate speech into sign language or sign language into speech. In the case of people with limited mobility (such as the elderly), a home robot with the device according to the invention can be used to pick up objects, transport objects, and interact with objects and users in a safe manner. Furthermore, the device according to the invention can be used in humanoid robots, particularly in the context of using a humanoid hand to pick up, hold, or place objects. Furthermore, the device according to the invention can be used in conjunction with an audio interface, particularly with a home robot that can function as a digital assistant with an interface to online or offline computer applications. Additionally, the device according to the invention can be used with robots capable of controlling switches and buttons for industrial and domestic applications. Furthermore, the device according to the invention can be used with smart home robots, such as Mayfield's Kuri. Furthermore, the device according to the invention can be used when robots operate with hazardous materials or objects or in hazardous environments.As a non-limiting example, the device according to the invention can be used in robots or unmanned remote-controlled vehicles to handle hazardous materials such as chemicals or radioactive materials, especially after a disaster, or to handle other dangerous or potentially dangerous objects such as landmines, unexploded weapons, etc., or to operate or conduct research in unsafe environments such as nearby burning objects or post-disaster areas, or to carry out manned or unmanned rescue operations in the air, at sea, underground, etc.

[0191] Furthermore, the device according to the invention can be used to inspect adhesive beads, sealing beads, etc., such as identifying cracks, thick spots, shrinkage, asymmetry, local defects, etc. Furthermore, the device according to the invention can be used to count objects such as dried fruit on a conveyor belt, such as in difficult situations, such as when fruits of similar color and shape may be in direct contact with each other. Furthermore, the device according to the invention can be used for quality control of die-cast or injection-molded parts, such as ensuring defect-free casting or molding, identifying surface damage, worn tools, etc. Furthermore, the device according to the invention can be used for laser scribing, such as for laser quality control and positioning. Furthermore, the device according to the invention can be used in classification systems, such as detecting the position, rotation, and shape of objects, comparing them with an object database, and classifying the objects. Furthermore, the device according to the invention can be used for stamping parts inspection, packaging inspection, such as food and pharmaceutical packaging inspection, filament inspection, etc.

[0192] Furthermore, the device according to the invention can be used for navigation purposes where the Global Positioning System (GPS) is not reliable enough. GPS signals typically use radio waves, which can be blocked or difficult to receive indoors or outdoors in valleys or forests below the treeline. Moreover, especially in unmanned autonomous vehicles, the weight of the system can be critical. Unmanned autonomous vehicles, in particular, require high-speed position data for reliable feedback and stability of their control systems. Using the device according to the invention allows for short response times and positioning without adding weight due to device weight.

[0193] Furthermore, the device according to the invention can be used in household, mobile, or entertainment devices, such as refrigerators, microwave ovens, washing machines, curtains or blinds, home alarms, air conditioning equipment, heating equipment, televisions, audio equipment, smartwatches, mobile phones, telephones, dishwashers, stoves, etc., to detect the presence of a person, monitor the content or function of the device, or interact with a person and / or share information about that person with other household, mobile, or entertainment devices.

[0194] Furthermore, the device according to the invention can be used to support the elderly or disabled or those with limited or no vision, such as when doing housework or at work, such as in devices for holding, carrying or picking up objects, or in safety systems that use optical or acoustic signals to signal obstacles in the environment.

[0195] The device according to the invention can be further used in agriculture, for example, to detect and classify pests, weeds, and / or infected crop plants, which may be infected by fungi or insects, either completely or partially. Furthermore, for the purpose of harvesting crops, the device according to the invention can be used to detect animals (e.g., deer) that might otherwise be harmed by harvesting equipment. Additionally, the device according to the invention can be used to monitor plant growth in fields or greenhouses, particularly for regulating the amount of water, fertilizer, or crop protection products in a given area of ​​a field or greenhouse, or even for a given plant. Furthermore, in agricultural biotechnology, the device according to the invention can be used to monitor the size and shape of plants.

[0196] Furthermore, the device according to the invention can be used to guide users during processes such as shaving, haircutting, or makeup application. Additionally, the device according to the invention can be used to record or monitor the content played on musical instruments such as violins. Furthermore, the device according to the invention can be used in smart home appliances such as smart refrigerators, for example, to monitor the contents of the refrigerator and send notifications based on the content. Furthermore, the device according to the invention can be used to monitor or track populations of humans, animals, or plants, such as precious or tree species in forests. Furthermore, the device according to the invention can be used in harvesters, for example, for harvesting crops, flowers, or fruits, such as grapes, corn, hops, apples, grains, rice, strawberries, asparagus, tulips, roses, soybeans, etc. Furthermore, the device according to the invention can be used to monitor the growth of plants, animals, algae, fish, etc., for example in breeding, food production, agricultural, or research applications, to control irrigation, fertilization, humidity, temperature, and the use of herbicides, pesticides, fungicides, rodenticides, etc. Furthermore, the device according to the invention can be used in animal or pet feeding machines, for example, for cattle, pigs, cats, dogs, birds, fish, etc. Furthermore, the device according to the invention can be used in animal product production processes, such as for collecting milk, eggs, fur, meat, etc., for example, in automated milking or slaughtering processes. Furthermore, the device according to the invention can be used in automated seeders, seed-bearers, or planters, for example, for planting corn, garlic, trees, salads, etc. Furthermore, the device according to the invention can be used to assess or monitor weather phenomena, such as clouds, fog, etc., or to warn of dangers such as avalanches, tsunamis, strong winds, earthquakes, thunderstorms, etc. Furthermore, the device according to the invention can be used to measure motion, impact, vibration, etc., in order to monitor earthquake risk. Furthermore, the device according to the invention can be used in traffic technology to monitor dangerous intersections, control traffic lights according to traffic rules, monitor public spaces, and monitor roads, gyms, stadiums, ski resorts, public events, etc. Furthermore, the device according to the invention can be used in medical applications, such as monitoring or analyzing tissues, medical or biological measurements, changes in tissues, such as moles or melanomas, counting bacteria, blood cells, cells, algae, for retinal scanning, respiratory or pulse measurement, gastroscopy, patient monitoring, etc. Furthermore, the device according to the invention can be used to monitor the shape, size, or circumference of droplets, streams, jets, etc., or to analyze, evaluate, or monitor, for example, the profile or gas or liquid flow in a duct. Furthermore, the device according to the invention can be used to warn drivers, such as when a car or train driver is sick or fatigued. Furthermore, the device according to the invention can be used for material testing to identify strain, tension, or cracks. Furthermore, the device according to the invention can be used in navigation to monitor and optimize navigation position, for example, automatically. Furthermore, the device according to the invention can be used for fuel level gauges.

[0197] Furthermore, the device according to the invention can be combined with sensors for detecting chemicals or contaminants, electronic nose chips, microbial sensor chips for detecting bacteria or viruses, Geiger counters, tactile sensors, thermal sensors, etc. For example, this can be used to construct intelligent robots configured to handle dangerous or difficult tasks, such as treating highly infected patients, handling or removing highly hazardous substances, cleaning highly contaminated areas, such as highly radioactive areas or chemical spills, or for agricultural pest control.

[0198] One or more devices according to the invention can also be used to scan objects, for example, in combination with CAD or similar software, for example, for additive manufacturing and / or 3D printing. High spatial accuracy can be used with the devices according to the invention, for example, in the x, y, or z directions, or any combination of these directions, for example, simultaneously. Furthermore, the devices according to the invention can be used for inspection and maintenance, such as pipe inspection instruments. Additionally, in a production environment, the devices according to the invention can be used to handle objects whose shapes are difficult to define, such as naturally grown objects, such as vegetables or other natural products sorted by shape or size, or objects such as meat that are cut or manufactured with less precision than required for processing steps.

[0199] Furthermore, the device according to the invention can be used in local navigation systems to allow automatic or partially automatic movement of vehicles or multi-rotor aircraft through indoor or outdoor spaces. Non-limiting examples may include vehicle movement through an automatic storage unit to pick up objects and place them at different locations. Indoor navigation can also be used in shopping malls, retail stores, museums, airports, or train stations to track the location of moving merchandise, mobile devices, luggage, customers, or employees, or to provide users with location-specific information, such as their current location on a map or information about the goods being sold.

[0200] Furthermore, the device according to the invention can be used to ensure safe driving of motorcycles, for example, by monitoring speed, tilt, approaching obstacles, road unevenness or curves, etc., to provide driving assistance to motorcycles. Additionally, the device according to the invention can be used in trains or trams to avoid collisions.

[0201] Furthermore, the device according to the invention can be used in handheld devices, for example, for scanning packaging or parcels to optimize the logistics process. Additionally, the device according to the invention can be used in other handheld devices, such as personal shopping devices, RFID readers, handheld devices for use in hospital or healthcare environments, such as handheld devices for medical purposes or for acquiring, exchanging, or recording patient or patient-related information, smart badges in retail or healthcare environments, etc.

[0202] As described above, the device according to the invention can also be used in manufacturing, quality control, or identification applications, such as product identification or size identification (e.g., for finding optimal locations or packaging to reduce waste, etc.). Furthermore, the device according to the invention can be used in logistics applications. Therefore, the device according to the invention can be used to optimize loading or packaging of containers or vehicles. Furthermore, the device according to the invention can be recalibrated using barcodes, QR codes, or pre-recorded symbols of known sizes by using at least one image matrix and comparing the measured characteristics of a pre-recorded size with a recorded image of a barcode, QR code, or pre-recorded symbol (e.g., by comparing the width or height of the symbol with a pre-recorded value). Furthermore, the device according to the invention can be used to monitor or control surface damage in the manufacturing field, to monitor or control rental items such as rental vehicles, and / or for insurance applications, such as for assessing damage. Furthermore, the device according to the invention can be used to identify the dimensions of materials, objects, or tools, for example, for optimal material handling, especially in combination with robots. Furthermore, the device according to the invention can be used for process control in production, for example, for observing the fill level of tanks. Furthermore, the device according to the invention can be used for maintaining production assets, such as, but not limited to, tanks, pipes, reactors, tools, etc. Furthermore, the device according to the invention can be used to analyze 3D quality marks. Furthermore, the device according to the invention can be used to manufacture customized products, such as dental inlays, orthodontic appliances, prostheses, clothing, etc. The device according to the invention can also be combined with one or more 3D printers for rapid prototyping, 3D replication, etc. Additionally, the device according to the invention can be used to detect the shape of one or more items, for example, for anti-piracy and anti-counterfeiting purposes.

[0203] Therefore, specifically, this application can be applied to the field of photography. Thus, the detector can be part of a photographic device, specifically part of a digital camera. Specifically, the detector can be used for 3D photography, specifically for digital 3D photography. Therefore, the detector can form part of a digital 3D camera or can be a component of a digital 3D camera. As used herein, the term photography generally refers to a technique for acquiring image information of at least one object. As further used herein, a camera is generally a device suitable for performing photography. As further used herein, the term digital photography generally refers to a technique for acquiring image information of at least one object by using a plurality of photosensitive elements adapted to generate electrical signals, preferably digital electrical signals, indicating illumination intensity and / or color. As further used herein, the term 3D photography generally refers to a technique for acquiring image information of at least one object in three spatial dimensions. Therefore, a 3D camera is a device suitable for performing 3D photography. A camera can generally be adapted to acquire a single image, such as a single 3D image, or can be adapted to acquire multiple images, such as an image sequence. Therefore, a camera can also be a video camera suitable for video applications, such as for acquiring digital video sequences.

[0204] Therefore, the present invention further relates to a camera for imaging at least one object, specifically a digital camera, more specifically a 3D camera or a digital 3D camera. As outlined above, the term imaging, as used herein, generally refers to acquiring image information of at least one object. The camera includes at least one detector according to the invention. As outlined above, the camera may be adapted to acquire a single image or to acquire multiple images (such as image sequences), preferably for acquiring digital video sequences. Thus, by way of example, the camera may be or may include a video camera. In the latter case, the camera preferably includes a data storage device for storing the image sequence.

[0205] As used in this invention, the term "position" generally refers to at least one of the information regarding the absolute position and orientation of one or more points of an object. Therefore, specifically, the position can be determined in the coordinate system of the detector (such as a Cartesian coordinate system). However, alternatively or separately, other types of coordinate systems, such as polar coordinate systems and / or spherical coordinate systems, can be used.

[0206] As described above and will be further described in detail below, the present invention is preferably applicable to the fields of human-computer interfaces, sports, and / or computer games. Therefore, preferably, the object can be selected from the group consisting of: sports equipment articles, preferably articles selected from the group consisting of rackets, cues, and clubs, clothing, hats, and shoes. Other embodiments are possible.

[0207] As used herein, an object can generally be any object selected from living and inanimate objects. Thus, by way of example, at least one object may include one or more items and / or one or more parts of items. Additionally or alternatively, an object may be or may include one or more organisms and / or one or more parts thereof, such as one or more body parts of a human (e.g., a user) and / or an animal.

[0208] Regarding the coordinate system used to determine the position of an object, it can be the coordinate system of a detector, where the detector's optical axis forms the z-axis, and additionally, x-axis and y-axis perpendicular to and perpendicular to each other can be provided. As an example, the detector and / or a portion of the detector can be positioned at a specific point in this coordinate system, such as the origin. In this coordinate system, directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered ordinates. Any direction perpendicular to the longitudinal direction can be considered transverse directions, and x-coordinates and / or y-coordinates can be considered transverse coordinates.

[0209] Alternatively, other types of coordinate systems can be used. Thus, as an example, a polar coordinate system can be used, where the optical axis forms the z-axis and the distance from the z-axis and the polar angle can be used as additional coordinates. Similarly, directions parallel or antiparallel to the z-axis can be considered longitudinal directions, and coordinates along the z-axis can be considered ordinates. Any direction perpendicular to the z-axis can be considered transverse directions, and polar coordinates and / or polar angles can be considered transverse coordinates.

[0210] The detector can be a device configured to provide at least one piece of information about the location of at least one object and / or a portion thereof. Thus, location can refer to an information item that fully describes the position of the object or a portion thereof, preferably, in the detector's coordinate system, or it can refer to partial information that only partially describes the location. The detector can typically be a device suitable for detecting light beams (such as beams propagating from a beacon device toward the detector).

[0211] Evaluation devices and detectors can be fully or partially integrated into a single device. Therefore, evaluation devices can often also form part of a detector. Alternatively, evaluation devices and detectors can be fully or partially embodied as separate devices. Detectors may include other components.

[0212] The detector can be a fixed or mobile device. Furthermore, the detector can be a standalone device or can be integrated into another device (such as a computer, vehicle, or any other device). Additionally, the detector can be a handheld device. Other embodiments of the detector are possible.

[0213] Specifically, the detector can be used to record the light field behind a lens or lens system of the detector, comparable to an all-light or light field camera. Therefore, specifically, the detector can be embodied as a light field camera suitable for simultaneously acquiring images in multiple focal planes. As used herein, the term light field generally refers to the spatial propagation of light inside the detector (such as inside a camera). The detector according to the invention (specifically a stack of optical sensors) can have the ability to directly record the light field within the detector or camera (such as behind a lens). Multiple sensors can record images at different distances from the lens. Using, for example, convolution-based algorithms (such as "depth from focus" or "depth from defocus"), the propagation direction, focus, and spread of light behind the lens can be modeled. From the modeled propagation of light behind the lens, images at different distances from the lens can be extracted, depth of field can be optimized, images focused at different distances can be extracted, or the distance to an object can be calculated. More information can be extracted.

[0214] The use of multiple optical sensors further allows for the correction of lens errors during image processing steps after image recording. Optical instruments often become expensive and complex when lens error correction is required. These are particularly problematic in microscopes and telescopes. In microscopes, a typical lens error is the different distortions of light rays at different distances from the optical axis (spherical aberration). In telescopes, varying atmospheric temperatures can cause changes in focal length. Static errors, such as spherical aberration or other manufacturing errors, can be corrected by identifying the error during the calibration step and then using fixed image processing (such as fixed pixel and sensor groups) or more sophisticated processing techniques using light propagation information. In cases where lens error is closely related to time, i.e., dependent on weather conditions within the telescope, lens errors can be corrected by using light propagation behind the lens, calculating extended depth-of-field images, using depth-of-focus techniques, etc.

[0215] The detector according to the invention further allows for color detection. For color detection, multiple optical sensors with different spectral characteristics can be used, and the sensor signals of these optical sensors can be compared. Furthermore, the device according to the invention can be used in the context of gesture recognition. In this context, gesture recognition combined with the device according to the invention can be particularly used as a human-machine interface for transmitting information to a machine via movement of the body, body parts, or objects. Here, information can preferably be transmitted via movement of the hand or parts of the hand (such as fingers), particularly by pointing at an object, applying sign language (e.g., for the deaf), making numerical gestures, waving to indicate approval or disapproval, for example, when asking someone to approach, leave, or greet someone, pressing an object, taking an object, or in sports or music, by hand or finger movements, such as warm-up exercises. Additionally, information can be transmitted via arm or leg movements, such as rotation, kicking, grabbing, twisting, spinning, rolling, browsing, pushing, bending, punching, shaking, arms, legs, both arms or legs, or combinations of arms and legs, for example for sports or music purposes, such as for the entertainment, exercise, or training functions of a machine. Furthermore, information can be transmitted through movement of the entire body or its major parts, such as jumping, spinning, or creating complex signs, like those used by traffic police at airports to convey information such as "turn right," "turn left," "go forward," "slow down," "stop," or "stop the engine," or by faking swimming, diving, running, shooting, etc., or by creating complex movements or body postures, such as yoga, Pilates, judo, karate, dance, or ballet. Additionally, information can be transmitted using real or simulated devices to control virtual devices corresponding to the simulated devices; for example, using a simulated guitar to control a virtual guitar function in a computer program, using a real guitar to control a virtual guitar function in a computer program, using real or simulated books to read ebooks or move pages or browse virtual documents, using real or simulated pens to draw in computer programs, etc. Furthermore, the transmission of information can be coupled to feedback to the user, such as sound, vibration, or movement.

[0216] In the context of music and / or musical instruments, the device according to the invention, combined with gesture recognition, can be used for exercise purposes, instrument control, instrument recording, playing or recording music by using a simulated instrument or simply pretending that an instrument is present, such as playing an air guitar, for example, to avoid noise or record, or for creating a virtual orchestra, choir, band, big band, etc., for practice, exercise, recording or entertainment purposes, etc.

[0217] Furthermore, in security and surveillance contexts, the device according to the invention, combined with gesture recognition, can be used to identify the movement profile of a person, such as by walking or moving the body, or by using gestures or movements, or markers or movements of body parts or the whole body, as access or identification controls, such as personal identification markers or personal identification movements.

[0218] Furthermore, in the context of smart home appliances or the Internet of Things, the device according to the present invention, combined with gesture recognition, can be used for central or decentralized control of home devices. Home devices can be part of an interconnected network of home appliances and / or home devices, such as refrigerators, central heating, air conditioners, microwave ovens, ice makers or water heaters, or entertainment devices such as televisions, smartphones, game consoles, video recorders, DVD players, personal computers, laptops, tablets or combinations thereof, or combinations of home devices and entertainment devices.

[0219] Furthermore, in virtual reality or augmented reality scenarios, the device according to the present invention, combined with gesture recognition, can be used to control the movement or function of virtual reality or augmented reality applications, such as playing or controlling games using signs, gestures, body movements or body part movements, moving in the virtual world, manipulating virtual objects, and using virtual objects to practice, exercise or engage in sports, art, crafts, music or games, such as balls, chess pieces, stones, instruments, tools, brushes.

[0220] Furthermore, in a medical context, the device according to the invention, combined with gesture recognition, can be used to support rehabilitation training, remote diagnosis, or monitor or investigate surgery or treatment to overlay and display medical images with the location of the medical device, or to overlay and display pre-recorded medical images, such as those from magnetic resonance imaging or X-rays, using images recorded during surgery or treatment from endoscopes or ultrasound.

[0221] Furthermore, in the context of manufacturing and process automation, the device according to the present invention, combined with gesture recognition, can be used to control, teach, or program robots, drones, unmanned autonomous vehicles, service robots, mobile objects, etc., for example, for programming, control, manufacturing, manipulation, repair, or teaching purposes, or for remotely manipulating objects or areas, for example, for safety reasons or for maintenance purposes.

[0222] Furthermore, in the context of business intelligence measurement, the device according to the present invention, combined with gesture recognition, can be used for headcount, investigating customer movement, areas where customers spend time, objects, customer testing, acceptance, probing, etc.

[0223] Furthermore, the device according to the invention can be used with hand or professional tools, especially electric or motor-driven tools or power tools, such as drills, saws, chisels, hammers, wrenches, nail guns, disc cutters, metal shears and impactors, angle grinders, die grinders, drill bits, hammer drills, hot air guns, wrenches, sanders, engraving machines, nail guns, jigsaws, biskies, wood milling machines, planers, polishing machines, tile cutters, washers, rollers, wall chasers, lathes, impact actuators, connectors, paint rollers, spray guns, mortises or welders, especially to support manufacturing precision, maintain minimum or maximum distances, or for safety measures.

[0224] Furthermore, the device according to the invention can be used to assist visually impaired individuals. Additionally, the device according to the invention can be used in touchscreens to avoid direct contact, for example, for hygiene reasons, in retail environments, medical applications, production environments, etc. Furthermore, the device according to the invention can be used in agricultural production environments, such as stable cleaning robots, egg collectors, milking machines, harvesting machines, agricultural implements, harvesters, freight forwarders, combine harvesters, tractors, tillers, plows, destoners, harrows, strip harvesters, seed spreaders, planters such as potato planters, fertilizer spreaders, sprayers, sprinkler fire suppression systems, rice harvesters, balers, loaders, forklifts, lawn mowers, etc.

[0225] Furthermore, the device according to the invention can be used to select and / or adjust clothing, shoes, glasses, hats, prostheses, and orthodontic appliances for people or animals with limited communication skills or abilities, such as children or people with disabilities. Additionally, the device according to the invention can be used in warehousing, logistics, distribution, transportation, loading, unloading, smart manufacturing, Industry 4.0, and other similar contexts. Furthermore, in manufacturing contexts, the device according to the invention can be used in processing, dispensing, bending, material handling, and other similar situations.

[0226] The evaluation device may be or may include one or more integrated circuits, such as one or more application-specific integrated circuits (ASICs), and / or one or more data processing devices, such as one or more computers, preferably one or more microcomputers and / or microcontrollers, field-programmable arrays, or digital signal processors. Additional components may be included, such as one or more preprocessing devices and / or data acquisition devices, such as one or more devices for receiving and / or preprocessing sensor signals, such as one or more analog-to-digital converters and / or one or more filters. Furthermore, the evaluation device may include one or more measuring devices, such as one or more measuring devices for measuring current and / or voltage. Additionally, the evaluation device may include one or more data storage devices. Furthermore, the evaluation device may include one or more interfaces, such as one or more wireless interfaces and / or one or more wired bonding interfaces.

[0227] At least one evaluation device may be adapted to execute at least one computer program, such as at least one computer program adapted to perform or support one or more, or even all, the method steps according to the invention. As an example, one or more algorithms may be implemented that determine the position of an object by using sensor signals as input variables.

[0228] The evaluation device may be connected to or may include at least one other data processing device that can be used for one or more of the following: display, visualization, analysis, distribution, communication, or further processing of information (such as information obtained by optical sensors and / or by the evaluation device). As an example, the data processing device may be connected to or combined with at least one of a display, projector, monitor, LCD, TFT, speaker, multi-channel sound system, LED pattern, or other visualization device. It may further be connected to or combined with at least one of a communication device or communication interface, connector, or port capable of sending encrypted or unencrypted information using one or more of the following: email, text messaging, telephone, Bluetooth, Wi-Fi, infrared, or internet interfaces, ports, or connections. It may further be connected to or combined with a processor, graphics processor, CPU, or Open Multimedia Application Platform (OMAP). TM This includes at least one of the following: integrated circuits, system-on-a-chip (such as products from Apple's A-series or Samsung's S3C2-series), microcontrollers or microprocessors, one or more memory blocks (such as ROM, RAM, EEPROM, or flash memory), timing sources (such as oscillators or phase-locked loops), counter timers, real-time timers or power-on reset generators, voltage regulators, power management circuitry, or DMA controllers. The individual units can be further connected via a bus such as the AMBA bus, or integrated into IoT or Industry 4.0 networks.

[0229] Evaluation equipment and / or data processing equipment may be connected via, or have, other external interfaces or ports, such as one or more of the following: serial or parallel interfaces or ports, USB, Centronics ports, FireWire, HDMI, Ethernet, Bluetooth, RFID, Wi-Fi, USART or SPI, or analog interfaces or ports (e.g., one or more of ADC or DAC), or standardized interfaces or ports, for connecting to other devices, such as 2D camera devices using RGB interfaces such as CameraLink. Evaluation equipment and / or data processing equipment may further be connected via inter-processor interfaces or ports, FPGA-FPGA interfaces, or one or more of serial or parallel interface ports. Evaluation equipment and data processing equipment may further be connected to one or more of optical disc drives, CD-RW drives, DVD+RW drives, flash drives, memory cards, disk drives, hard disk drives, solid-state drives, or solid-state drives.

[0230] The evaluation equipment and / or data processing equipment may be connected via one or more other external connectors, or have one or more other external connectors, such as telephone connectors, RCA connectors, VGA connectors, hermaphrodite connectors, USB connectors, HDMI connectors, 8P8C connectors, BCN connectors, IEC60320 C14 connectors, fiber optic connectors, D miniature connectors, RF connectors, coaxial connectors, SCART connectors, XLR connectors, and / or may include at least one suitable socket for one or more of these connectors.

[0231] Possible embodiments of a single device comprising one or more detectors, evaluation devices, or data processing devices according to the invention (e.g., comprising one or more of optical sensors, optical systems, evaluation devices, communication devices, data processing devices, interfaces, system-on-a-chip, display devices, or other electronic devices) include: mobile phones, personal computers, tablet PCs, televisions, game consoles, or other entertainment devices. In further embodiments, the 3D camera functionality, which will be described in more detail below, can be integrated into a device usable with a conventional 2D digital camera without significant differences in the device's housing or appearance, wherein the significant difference for the user may be only the functionality of acquiring and / or processing 3D information. Furthermore, the device according to the invention can be used with 360° digital cameras or surround-view cameras.

[0232] Specifically, embodiments including a detector and / or a portion thereof (e.g., evaluation equipment and / or data processing equipment) may be: a mobile phone comprising a display device, data processing equipment, an optical sensor, optional sensor optics, and evaluation equipment for use with a 3D camera. The detector according to the invention is specifically adaptable for integration into entertainment devices and / or communication devices such as mobile phones.

[0233] Another embodiment of the invention may be the integration of a detector or a portion thereof (e.g., evaluation equipment and / or data processing equipment) into a device for automobiles, for autonomous driving, or for passenger car safety systems (e.g., Daimler's intelligent drive system). As an example, the device, comprising one or more of an optical sensor, optionally one or more optical systems, an evaluation device, optionally a communication device, optionally a data processing device, optionally one or more interfaces, optionally a system-on-a-chip, optionally one or more display devices, or optionally other electronic devices, may be part of a vehicle, passenger car, truck, train, bicycle, aircraft, ship, or motorcycle. In automotive applications, integrating this device into the automotive design requires the integration of the optical sensor, optionally an optics device, or a device with minimal visibility from the outside or inside. A detector or a portion thereof (e.g., evaluation equipment and / or data processing equipment) may be particularly suitable for such integration into automotive designs.

[0234] As used herein, the term "light" generally refers to electromagnetic radiation in one or more of the visible, ultraviolet, and infrared spectral ranges. Specifically, the term "visible spectral range" generally refers to the spectral range of 380 nm to 780 nm. The term "infrared spectral range" generally refers to electromagnetic radiation in the range of 780 nm to 1 mm, preferably in the range of 780 nm to 3.0 micrometers. The term "ultraviolet spectral range" generally refers to electromagnetic radiation in the range of 1 nm to 380 nm, preferably in the range of 100 nm to 380 nm. Preferably, the light used in this invention is visible light, i.e., light in the visible spectral range.

[0235] The term "beam" can generally refer to the amount of light emitted and / or reflected in a particular direction. Therefore, a beam can be a ray of light having a predetermined extension in a direction perpendicular to the beam's propagation direction. Preferably, a beam can be or can include one or more Gaussian beams (such as a linear combination of Gaussian beams) that can be characterized by one or more Gaussian beam parameters, such as beam waist, Rayleigh length, or any other beam parameter, or a combination of beam parameters suitable for characterizing the beam diameter and / or the development of the beam's propagation in space.

[0236] The detector according to the invention can be further combined with one or more other types of sensors or detectors. Therefore, the detector can further include at least one additional detector. This at least one additional detector can be adapted to detect at least one parameter, such as at least one of the following: parameters of the surrounding environment, such as ambient temperature and / or brightness; parameters relating to the position and / or orientation of the detector; parameters specifying the state of the object to be detected, such as the object's position, for example, the object's absolute position and / or the object's orientation in space. Therefore, in general, the principles of the invention can be combined with other measurement principles to obtain additional information, and / or to verify measurement results or reduce measurement errors or noise.

[0237] The human-machine interface may include multiple beacon devices adapted to be directly or indirectly attached to and held by at least one of the user. Thus, the beacon devices can be independently attached to the user by any suitable means (e.g., by appropriate securing devices). Additionally or alternatively, the user may hold and / or carry at least one beacon device or one or more of the beacon devices in his or her hand and / or by wearing at least one beacon device on a body part and / or clothing containing the beacon device.

[0238] A beacon device can typically be any device that is detectable by at least one detector and / or easily detectable by at least one detector. Thus, as described above or as will be described in more detail below, a beacon device can be an active beacon device adapted to generate at least one beam of light that will be detected by a detector, for example by having one or more illumination sources for generating at least one beam of light. Additionally or alternatively, the beacon device can be designed wholly or partially as a passive beacon device, for example by providing one or more reflective elements adapted to reflect beams of light generated by a single illumination source. The at least one beacon device can be permanently or temporarily attached to a user, directly or indirectly, and / or can be carried or held by the user. This attachment can be achieved by using one or more attachment devices and / or by the user themselves, for example by the user holding the at least one beacon device and / or by the user wearing the beacon device.

[0239] Additionally or alternatively, the beacon device may be at least one of being attached to an object and integrated into an object held by a user; for the purposes of this invention, this should be included in the meaning of the option of a user holding the beacon device. Thus, as described in more detail below, the beacon device may be attached to or integrated into a control element, which may be part of a human-machine interface and may be held or carried by a user, its orientation being identifiable by the detector device. Therefore, in general, the invention also relates to a detector system comprising at least one detector device according to the invention, and may further comprise at least one object, wherein the beacon device is one of being attached to the object, held by the object, or integrated into the object. As an example, the object may preferably form a control element whose orientation can be identifiable by a user. Thus, as described above or further below, the detector system may be part of a human-machine interface. As an example, a user may operate the control element in a particular manner to send one or more pieces of information to a machine, for example, to send one or more commands to a machine.

[0240] Alternatively, the detector system can be used in other ways. Thus, as an example, the object of the detector system can be different from a user or a part of a user's body, and as an example, it can be an object that moves independently of the user. As an example, the detector system can be used to control equipment and / or industrial processes, such as manufacturing processes and / or robotic processes. Thus, as an example, the object can be a machine and / or machine parts, such as a robotic arm, whose orientation can be detected using the detector system.

[0241] The human-machine interface can be adapted to enable the detector device to generate at least one piece of information about the position of a user or at least one body part of the user. Specifically, given that the attachment method of at least one beacon device to the user is known, at least one piece of information about the position and / or orientation of the user or at least one body part of the user can be obtained by evaluating the position of the at least one beacon device.

[0242] The beacon device is preferably one of an attachmentable beacon device to a user's body or body part and a beacon device that can be held by the user. As mentioned above, the beacon device can be designed wholly or partially as an active beacon device. Therefore, the beacon device may include at least one illumination source adapted to generate at least one light beam to be transmitted to a detector, preferably at least one light beam having known beam characteristics. Additionally or alternatively, the beacon device may include at least one transmitter adapted to reflect the light generated by the illumination source, thereby generating a reflected light beam to be transmitted to the detector.

[0243] Objects that can form part of a detector system can generally have any shape. Preferably, as described above, the object as part of the detector system can be a control element that can be operated by a user (e.g., manually). As an example, the control element can be or can include at least one element selected from the group consisting of: gloves, coats, hats, shoes, trousers and suits, canes that can be held in the hand, baseball bats, clubs, rackets, walking sticks, toys (e.g., toy guns). Therefore, as an example, the detector system can be part of a human-machine interface and / or entertainment device.

[0244] As used herein, an entertainment device is a device that can be used for the leisure and / or entertainment purposes of one or more users (hereinafter also referred to as one or more players). As an example, an entertainment device can be used for gaming purposes, preferably for computer gaming purposes. Therefore, an entertainment device can be implemented as a computer, computer network, or computer system, or may include a computer, computer network, or computer system running one or more game software programs.

[0245] The entertainment device includes at least one human-machine interface according to the present invention (e.g., according to one or more of the embodiments disclosed above and / or one or more of the embodiments disclosed below). The entertainment device is designed to allow a player to input at least one piece of information via the human-machine interface. This at least one piece of information can be transmitted to, and / or used by, the controller and / or computer of the entertainment device. The at least one piece of information preferably includes at least one command suitable for influencing the game process. Thus, as an example, the at least one piece of information may include at least one piece of information relating to at least one orientation of the player and / or one or more body parts of the player, thereby allowing the player to simulate specific positions and / or orientations and / or actions required for the game. As an example, one or more of the following movements can be simulated and transmitted to the controller and / or computer of the entertainment device: dancing; running; jumping; swinging a racket; swinging a bat; swinging a club; pointing an object at another object, such as pointing a toy gun at a target.

[0246] As part of or as a whole, the entertainment device, preferably its controller and / or computer, is designed to modify entertainment functions based on information. Therefore, as described above, the game process can be influenced based on at least one piece of information. Thus, the entertainment device may include one or more controllers, which may be separate from and / or completely or partially identical to at least one evaluation device, or may even include at least one evaluation device. Preferably, at least one controller may include one or more data processing devices, such as one or more computers and / or microcontrollers.

[0247] As further used herein, a tracking system is a device adapted to collect information relating to a series of past locations of at least one object and / or at least a portion of that object. Additionally, the tracking system may be adapted to provide information relating to at least one predicted future location and / or orientation of at least one object or at least a portion of that object. The tracking system may have at least one tracking controller, which may be wholly or partially embodied as an electronic device, preferably as at least one data processing device, more preferably as at least one computer or microcontroller. Furthermore, the at least one tracking controller may wholly or partially include at least one evaluation device and / or may be part of at least one evaluation device and / or may be wholly or partially identical to at least one evaluation device.

[0248] The tracking system includes at least one detector according to the invention, such as at least one detector disclosed in one or more embodiments listed above and / or disclosed in one or more embodiments below. The tracking system further includes at least one tracking controller. The tracking controller is adapted to track a series of locations of an object at specific points in time, for example by recording multiple sets of data or data pairs, each set of data or data pair including at least one location information and at least one time information.

[0249] The tracking system may further include at least one detector system according to the invention. Therefore, in addition to at least one detector and at least one evaluation device, and optionally at least one beacon device, the tracking system may further include the object itself or a portion thereof, such as at least one control element comprising a beacon device or at least one beacon device, wherein the control element is directly or indirectly attached to or integrated into the object to be tracked.

[0250] The tracking system can be adapted to initiate one or more actions by the tracking system itself and / or one or more individual devices. For the latter purpose, the tracking system, preferably a tracking controller, can have one or more wireless and / or wired interfaces and / or other types of control connections for initiating at least one action. Preferably, at least one tracking controller can be adapted to initiate at least one action based on at least one actual location of the object. As an example, the action can be selected from the group consisting of: predicting the future location of the object; pointing at least one device at the object; pointing at at least one device at a detector; illuminating the object; illuminating the detector.

[0251] As an example of an application of a tracking system, it can be used to continuously point at least one first object to at least one second object, even if the first and / or second objects may be moving. Additionally, potential examples can be found in industrial applications (e.g., in robotics), and / or for continuous operation on an article, even if the article is moving, such as during manufacturing on a production or assembly line. Additionally or alternatively, a tracking system can be used for illumination purposes, such as for continuously illuminating an object by continuously pointing an illumination source at it, even if the object may be moving. Other applications can be found in communication systems, for example, to continuously send information to a moving object by pointing a transmitter at it.

[0252] In general, the following embodiments are considered preferred in the context of this invention:

[0253] Example 1: A detector for determining the location of at least one object, the detector comprising

[0254] - At least one sensor element having a matrix of optical sensors, each of the optical sensors having a photosensitive region, wherein the sensor element is configured to determine at least one reflected image;

[0255] - At least one evaluation device, wherein the evaluation device is configured to select at least one reflection feature of the reflection image at at least one first image location in the reflection image, wherein the evaluation device is configured to optimize at least one blur function f a To determine at least one ordinate z of a selected reflection feature, wherein the evaluation device is configured to determine at least one reference feature in at least one reference image at at least one second image position in the reference image corresponding to the at least one reflection feature, wherein the reference image and the reflection image are determined in two different spatial configurations, wherein the difference between the spatial configurations lies in the relative spatial constellation, wherein the evaluation device is configured to determine the relative spatial constellation based on the ordinate z, the first image position, and the second image position.

[0256] Example 2: The detector according to the foregoing embodiment, wherein the ordinate z is determined by using at least one convolution-based algorithm such as the defocus depth algorithm.

[0257] Example 3: The detector according to any one of the foregoing embodiments, wherein the fuzzing function is optimized by changing the parameters of the at least one fuzzing function.

[0258] Example 4: The detector according to the foregoing embodiments, wherein the reflected image is a blurred image i bThe evaluation device is configured to evaluate the blurred image i b and the fuzzy function f a To reconstruct the ordinate z.

[0259] Example 5: The detector according to the foregoing embodiments, wherein the blurred image i is minimized by changing the parameter σ of the blur function. b With the fuzzy function f a And other images i' b The difference between the convolutions is used to determine the ordinate z.

[0260] min||(i′ b *f a (σ(z))-i b )||.

[0261] Example 6: The detector according to the foregoing embodiments, wherein the other images are blurry or clear images.

[0262] Example 7: A detector according to any one of the foregoing embodiments, wherein the at least one ambiguity function f a It is a function or composite function consisting of at least one function from a group including the following: Gaussian function, sine function, parabolic cylindrical function, square function, Lorentz function, radial function, polynomial, Hermite polynomial, Zernike polynomial, Legendre polynomial.

[0263] Example 8: The detector according to any one of the preceding embodiments, wherein the relative spatial constellation is at least one constellation selected from the group consisting of: relative spatial orientation; relative angular position; relative distance; relative displacement; relative movement.

[0264] Example 9: A detector according to any one of the foregoing embodiments, wherein the detector comprises at least two sensor elements separated by a relative space constellation, wherein at least one first sensor element is adapted to record the reference image, and at least one second sensor element is adapted to record the reflected image.

[0265] Example 10: A detector according to any one of the foregoing embodiments, wherein the detector is configured to use the same matrix of optical sensors at different times to record the reflected image and the reference image.

[0266] Example 11: The detector according to the foregoing embodiments, wherein the evaluation device is configured to determine at least one scaling factor for the relative space constellation.

[0267] Example 12: The detector according to any one of the preceding embodiments, wherein the evaluation device is configured to determine the displacement of the reference feature relative to the reflection feature.

[0268] Example 13: The detector according to the foregoing embodiments, wherein the evaluation device is configured to use triangulation of the object to measure the ordinate z. triang A predefined relationship between the displacement and the object is used to determine at least one triangulated ordinate z of the object. triang .

[0269] Example 14: A detector according to any one of the preceding two embodiments, wherein the evaluation device is configured to consider the determined relative spatial constellation and determine the actual relationship between the ordinate z and the displacement, wherein the evaluation device is configured to adjust the predefined relationship based on the actual relationship.

[0270] Example 15: The detector according to the foregoing embodiments, wherein the evaluation device is configured to replace the predefined relationship with the actual relationship, and / or the evaluation is configured to determine a moving average and replace the predefined relationship with the moving average.

[0271] Example 16: A detector according to any one of the preceding three embodiments, wherein the evaluation device is configured to determine the ordinate z and the triangulation coordinate z triang The difference between the two, wherein the evaluation device is configured to compare the determined difference with at least one threshold, and adjust the predefined relationship if the determined difference is higher than or equal to the threshold.

[0272] Example 17: A detector according to any one of the foregoing four embodiments, wherein the evaluation device is configured to determine an estimate of the corrected relative spatial relationship using a mathematical model, the mathematical model including the following parameters: various sensor signals and / or positions and / or image positions and / or system characteristics and / or ordinates, displacement d on the sensor, focal length f of the transmission device, temperature, z triang , baseline b, angle β between the illumination source and the baseline, ordinate z, etc., wherein the mathematical model is at least one mathematical model selected from the group consisting of: Kalman filter, linear quadratic estimator, Kalman-Busch filter, Stratonovich-Kalman-Busch filter, Kalman-Busch-Stratonovich filter, minimum variance estimator, Bayesian estimator, optimal linear unbiased estimator, invariant estimator, Wiener filter, etc.

[0273] Example 18: A detector according to any one of the preceding embodiments, wherein the evaluation device is configured to determine at least one longitudinal region, wherein the longitudinal region is given by the ordinate z and the error interval ±ε, wherein the evaluation device is configured to determine at least one displacement region in the reference image corresponding to the longitudinal region, wherein the evaluation device is configured to determine an epipolar line in the reference image, wherein the displacement region extends along the epipolar line, wherein the evaluation device is configured to determine the reference feature along the epipolar line corresponding to the ordinate z, and determine the range of the displacement region along the epipolar line corresponding to the error interval ±ε.

[0274] Example 19: The detector according to the foregoing embodiments, wherein the evaluation device is configured to perform the following steps:

[0275] - Determine the displacement region for the second image position of each reflection feature;

[0276] - For example, by assigning the epipolar line to the displacement region of each reflection feature by assigning the epipolar line closest to the displacement region and / or within the displacement region and / or along a direction orthogonal to the epipolar line.

[0277] - For example, assigning and / or determining at least one reference feature to each reflection feature by assigning a reference feature that is closest to the assigned displacement region and / or along the assigned epipolar line within the assigned displacement region.

[0278] Example 20: A detector according to any one of the preceding two embodiments, wherein the evaluation device is configured to match the reflection feature with the at least one reference feature in the displacement region.

[0279] Example 21: A detector according to any one of the foregoing embodiments, wherein the sensor element is configured to determine at least one second reflection image, wherein the evaluation device is configured to select at least one second reflection feature of the second reflection image at at least one third image location in the second reflection image, and to optimize the at least one blur function f aTo determine at least one second ordinate of the second reflection feature, wherein the evaluation device is adapted to determine at least one second reference feature corresponding to the at least one second reflection feature in at least one second reference image at at least one fourth image position in the second reference image, wherein the second reference image and the second reflection image are determined with two second different spatial configurations, wherein the difference between the spatial configurations lies in the actual relative spatial constellation, wherein the evaluation device is configured to determine the actual relative spatial constellation based on the second ordinate and the third image position and the fourth image, wherein the evaluation device is configured to compare the relative spatial constellation and the actual relative spatial constellation.

[0280] Example 22: The detector according to the foregoing embodiments, wherein the evaluation device is configured to adjust the relative spatial constellation depending on the actual relative spatial constellation.

[0281] Example 23: The detector according to the foregoing embodiments, wherein the evaluation device is configured to replace the relative spatial constellation with the actual relative constellation and / or the evaluation is configured to determine a moving average and replace the relative spatial constellation with the moving average.

[0282] Example 24: A detector according to any one of the foregoing two embodiments, wherein the evaluation device is configured to determine the difference between the relative space constellation and the actual relative space constellation, wherein the evaluation device is adapted to compare the determined difference with at least one threshold and adjust the relative space constellation if the determined difference is higher than or equal to the threshold.

[0283] Example 25: A detector according to any one of the preceding embodiments, wherein the detector includes at least one illumination source, wherein the illumination source is adapted to generate at least one illumination pattern for illuminating the object, wherein the illumination pattern includes at least one pattern selected from the group consisting of: at least one dot pattern, particularly a pseudo-random dot pattern; a random dot pattern or a quasi-random pattern; at least one Sobol pattern; at least one quasi-periodic pattern; at least one pattern including at least one known feature; at least one regular pattern; at least one triangular pattern; at least one hexagonal pattern; at least one rectangular pattern; at least one pattern including a convex uniform tessellation pattern; at least one line pattern including at least one line; at least one line pattern including at least two lines such as parallel lines or intersecting lines.

[0284] Example 26: A detector according to any one of the foregoing embodiments, wherein the detector includes at least one temperature determining unit, wherein the temperature determining unit is configured to determine at least one temperature value of the detector.

[0285] Example 27: The detector according to the foregoing embodiments, wherein the evaluation device is configured to determine the relative space constellation by taking into account the temperature value and / or to adjust the relative space constellation depending on the temperature value.

[0286] Example 28: A detector system for determining the location of at least one object, the detector system comprising at least one detector according to any one of the preceding embodiments, the detector system further comprising at least one beacon device adapted to guide at least one light beam toward the detector, wherein the beacon device is at least one of: attachable to the object, holdable by the object, and integrated into the object.

[0287] Example 29: A human-machine interface for exchanging at least one piece of information between a user and a machine, wherein the human-machine interface includes at least one detector system according to the foregoing embodiments, wherein the at least one beacon device is adapted to be at least one of the following: directly or indirectly attached to the user and held by the user, wherein the human-machine interface is designed to determine at least one location of the user by means of the detector system, wherein the human-machine interface is designed to assign at least one piece of information to the location.

[0288] Example 30: An entertainment device for performing at least one entertainment function, wherein the entertainment device includes at least one human-computer interface according to the foregoing embodiments, wherein the entertainment device is designed to enable a player to input at least one piece of information via the human-computer interface, wherein the entertainment device is designed to change the entertainment function based on the information.

[0289] Example 31: A tracking system for tracking the position of at least one movable object, the tracking system comprising at least one detector system according to any of the foregoing embodiments involving detector systems, the tracking system further comprising at least one tracking controller, wherein the tracking controller is adapted to track a series of positions of the object at specific points in time.

[0290] Example 32: A scanning system for determining the depth profile of a scene, the scanning system comprising at least one detector according to any of the foregoing embodiments involving detectors, the scanning system further comprising at least one illumination source adapted to scan the scene with at least one light beam.

[0291] Example 33: A camera for imaging at least one object, said camera comprising at least one detector according to any of the foregoing embodiments involving detectors.

[0292] Example 34: An inertial measurement unit for use in an electronic device, wherein the inertial measurement unit is adapted to receive data determined by at least one detector according to any of the foregoing embodiments involving detectors, wherein the inertial measurement unit is further adapted to receive data determined by at least one other sensor selected from the group consisting of: wheel speed sensor, turning rate sensor, tilt sensor, orientation sensor, motion sensor, magnetohydrodynamic sensor, force sensor, angle sensor, angular rate sensor, magnetic field sensor, magnetometer, accelerometer; gyroscope, wherein the inertial measurement unit is adapted to determine at least one characteristic of the electronic device by evaluating data from the detector and the at least one other sensor, the characteristic selected from the group consisting of: position in space, relative or absolute motion in space, rotation, acceleration, orientation, angular position, tilt, turning rate, velocity.

[0293] Example 35: A method for determining the location of at least one object using at least one detector according to any of the foregoing embodiments involving detectors, the method comprising the steps of:

[0294] - At least one reflected image of the object is determined by using at least one sensor element of a matrix having optical sensors, each of the optical sensors having a photosensitive area;

[0295] - Select at least one reflection feature of the reflected image at at least one first image location in the reflected image, and optimize at least one blur function f a To determine at least one ordinate z of the selected reflection feature;

[0296] - Provide at least one reference image, wherein the reference image and the reflected image are determined by two different spatial configurations, wherein the spatial configurations differ in their relative spatial constellations;

[0297] - Determine at least one reference feature in the reference image at at least one second image position corresponding to the ordinate z in the reference image;

[0298] - The relative spatial constellation is determined based on the vertical coordinate z, the position of the first image, and the position of the second image.

[0299] Example 36: The method according to the foregoing embodiments, wherein the method includes monitoring the relative space constellation, wherein the relative space constellation is repeatedly determined.

[0300] Example 37: The method according to any one of the foregoing two embodiments, wherein the method includes at least one temperature determination step, wherein at least one temperature value of the detector is determined.

[0301] Example 38: The method according to the foregoing embodiments, wherein the relative spatial constellation is determined by taking the temperature value into account and / or the relative spatial constellation is adapted to depend on the temperature value.

[0302] Example 39: The method according to any one of the foregoing four embodiments, wherein the detector includes at least one illumination source, wherein the method is used to determine the relative position of the sensor element and the illumination source.

[0303] Example 40: A method according to any one of the preceding five embodiments, wherein the detector includes at least one first sensor element and at least one second sensor element, wherein the first sensor element and the at least one second sensor element are positioned in different spatial configurations, wherein the method includes selecting at least one image determined by the first sensor element or the second sensor element as a reflected image, and selecting at least one image determined by the other sensor element of the first sensor element or the second sensor element as a reference image, wherein the method is used to determine the relative spatial constellation of the first sensor element and the second sensor element.

[0304] Example 41: A method for calibrating at least one detector according to any of the foregoing embodiments involving detectors, the method comprising the following steps:

[0305] - At least one reflected image of the object is determined by using at least one sensor element of a matrix having optical sensors, each of the optical sensors having a photosensitive area;

[0306] - Select at least one reflection feature of the reflected image at at least one first image location in the reflected image, and optimize at least one blur function f a To determine at least one ordinate z of the selected reflection feature;

[0307] - Provide at least one reference image, wherein the reference image and the reflected image are determined by two different spatial configurations, wherein the spatial configurations differ in their relative spatial constellations;

[0308] - Determine at least one reference feature in the reference image at at least one second image position corresponding to the ordinate z in the reference image;

[0309] -The relative spatial constellation is determined based on the vertical coordinate z, the position of the first image, and the position of the second image.

[0310] - The relative spatial constellation is stored as a calibration value in at least one data storage device of at least one evaluation unit.

[0311] Example 42: The method according to the foregoing embodiments, wherein the method includes at least one temperature determination step, wherein at least one temperature value of the detector is determined.

[0312] Example 43: According to the method described in the foregoing embodiments, the relative spatial constellation and / or the actual relative spatial constellation are determined taking into account the temperature value, and / or the relative spatial constellation and / or the actual relative spatial constellation are adjusted depending on the temperature value.

[0313] Example 44: The intended use of the detector (110) according to any one of the foregoing embodiments relating to the detector is selected from: position measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; logistics applications; tracking applications; outdoor applications; mobile applications; communication applications; photography applications; machine vision applications; robotics applications; quality control applications; and manufacturing applications. Attached Figure Description

[0314] Other optional details and features of the invention will become apparent from the description of the following preferred exemplary embodiments in conjunction with the dependent claims. In this context, certain features may be implemented in isolation or in combination with other features. The invention is not limited to exemplary embodiments. Exemplary embodiments are schematically illustrated in the accompanying drawings. In the various drawings, the same reference numerals refer to the same elements or elements having the same function, or elements that correspond to each other in terms of their function.

[0315] Specifically, in the attached diagram:

[0316] Figure 1 A first embodiment of the detector, detector system, camera, entertainment device, and tracking system according to the present invention is shown;

[0317] Figure 2 A second embodiment of the detector, detector system, camera, entertainment device, and tracking system according to the present invention is shown;

[0318] Figure 3A and Figure 3B An embodiment of the product concept is shown;

[0319] Figure 4 Another embodiment of the detector according to the invention is shown; and

[0320] Figure 5 This shows three scenarios for obtaining relative spatial constellations. Detailed Implementation

[0321] Figure 1 A first embodiment of a detector 110 for determining the position of at least one object 112 is illustrated in a highly schematic diagram. The detector 110 may specifically be embodied as a camera 114 and / or may be part of a camera 114. The camera 114 may be manufactured for imaging, particularly for 3D imaging, and may be manufactured for acquiring still images and / or image sequences (such as digital video clips). Other embodiments are possible. Figure 1 An embodiment of detector system 116 is further shown, which includes one or more beacon devices 118 in addition to at least one detector 110. In this example, the beacon devices may be attached to and / or integrated into object 112, and the location of the beacon devices should be detected by using detector 110. Figure 1 Further exemplary embodiments are shown below: a human-machine interface 120, which includes at least one detector system 116; and further, an entertainment device 122, which includes the human-machine interface 120. Figure 1 An embodiment of a tracking system 124 for tracking the position of object 112 is further shown, the tracking system including a detector system 116. The components of the device and system will be explained in more detail below.

[0322] In this exemplary embodiment, the object 112, whose position can be detected, may be designed as a sporting item and / or may form a control element or control device, whose position can be manipulated by the user 113. For example, the object 112 may be or may include a bat, racket, club, or any other sporting equipment and / or dummy sporting equipment. Other types of objects 112 are possible. Furthermore, the user 113 himself may be considered as object 112, whose position will be detected.

[0323] Figure 1Further exemplary embodiments of a scanning system 126 for scanning a scene including object 112 (such as for scanning object 112 and / or for determining at least one location of at least one object 112) are shown. The scanning system 126 includes at least one detector 110, and optionally at least one illumination source 128, and optionally at least one other illumination source (not described herein). The illumination source 128 is generally configured to emit at least one illumination beam, such as for illuminating at least one point, for example, a point located at one or more locations on beacon device 118 and / or on the surface of object 112. The scanning system 126 may be designed to generate an outline of the scene including object 112 and / or an outline of object 112, and / or may be designed to generate at least one piece of information regarding the distance between at least one point and the scanning system 126 (particularly detector 110) by using at least one detector 110.

[0324] Detector 110 includes at least one sensor element 130 of a matrix 132 having optical sensors 134. Each optical sensor 134 has a photosensitive area 136. The sensor element 130 can be formed as a single device or a combination of multiple devices. Matrix 132 can specifically be or can include a rectangular matrix having one or more rows and one or more columns. The rows and columns can specifically be arranged in a rectangular manner. However, other arrangements are also possible, such as triangular, circular, hexagonal, or other non-rectangular arrangements. As an example, a circular arrangement is also possible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, matrix 132 can be a single row of pixels. Other arrangements are also possible.

[0325] Specifically, the optical sensors 134 of matrix 132 may be equal in size, sensitivity, and one or more other optical, electrical, and mechanical characteristics. The photosensitive areas 136 of all the optical sensors 134 of matrix 132 may specifically be located in a common plane, preferably facing object 112, such that a light beam propagating from the object to detector 110 can generate a light spot on the common plane. The photosensitive areas 136 may specifically be located on the surface of the respective optical sensor 134. However, other embodiments are also possible. The optical sensor 134 may include, for example, at least one CCD and / or CMOS device. As an example, the optical sensor 134 may be part of or constitute a pixelated optical device. As an example, the optical sensor may be part of or constitute at least one CCD and / or CMOS device having a pixel matrix, with each pixel forming a photosensitive area 136.

[0326] The optical sensor 134 may specifically be or may include a photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, and most preferably a silicon photodetector. Specifically, the optical sensor 134 may be sensitive in the infrared spectral range. All optical sensors 134 of the matrix 132, or at least a group of optical sensors 134 of the matrix 132, may specifically be identical. Groups of identical optical sensors 134 of the matrix 132 may specifically be provided for different spectral ranges, or all optical sensors may be identical in spectral sensitivity. Furthermore, the optical sensors 134 may be identical in size and / or with respect to their electronic or photoelectric properties. The matrix 132 may consist of individual optical sensors 134. Thus, an inorganic photodiode matrix 132 may be constructed. However, alternatively, one or more commercially available matrices, such as CCD detectors (such as CCD detector chips), and / or CMOS detectors (such as CMOS detector chips), may be used.

[0327] The optical sensor 134 can form a sensor array or be part of a sensor array, such as the matrix described above. Therefore, as an example, the detector 110 can include an array of optical sensors 134, such as a rectangular array with m rows and n columns, where m and n are independently positive integers. Preferably, more than one column and more than one row are given, i.e., n>1, m>1. Therefore, as an example, n can be 2 to 16 or higher, and m can be 2 to 16 or higher. Preferably, the ratio of the number of rows to the number of columns is close to 1. As an example, n and m can be chosen such that 0.3≤m / n≤3, such as by choosing m / n = 1:1, 4:3, 16:9, or similar. As an example, the array can be a square array with equal numbers of rows and columns, such as by choosing m=2, n=2 or m=3, n=3, etc.

[0328] Matrix 132 may specifically be a rectangular matrix having at least one row, preferably multiple rows and columns. As an example, the rows and columns may be substantially vertically oriented. To provide a wide field of view, matrix 132 may specifically have at least 10 rows, preferably at least 50 rows, more preferably at least 100 rows. Similarly, matrix 132 may have at least 10 columns, preferably at least 50 columns, more preferably at least 100 columns. Matrix 132 may include at least 50 optical sensors 134, preferably at least 100 optical sensors 134, more preferably at least 500 optical sensors 134. Matrix 132 may include multiple pixels within a range of millions of pixels. However, other embodiments are possible.

[0329] exist Figure 1In the illustrated embodiment, detector 110 further includes an illumination source 138, which is the same as illumination source 128 in this embodiment. As an example, illumination source 138 may be configured to generate an illumination beam for illuminating object 112. Detector 110 may be configured such that the illumination beam propagates from detector 110 along the optical axis of detector 110 toward object 112. For this purpose, detector 110 may include at least one reflective element, preferably at least one prism, for deflecting the illumination beam along the optical axis.

[0330] Illumination source 138 may be adapted to generate at least one illumination pattern for illuminating object 112. Specifically, illumination source 138 may include at least one laser and / or laser source. Various types of lasers may be employed, such as semiconductor lasers. Alternatively or alternatively, non-laser light sources, such as LEDs and / or bulbs, may be used. The pattern may include multiple features. The pattern may include an arrangement of periodic or non-periodic features. The illumination pattern may include at least one pattern selected from the group consisting of: at least one point pattern, particularly a pseudo-random point pattern; at least one pattern including at least one known feature. For example, illumination source 138 may be adapted to generate and / or project point clouds. Illumination source 138 may include one or more of the following: at least one optical projector; at least one digital light processing (DLP) projector, at least one LCoS projector, at least one spatial light modulator; at least one diffractive optical element; at least one light-emitting diode array; at least one laser source array. Illumination source 138 may include at least one light source adapted to directly generate the illumination pattern. Illumination source 138 may include at least one optical projector adapted to generate point clouds such that the illumination pattern may include multiple point patterns. Illumination source 138 may include at least one mask adapted to generate an illumination pattern from at least one light beam generated by illumination source 138. Illumination source 138 may illuminate at least one object 112 with the illumination pattern. Illumination pattern may include a plurality of points as image features. These points are shown as light beam 140 emitted from illumination source 138.

[0331] Each optical sensor 134 can be designed to generate at least one sensor signal in response to illumination of its corresponding photosensitive region 136 by a beam of light 141 propagating from object 112 to detector 110.

[0332] Furthermore, sensor element 130 is adapted to determine at least one reflected image 142. Matrix 132 may include the reflected image 142. The reflected image 142 may include points as reflection features. These points are generated by a light beam 141 originating from at least one object 112.

[0333] Detector 110 may include at least one transmission device 144, which includes one or more of the following: at least one lens, for example, at least one lens selected from the group consisting of at least one focusing adjustable lens, at least one aspherical lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one reflector; at least one beam splitting element, preferably at least one of a beam splitter cube or a beam splitter mirror; at least one multi-lens system. In particular, transmission device 144 may include at least one collimating lens adapted to focus at least one object point in the image plane.

[0334] The detector 110 includes at least one evaluation device 146.

[0335] As summarized above, determining the location of object 112 and / or a portion thereof using detector 110 can be used to provide human-machine interface 120 to provide at least one piece of information to a machine (not shown). The machine may be a computer and / or may include a computer. Other embodiments are possible. Evaluation device 146 may even be fully or partially integrated into the machine, such as integrated into a computer.

[0336] The tracking system 124 includes a detector 110 and at least one tracking controller 147. The tracking controller 147 can be adapted to track a series of locations of the object 112 at specific points in time. The tracking controller 147 can be a standalone device and / or can be wholly or partially integrated into a machine, particularly a computer, and / or as... Figure 1 Among the 146 evaluation devices.

[0337] Similarly, as outlined above, the human-machine interface 120 can form part of the entertainment device 122. A machine (specifically a computer) can also form part of the entertainment device 122. Therefore, by means of a user 113 acting as an object 112 and / or by means of a user 113 processing a control device acting as an object 112, the user 113 can input at least one piece of information (such as at least one control command) into the computer, thereby altering entertainment functions, such as controlling the computer's processes.

[0338] The evaluation device 146 is adapted to select at least one reflection feature of the reflection image 142 at at least one first image location 148. The evaluation device 146 may be adapted to perform at least one image analysis and / or image processing to identify the reflection feature. The image analysis and / or image processing may use at least one feature detection algorithm. Image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image generated by a sensor signal and at least one offset; inverting a sensor signal by inverting an image generated by a sensor signal; forming a difference image between images generated by a sensor signal at different times; background correction; decomposition into color channels; decomposition into hue, saturation, and luminance channels; frequency decomposition; singular value decomposition; application of a Canny edge detector; application of a Laplacian operator with a Gaussian filter; application of a difference Gaussian filter; application of a Sobel operator; application of a Laplacian operator; application of a Scharr operator; application of a Prewitt operator; application of a Roberts operator; application of a Kirsch operator; application of a high-pass filter; application of blob analysis; application of an edge filter; application of a low-pass filter; application of a Fourier transform; application of a Radon transform; application of a Hough transform; application of a wavelet transform; thresholding; and creation of a binary image. The region of interest may be determined manually by the user or automatically, such as by identifying objects within the image generated by the optical sensor 134.

[0339] The evaluation device 146 is configured to optimize at least one fuzzy function f aThe evaluation device 146 is configured to determine at least one ordinate z of the selected reflective feature. Specifically, the evaluation device 146 can be configured to determine at least one distance estimate. The distance estimate can be at least one uncertainty interval defined by the ordinate z and the measurement uncertainty ± ε that determines the ordinate. The error interval ε can depend on the measurement uncertainty of the optical sensor and / or other parameters such as temperature, motion, etc. The measurement uncertainty of the optical sensor 134 can be predetermined and / or estimated and / or stored in at least one data storage unit of the evaluation device 146. For example, the error interval can be ±10%, preferably ±5%, more preferably ±1%. The determination of the distance estimate can produce a distance estimate with an error bar, which is typically larger than the error bar of the triangulation method. The ordinate z can be determined by using at least one convolution-based algorithm (such as a defocus depth algorithm). To obtain the distance to the reflective feature, the defocus depth algorithm estimates the defocus of the object. For this estimate, a blur function is assumed. Specifically, the blur function models the blur of the defocused object. At least one fuzzy function fa can be a function or composite function consisting of at least one function from the group consisting of: Gaussian function, sine function, rollerbox function, square function, Lorentz function, radial function, polynomial, Hermite polynomial, Zernike polynomial, Legendre polynomial.

[0340] Sensor element 130 may be adapted to determine at least one reflection pattern. Evaluation device 146 may be adapted to select at least one feature of the reflection pattern and optimize at least one ambiguity function f. a To determine the ordinate z of the selected feature of the reflection pattern.

[0341] The blur function can be optimized by changing at least one parameter of the blur function. The reflected image 142 can be the blurred image i. b The evaluation device 146 can be configured to evaluate based on the blurred image i b and fuzzy function f a Reconstruct the ordinate z. The blurred image i can be minimized by changing the parameter σ of the blur function. b and fuzzy function f a With at least one other image i' b The difference between convolutions (*), min||(i′) b *f a (σ(z))-i b The ordinate z is determined by σ(z), which is a set of distance-dependent blur parameters. Other images may be blurry or sharp. Blurry images i can be obtained by convolving with a known blur function. bAt least one other image is generated. Therefore, the defocus depth algorithm can be used to obtain a distance estimate of the reflection features. This distance estimate can be used to efficiently select the region in which the epipolar line is chosen, which will be outlined in more detail below. The distance can then be calculated using triangulation and the selected epipolar line. Unlike most triangulation methods, the determination of the distance estimate can be applied to a single feature of the reflection image. Therefore, the determination of the distance estimate can be used to accelerate triangulation methods by generating a smaller region that solves the correspondence problem.

[0342] Sensor element 130 may be adapted to determine at least one reflection pattern. The reflection pattern may include at least one feature corresponding to at least one feature of the illumination pattern. Compared to the illumination pattern, the reflection pattern may include at least one distortion pattern, wherein the distortion depends on the distance to the object, such as the surface characteristics of the object. As described above, evaluation device 146 may be adapted to select at least one feature of the reflection pattern and determine the ordinate z of the selected feature of the reflection pattern.

[0343] The evaluation device 146 is adapted to determine at least one reference feature in at least one reference image at at least one second image location corresponding to at least one reflection feature in the reference image. The reference image and the reflection image are determined using two different spatial configurations. The difference in the spatial configurations lies in their relative spatial constellations. Figure 1 In the illustrated embodiment, the reference image may be an image of an illumination pattern at the location of illumination source 128 in the image plane. Evaluation device 146 may be adapted to perform image analysis and identify features of the reference image. Evaluation device 146 may be adapted to identify at least one reference feature in the reference image having a ordinate substantially the same as a selected reflective feature. The reference feature corresponding to the reflective feature may be determined using epipolar geometry. Epipolar geometry may assume that the reference image and the reflective image are, for example, images of an object 112 determined at different spatial locations and / or spatial orientations at a fixed distance during the recording of the reference image and the reflective image. This distance may be a relative distance, also represented as a baseline. Evaluation device 146 may be adapted to determine an epipolar line in the reference image. The relative positions of the reference image and the reflective image may be known. For example, the relative positions of the reference image and the reflective image may be stored in at least one storage unit of the evaluation device. Evaluation device 146 may be adapted to determine a straight line extending from a selected reflective feature of the reflective image to a real-world feature from which it originates. Thus, the straight line may include possible object features corresponding to the selected reflective feature. The straight line and the baseline cross the epipolar plane. Since the reference image is located at a different spatial constellation than the reflected image, the corresponding possible object features can be imaged along a straight line in the reference image called the epipolar line. The epipolar line can be the intersection of the epipolar plane and the reference image. Therefore, the features of the reference image corresponding to the selected features of the reflected image lie on the epipolar line.

[0344] Depending on the distance to object 112, a reference feature corresponding to the second image position of the reflection feature is displaced within the reference image compared to the first image position. The reference image may include at least one displaced region in which the reference feature corresponding to the selected reflection feature can be imaged. The displaced region may include only one reference feature. Evaluation device 146 may be configured to determine at least one longitudinal region, wherein the longitudinal region is given by a ordinate z and an error interval ±ε. Evaluation device 146 may be configured to determine at least one displaced region in the reference image corresponding to the longitudinal region. The displaced region may extend along an epipolar line. Evaluation device 146 may be adapted to determine the reference feature along the epipolar line. Evaluation device 146 may be adapted to determine the ordinate z and error interval ±ε for the reflection feature to determine the displaced region corresponding to z±ε along the epipolar line. Evaluation device 146 may be adapted to match the selected reflection feature with at least one reference feature within the displaced region. Evaluation device 146 may be adapted to match the selected feature of the reflection image with the reference feature within the displaced region by considering the determined ordinate z using at least one evaluation algorithm. The evaluation algorithm may be a linear scaling algorithm.

[0345] Evaluation device 146 may be adapted to determine the displacements of a reference feature and a reflection feature. Evaluation device 146 may be adapted to determine the displacements of a matched reference feature and a selected reflection feature. Evaluation device 146 may be adapted to determine longitudinal information of the matched feature using a predetermined relationship between the ordinate and the displacement. For example, the longitudinal information may be a distance value. Evaluation device 146 may be adapted to determine the predetermined relationship using a triangulation method. Given that the position of the selected reflection feature and the position of the matched reference feature in the reflection image and / or the relative displacement of the selected reflection feature to the matched reference feature are known, the ordinate of the corresponding object feature can be determined by triangulation. Therefore, evaluation device 146 may be adapted to select, for example, subsequent and / or column-by-column reflection features and use triangulation to determine the corresponding distance value for each potential position of the reference feature. The displacement and the corresponding distance value may be stored in at least one storage device of evaluation device 146. As an example, evaluation device 146 may include at least one data processing device, such as at least one processor, at least one DSP, at least one FPGA, and / or at least one ASIC. Furthermore, to store at least one predetermined or determinable relationship between the ordinate z and the displacement, at least one data storage device may be provided, such as for providing one or more lookup tables to store the predetermined relationship.

[0346] Evaluation device 146 can be adapted to determine the relative spatial constellation based on the ordinate z and the positions of the first and second images. As summarized above, epipolar geometry may require a good understanding of the relative spatial constellations of the reflected and reference images, particularly the baseline. However, the relative spatial constellations of detector components such as illumination source 138 and sensor element 130 may be unknown and / or may change during measurement time, for example, due to thermal effects. The ordinate z determined by using at least one defocus depth algorithm can be used to calibrate and / or recalibrate the triangulation system. As summarized above, evaluation device 146 can be adapted to determine the displacement of the reference and reflected features. Evaluation device 146 can be adapted to use the triangulation of object 112 to determine the ordinate z. triang The predefined relationship between the displacement and the triangulation determines at least one triangulation ordinate z of object 112. triang Triangulation of the ordinate z triang The positions of the first and second images can be determined using epipolar geometry (assuming a fixed relative spatial constellation and employing predefined and / or predetermined values ​​of the relative spatial constellation). Specifically, the predefined relationship can depend on the relative spatial constellation. The evaluation device can be adapted to store the predefined relationship. Evaluation device 146 can be adapted to compare the ordinate z-coordinate with the triangulated ordinate z-coordinate. triang Considering the determined relative spatial constellation, the evaluation device 146 can be adapted to determine the actual relationship between the ordinate z and the displacement. The evaluation device 146 can be adapted to adjust a predefined relationship based on the actual relationship. The evaluation device 146 can be adapted to replace the predefined relationship with the actual relationship, particularly a stored predefined relationship, and / or the evaluation can be adapted to determine a moving average and replace the predefined relationship with the moving average. The evaluation device 146 can be adapted to determine the ordinate z and the triangulated ordinate z. triang The evaluation device 146 can be adapted to compare the determined difference with at least one threshold and adjust a predefined relationship if the determined difference is higher than or equal to the threshold. The evaluation device 146 can be adapted to determine a relative spatial constellation based on the actual relationship and the ordinate z. For example, the illumination source 138 and the sensor element 130 can be separated by a baseline b, d is the displacement on the sensor, f is the focal length of the detector's transfer device, and β is the angle between the illumination source and the baseline. Typical values ​​for the baseline and displacement are discussed in Kurt Konolige et al., A Low-Cost Laser Distance Sensor, 2008 IEEE International Conference on Robotics and Automation, Pasadena, CA, USA, May 19-23, 2008. For β = 90°,

[0347] as well as,

[0348]

[0349] Therefore, given that the distance to the object (i.e., the ordinate z) is known, z triang z can be used instead, and the corrected baseline b cor It can be calculated using the following formula

[0350]

[0351] For β less than 90°, it is Therefore, the baseline b is corrected. cor It can be calculated using the following formula

[0352]

[0353] And the angle β can be determined from the following formula.

[0354]

[0355] Since β and b may vary simultaneously, these two values ​​can be determined using subsequent measurements. Therefore, measuring the ordinate z of a feature point can be used to correct predefined relationships; furthermore, triangulation of the ordinate z... triang The distance from the sensor element to the object (i.e., determined by triangulation) is known. The evaluation device 146 can be adapted to use the vertical coordinate z and the triangulated vertical coordinate z. triang To determine and / or correct and / or calibrate the values ​​of the relative spatial constellation (such as baseline values). Various sensor signals can be used within the mathematical model, which can be selected from Kalman filters, linear quadratic estimators, Kalman-Busch filters, Stratonovich-Kalman-Busch filters, Kalman-Busch-Stratonovich filters, minimum variance estimators, Bayesian estimators, optimal linear unbiased estimators, invariant estimators, Wiener filters, etc., to account for the measurement errors and inaccuracies inherent in each sensor signal. The fusion of these sensor signals within the mathematical model (such as Kalman filters), such as measurements of the relative spatial constellation and / or ordinate, can produce improved estimates.

[0356] The ordinate z can be determined for multiple feature points, and the ordinate z can be triangulated. triang This is particularly important for obtaining statistical confirmation values ​​of calibration relationships and / or calibration relative space constellations. Since relative space constellations do not change abruptly, such statistical assessments may be well-suited for this purpose.

[0357] Figure 2A second embodiment is shown, comprising detector 110, detector system 116, camera 114, entertainment device 122, and tracking system 124. References regarding detector 110, detector system 116, camera 114, entertainment device 122, and tracking system 124 are provided. Figure 1 The description. Besides Figure 1 In addition, in this embodiment, detector 110 may include two sensor elements 130, a first sensor element 150, and a second sensor element 152. The first sensor element 150 and the second sensor element 152 may be connected via a mechanical connector 156. The mechanical connector 156 may be adjustable and / or non-permanent. Some points on object 112 may be illuminated by an optional illumination source 138 and may be detected by both sensor elements 150 and 152. Evaluation device 146 may be configured to optimize the ambiguity function f. a The ordinate z of object 112 is determined. The first sensor element 150 and the second sensor element 152 can be adapted to image the object features (particularly points illuminated by an optional illumination source 138). The image of the first sensor element 150 or the second sensor element 152 can be selected as a reflective image, wherein another corresponding image from the other sensor element can be selected as a reference image. (See regarding...) Figure 1 The evaluation device 146 is adapted to determine at least one reflection feature at at least one first image location 148 and to determine a reference feature corresponding to the reflection feature at at least one second image location 154. (As per...) Figure 1 The evaluation device 146 can be adapted to determine a relative spatial constellation, particularly a baseline, based on the vertical coordinate z and the positions of the first and second images.

[0358] The determination of the ordinate z can be performed on a single reflection feature or multiple or all reflection object features in the image determined by the first sensor element 150 and / or the second sensor element 152. Feature points not illuminated by the optional illumination source 138 can be used to calculate additional ordinate z and / or can be used to calculate attitude estimation.

[0359] Figure 3A and Figure 3B An example of the product concept is shown. Figure 3AThe product package 158 is shown, which includes components of detector 110, specifically sensor element 130, delivery device 144, illumination source 138, mechanical connector 156, evaluation device 146, and multiple cables 160 for connecting the various components. At least one delivery device 144 and at least one sensor element 130 can be pre-assembled in the product package 158. Other detector components within the product package 158 can be stored as separate, unassembled components. The user can remove the components from the package and connect them via the mechanical connector 156 and cables 160. The evaluation device 146 can be adapted to mount detector 110. The assembled detector 110 is in... Figure 3B The text is a mix of Chinese characters and symbols, and doesn't form coherent sentences. A direct translation isn't possible without further Figure 2 The detector settings are described in [the document / reference]. For example, regarding [the following]... Figure 2 As outlined above, the evaluation device 146 can be adapted to determine the ordinate z of at least one reflection feature in a reflection image determined by one or two sensor elements 130 using at least one defocus depth algorithm. As outlined above, the evaluation device 146 can be adapted to use the ordinate z to determine the relative spatial constellation of the illumination source 138 and the corresponding sensor element 130.

[0360] Figure 4 Another embodiment of detector 110 is shown, particularly for use in mobile systems employing a 3D sensing method based on structure recovery from motion. The evaluation device can be adapted to set up mobile systems with flexible relative constellations, particularly flexible baselines. Figure 1 The sensor element 130 and the optional irradiation source 138 shown are mechanically connected in opposite directions, as... Figure 4 The sensor element 130 and optional illumination source 138 of the detector 110 shown may not be mechanically connected. The position and / or orientation of the optional illumination source 138 may be varied with respect to the sensor element 130. The detector 110 may be adapted to determine a first image in a first spatial configuration 162. The optional illumination source 138 may be adapted to illuminate some points that can be detected as reflective features in the first image. The evaluation device 146 may be adapted to determine the ordinate z of these points. Furthermore, additional features, particularly points not illuminated by the illumination source 138, may be detected and / or imaged by the sensor element. Alternatively, embodiments are possible in which the detector may not include the optional illumination source 138. Feature points may be designed such that reflective images can be generated passively from them. For example, a feature point may be a white circle.

[0361] Detector 110 can be adapted to use matrix 132 of the same optical sensor 134 to record the reflected image and the reference image. Specifically, as... Figure 4As shown, sensor element 130 moves, for example, from a first spatial configuration 162 or is moved to at least a second spatial configuration 164 at a constant or variable speed. Illumination source 138 may be adapted to illuminate some points that can be detected as reflective features in a second image via the second spatial configuration 164. Evaluation device 146 may be adapted to determine the ordinate z of these points. Furthermore, additional features, particularly points not illuminated by illumination source 138, may be detected and / or imaged by sensor element 130.

[0362] Detector 110 can be adapted to determine, in particular, multiple images, wherein one image can be selected as a reflective image and another image can be selected as a reference image. Evaluation device 146 can be adapted to perform 3D sensing methods, such as reconstructing structure or pose estimation from motion, as described in Ramalingam et al., “Pose Estimation using BothPoints and Lines for Geo-Localization”, published in Robotics and Automation (ICRA), 2011 IEEE International Conference on Robotics and Automation, Publisher: IEEE ISBN: 978-1-61284-385-8. The term “reconstructing structure from motion” will be used as a synonym for both reconstructing structure from motion and reconstructing shape from motion. Evaluation device 146 can be adapted to estimate the pose of sensor element 130 using unilluminated feature points and to estimate the relative spatial constellation corresponding to a scaling factor. The scaling factor can be obtained from the feature points from which the ordinate z is calculated.

[0363] The lack of a fixed relative spatial constellation (such as a baseline) for reference and reflected images can lead to so-called scale drift and loss of distance determination accuracy, or may not allow for absolute distance measurements without additional information. Specifically, motion-reconstruction structure and pose estimation algorithms can determine longitudinal and lateral information, such as object size, dimensions, distance, and / or orientation, which is equivalent to a scaling factor that scales arbitrary distance units within the evaluation device 146 to an absolute real-world distance scale. In particular, motion-reconstruction structure and pose estimation algorithms require additional image reconstruction information to scale the image information to an absolute distance scale. The evaluation device 146 may be adapted to determine at least one scaling factor for the relative spatial constellation. For example, in a structured light system consisting of at least one image sensor and at least one illumination source, the baseline is elongated due to increased system temperature, resulting in an increased distance between at least one image sensor and at least one illumination source, while the focal length of the lens and the distance from the lens to the sensor remain fixed. In this example, when comparing two objects at the same position in the reflected image, the first object is measured using the original baseline in the first measurement, while the second object is measured using an extended baseline in the second measurement. The object measured with the extended baseline is farther than the object measured with the original baseline. The angle between the baseline and the straight line connecting the feature point in the reflected image to the corresponding feature point on the object itself is the same for both objects, allowing the principle of similar triangles to be used to compare the two measurements. The distance to the object is measured along the straight line. In this example, according to the principle of similar triangles, the quotient of the distance from the object to the lens and the baseline is the same for both the measurement of the original baseline and the measurement of the extended baseline. Therefore, the scaling factor for scaling the original baseline to the extended baseline is the same as the scaling factor for scaling the original object distance to the increased object distance. Therefore, according to the principle of similar triangles, the scaling factor of the baseline also scales the distance, specifically the ordinate z. The evaluation device 146 can be adapted to optimize the blur function f. a The absolute measurement of the ordinate z is performed. The evaluation device 146 can be adapted to determine the scaling factor based on the ordinate z. The determination of the scaling factor can be further refined by using sensor data from the inertial measurement unit.

[0364] Evaluation device 146 can be adapted to determine, based on corresponding combined sensor signals, the ordinate z of at least one feature point in at least one image recorded by sensor element 130 in a first spatial configuration 162, and from there determine a scaling factor. The scaling factor can be maintained for the remaining measurements and / or as long as at least one feature point can be traced from one image to another and / or can be recalculated during measurement. For example, the scaling factor can be determined in each image recorded by sensor element 130. This ensures statistical verification and consistent measurement of the scaling factor.

[0365] The scaling factor can be determined based on a single measurement point of the image and / or the scaling factor can be determined based on multiple measurements. In particular, the evaluation device can be adapted to determine a medium scaling factor.

[0366] For example, Figure 4 In the embodiment shown, the detector can be used in a mobile phone or a smartphone. The optical sensor 134 can be an integrated CMOS commonly used for photos or videos. The illumination source 138 can be an integrated laser for autofocus or an additional illumination source attached and connected via a headset jack or the like. Additional means for attachment can be used. The distance between the illumination source 138 and the optical sensor 134 can depend on the type of mobile phone or smartphone and can be determined with the aid of the present invention. For example, the relative spatial constellation can be determined using the ordinate z, and structure from motion can be used for further distance determination.

[0367] Figure 5 Three cases of obtaining the relative spatial constellation are shown. Assigning one image as the reflected image and one image as the reference image is completely interchangeable, and the assignment is only for the convenience of discussion and should not limit its generality.

[0368] In Figure 5 In the first case shown, the first reflected image 166 and the reference image 168 are determined. The relative spatial constellation can be known, for example, from factory calibration. The distance of the object 112 can be obtained by determining the image positions of the corresponding feature points 170 and 172, determining the displacement of the feature points, and calculating the distance of the object via triangulation using a predetermined relationship. This distance determination can be performed or used in structured light and stereo systems.

[0369] Compared with the first case, in Figure 5 In the second case shown, for example due to temperature effects, the baseline b has been elongated. The feature point 170 at the same image position in the first reflected image 166 and the second reflected image 174 corresponds to an increased object distance in the case of the elongated baseline. The corresponding feature point 172 in the reference image 168 is the same as in the first case because the focal length remains unchanged. Since the length of the baseline has increased and the corresponding feature points have not changed, all angles in the first and second cases are the same. Since all angles are the same, the principle of similar triangles yields that the quotient of the two baselines is equal to the quotient of the two distances. The correct distance in the second case can be determined by using at least one ordinate z determined by optimizing the ambiguity function f a and the triangulation ordinate z triang to determine. The quotient z / z triangA scaling factor can be provided to scale the distance determined by triangulation from the first case to the second case. This determination of the relative spatial constellation can be universally applied to structured light and stereo systems. An attitude estimation algorithm applied to at least one of the reference image 168, the first reflected image 166, or the second reflected image 174 allows estimation in both cases whether only the baseline or other parameters such as the orientation angle have changed.

[0370] In addition, Figure 5 In the third case, the changed baseline and orientation are shown. Specifically, the third reflection image 176 can be determined. Compared to the first case, the baseline and orientation angle have changed. The attitude estimation or structure-from-motion algorithm may produce the fact that the orientation angle has changed using at least six feature points. Furthermore, the attitude estimation or structure-from-motion algorithm can be able to determine the relative distance and orientation angle between the feature points and the detector. However, since the baseline length is unknown, the problem of determining the absolute distance after determining the relative distance and angle is similar to that of the second case. Therefore, after determining the relative distance and orientation angle, a single distance measurement for the ordinate z of a feature point will be sufficient to determine the scaling factor to scale the relative distance to an absolute value. As an example, the original baseline length or approximate baseline length can be used to calculate the triangular distance using the new orientation angle. The scaling factor is z / z triang Given a case similar to the second one, this determination of the relative spatial constellation can be applied to three-dimensional, structured light, and structure recovery systems from motion.

[0371] Reference Number List

[0372] 110 detector

[0373] 112 Objects

[0374] 113 users

[0375] 114 cameras

[0376] 116 Detector System

[0377] 118 Beacon Equipment

[0378] 120 Human-Machine Interface

[0379] 122 Entertainment Equipment

[0380] 124 Tracking System

[0381] 126 Scanning System

[0382] 128 Irradiation Source

[0383] 130 sensor element

[0384] 132 matrix

[0385] 134 Optical Sensor

[0386] 136 Photosensitive areas

[0387] 138 Irradiation Source

[0388] 140 beams

[0389] 141 beams

[0390] 142 Reflection Image

[0391] 144. Transfer equipment

[0392] 146 Evaluation Equipment

[0393] 147 Tracking Controller

[0394] 148 First image position

[0395] 150 First sensor element

[0396] 152 Second sensor element

[0397] 154 Second image position

[0398] 156 Mechanical Connector

[0399] 158 Product Packaging

[0400] 160 cable

[0401] 162 First Space Configuration

[0402] 164 Second Space Configuration

[0403] 166 First Reflection Image

[0404] 168 Reference Images

[0405] 170 feature points

[0406] 172 feature points

[0407] 174 Second Reflection Image

[0408] 176 Third Reflection Image

Claims

1. A detector (110) for determining the location of at least one object (112), the detector (110) comprising: - At least one sensor element (130) having a matrix (132) of optical sensors (134), each of the optical sensors (134) having a photosensitive region (136), wherein, The sensor element (130) is configured to determine at least one reflected image (142); - At least one evaluation device (146), wherein the evaluation device (146) is configured to select at least one reflection feature of the reflection image (142) at at least one first image location (148) in the reflection image (142), wherein the evaluation device (146) is configured to optimize at least one blur function f a To determine at least one ordinate z of a selected reflection feature, wherein the evaluation device (146) is configured to determine at least one reference feature in at least one reference image (168) at at least one second image position (154) in the reference image (168), wherein the at least one reference feature corresponds to the at least one reflection feature, wherein the reference image (168) and the reflection image (142) are determined in two different spatial configurations, wherein the difference between the spatial configurations lies in the relative spatial constellation, wherein the relative spatial constellation refers to the relative alignment of the reference image and the reflection image in space, wherein the evaluation device (146) is configured to determine the relative spatial constellation based on the ordinate z, the first image position (148) and the second image position (154).

2. The detector (110) according to claim 1, wherein, The ordinate z is determined by using at least one convolution-based algorithm.

3. The detector (110) according to claim 1, wherein, The vertical coordinate z is determined by using a defocus depth algorithm.

4. The detector (110) according to claim 2 or 3, wherein, The fuzzy function is optimized by changing the parameters of the at least one fuzzy function.

5. The detector (110) according to claim 4, wherein, The reflected image (142) is a blurred image i b The evaluation device (146) is configured to evaluate the blurred image i b and the fuzzy function f a To reconstruct the ordinate z.

6. The detector (110) according to claim 5, wherein, The blurred image i is minimized by changing the parameter σ of the blur function. b With the fuzzy function f a And other images i' b The difference between the convolutions is used to determine the ordinate z. min||(i′ b *f a (σ(z))-i b )||。 7. The detector (110) according to claim 6, wherein, The at least one fuzzy function f a It is a function or composite function consisting of at least one function from a group including the following: Gaussian function, sine function, parabolic cylindrical function, square function, Lorentz function, radial function, Hermite polynomial, Zernike polynomial, Legendre polynomial.

8. The detector (110) according to claim 7, wherein, The relative spatial constellation is selected from at least one constellation from the group consisting of: relative spatial orientation; relative angular position; relative distance; relative displacement; relative movement.

9. The detector (110) according to claim 8, wherein, The detector (110) includes at least two sensor elements (130) separated by a relative spatial constellation, wherein at least one first sensor element (150) is adapted to record the reference image (168), and at least one second sensor element (152) is adapted to record the reflected image (142).

10. The detector (110) according to claim 9, wherein, The detector (110) is configured to use a matrix (132) of the same optical sensors (134) at different times to record the reflected image (142) and the reference image (168).

11. The detector (110) according to claim 10, wherein, The evaluation device (146) is configured to determine at least one scaling factor for the relative space constellation.

12. The detector (110) according to claim 11, wherein, The evaluation device (146) is configured to determine the displacement of the reference feature relative to the reflection feature, wherein the evaluation device (146) is configured to use triangulation of the object to measure the ordinate z. triang A predefined relationship between the displacement and the object is used to determine at least one triangulated ordinate z of the object. triang The evaluation device (146) is configured to consider the determined relative spatial constellation and determine the actual relationship between the ordinate z and the displacement, wherein the evaluation device (146) is configured to adjust the predefined relationship based on the actual relationship.

13. The detector (110) according to claim 12, wherein, The evaluation device (146) is configured to replace the predefined relationship with the actual relationship, and / or the evaluation is configured to determine a moving average and replace the predefined relationship with the moving average.

14. The detector (110) according to claim 12 or 13, wherein, The evaluation device (146) is configured to determine the vertical coordinate z and the triangulation coordinate z. triang The difference between the two, wherein the evaluation device (146) is configured to compare the determined difference with at least one threshold and adjust the predefined relationship if the determined difference is higher than or equal to the threshold.

15. The detector (110) according to claim 14, wherein, The evaluation device (146) is configured to determine an estimate of the corrected relative spatial relationship using a mathematical model, the mathematical model including parameters such as: various sensor signals and / or positions and / or image positions and / or system characteristics and / or ordinates, displacement d on the sensors, focal length f of the transmission device, temperature, and z. triang , baseline b, angle β between the illumination source and the baseline, ordinate z, wherein the mathematical model is at least one mathematical model selected from the group consisting of: Kalman filter, linear quadratic estimator, Kalman-Busch filter, Stratonovich-Kalman-Busch filter, minimum variance estimator, Bayesian estimator, optimal linear unbiased estimator, invariant estimator, Wiener filter.

16. The detector (110) according to claim 15, wherein, The evaluation device (146) is configured to determine at least one longitudinal region, wherein the longitudinal region is given by the ordinate z and the error interval ±ε, wherein the evaluation device is configured to determine at least one displacement region in the reference image (168) corresponding to the longitudinal region, wherein the evaluation device (146) is configured to determine the epipolar line in the reference image (168), wherein the displacement region extends along the epipolar line, wherein the evaluation device (146) is configured to determine the reference feature along the epipolar line corresponding to the ordinate z, and determine the range of the displacement region along the epipolar line corresponding to the error interval ±ε.

17. The detector (110) according to claim 16, wherein, The evaluation device (146) is configured to perform the following steps: - Determine the displacement region for the second image position (154) for each reflection feature; - Assign epipolar lines to the displacement regions of each reflection feature by assigning epipolar lines closest to the displacement region and / or within the displacement region and / or along a direction orthogonal to the epipolar lines. - At least one reference feature is assigned and / or determined for each reflection feature by assigning a reference feature that is closest to the assigned displacement region and / or along the assigned epipolar line within the assigned displacement region.

18. The detector (110) according to claim 16 or 17, wherein, The evaluation device (146) is configured to match the reflection feature with at least one reference feature within the displacement region.

19. A detector system (116) for determining the location of at least one object (112), the detector system (116) comprising at least one detector (110) according to any one of the preceding claims, the detector system (116) further comprising at least one beacon device (118) adapted to direct at least one light beam toward the detector (110), wherein, The beacon device (118) is at least one of the following: attachable to the object (112), holdable by the object (112), and integrated into the object (112).

20. A human-machine interface (120) for exchanging at least one piece of information between a user (113) and a machine, wherein, The human-machine interface (120) includes at least one detector system (116) according to claim 19, wherein the at least one beacon device (118) is adapted to be at least one of the following: directly or indirectly attached to the user (113) and held by the user (113), wherein the human-machine interface (120) is designed to determine at least one location of the user (113) by means of the detector system (116), wherein the human-machine interface (120) is designed to assign at least one piece of information to the location.

21. An entertainment device (122) for performing at least one entertainment function, wherein, The entertainment device includes at least one human-computer interface (120) according to claim 20, wherein the entertainment device (122) is designed to enable a player to input at least one piece of information via the human-computer interface (120), wherein the entertainment device (122) is designed to change the entertainment function according to the information.

22. A tracking system (124) for tracking the position of at least one movable object, the tracking system (124) comprising at least one detector system (116) according to claim 19 relating to a detector system, the tracking system (124) further comprising at least one tracking controller, wherein, The tracking controller is adapted to track a series of locations of the object at a specific point in time.

23. A scanning system (126) for determining the depth profile of a scene, the scanning system (126) comprising at least one detector (110) according to any one of claims 1 to 18 relating to a detector, the scanning system (126) further comprising at least one illumination source (128) adapted to scan the scene with at least one light beam.

24. A camera (114) for imaging at least one object (112), said camera (114) comprising at least one detector (110) according to any one of claims 1 to 18 relating to a detector.

25. An inertial measurement unit for use in electronic devices, wherein, The inertial measurement unit is adapted to receive data determined by at least one detector (110) according to any one of claims 1 to 18 relating to a detector, wherein the inertial measurement unit is further adapted to receive data determined by at least one other sensor selected from the group consisting of: wheel speed sensor, turning rate sensor, tilt sensor, orientation sensor, motion sensor, magnetohydrodynamic sensor, force sensor, angle sensor, angular rate sensor, magnetic field sensor, magnetometer, accelerometer; wherein the inertial measurement unit is adapted to determine at least one characteristic of the electronic device by evaluating data from the detector and the at least one other sensor, the characteristic selected from the group consisting of: position in space, relative or absolute motion in space, rotation, acceleration, orientation, angular position, tilt, turning rate, speed.

26. The inertial measurement unit according to claim 25, wherein, The at least one other sensor also includes a gyroscope.

27. A method for determining the location of at least one object (112) using at least one detector (110) according to any one of claims 1 to 18 involving a detector, the method comprising the steps of: - At least one reflected image (142) of the object (112) is determined by using at least one sensor element (130) of a matrix (132) having optical sensors (134), each of the optical sensors (134) having a photosensitive area (136); - Select at least one reflection feature of the reflected image (142) at at least one first image location (148) in the reflected image (142), and optimize at least one blur function f a To determine at least one ordinate z of the selected reflection feature; - Provide at least one reference image (168), wherein the reference image (168) and the reflected image (142) are determined by two different spatial configurations, wherein the spatial configurations differ in their relative spatial constellations; - Determine at least one reference feature in the reference image (168) at at least one second image position (154) corresponding to the vertical coordinate z in the reference image (168); - The relative spatial constellation is determined based on the vertical coordinate z, the first image position (148), and the second image position (154).

28. A method for calibrating at least one detector (110) according to any one of claims 1 to 18 relating to a detector, the method comprising the steps of: - At least one reflected image (142) of the object (112) is determined by using at least one sensor element (130) of a matrix (132) having optical sensors (134), each of the optical sensors (134) having a photosensitive area (136); - Select at least one reflection feature of the reflected image (142) at at least one first image location (148) in the reflected image (142), and optimize at least one blur function f a To determine at least one ordinate z of the selected reflection feature; - Provide at least one reference image (168), wherein the reference image (168) and the reflected image (142) are determined by two different spatial configurations, wherein the spatial configurations differ in their relative spatial constellations; - Determine at least one reference feature in the reference image (168) at at least one second image position (154) corresponding to the vertical coordinate z in the reference image (168); -The relative spatial constellation is determined based on the vertical coordinate z, the position of the first image, and the position of the second image. - The relative spatial constellation is stored as a calibration value in at least one data storage device of at least one evaluation unit.

29. The use of the detector (110) according to any one of claims 1 to 18 relating to the detector is selected from: position measurement in traffic technology; entertainment applications; security applications; monitoring applications; human-machine interface applications; logistics applications; tracking applications; outdoor applications; mobile applications; communication applications; photography applications; machine vision applications; robotic applications; quality control applications; and manufacturing applications.

30. The use according to claim 29, wherein, The uses also include: security applications.

Images