Automatic calibration from epipolar distance in projected patterns

Abstract

A detector (110) is provided for determining a position of at least one object (112). The detector (110) comprises: at least one projector (122) for illuminating the object (112) with at least one illumination pattern (124), the illumination pattern (124) including a plurality of illumination features (125); at least one sensor element (114) having a matrix (116) of optical sensors (118), each having a photosensitive area (120), each optical sensor (118) designed to generate at least one sensor signal in response to illumination of its respective photosensitive area (120) by a reflected light beam propagating from the object (112) to the detector (110), the sensor element being configured to determine at least one reflection image (142) comprising a plurality of reflection features, each of the reflection features comprising a beam profile; at least one evaluation device (144) configured to determine initial distance information of the reflection features by analysis of a beam profile of each of said reflection features, the analysis of the beam profile comprising evaluating a combined signal Q from each of the sensor signals, the evaluation device comprising: a) (150) matching reflectance features to reference features of a reference image, taking into account initial distance information, thereby determining pairs of matched reflectance features and reference features; b) (152) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; c) determining the epipolar line distance d of the matched reflection feature to the (154) epipolar line; and d) (156) Evaluating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; and e) (158) determining at least one correction to a rotation and / or translation of the reflected image (142) in response to the geometric pattern; and an evaluation device (144) configured to perform a calibration method including:

Description

[Technical field]

[0001] The invention relates to a detector for determining the position of at least one object and a method for calibrating said detector. The invention further relates to various uses of the detector. The device, method and use according to the invention may be employed in particular in various areas of photography, documentation or technical purposes, for example digital photography or video photography for daily life, gaming, traffic technology, production technology, security technology, art, medical technology or in science. Furthermore, the invention may be used in particular for scanning one or more objects and / or for scanning scenery, for example for generating a depth profile of the object or a depth profile of the scenery, for example in the fields of architecture, metrology, archaeology, art, medicine, engineering or manufacturing. However, other applications are possible. [Background technology]

[0002] Active triangulation systems typically include at least one camera and at least one light projector, e.g., a structured light system. Other triangulation systems, such as stereo cameras, may include at least two cameras. For proper 3D reconstruction by triangulation, knowledge of the position and rotation of components such as the camera and the projector is mandatory. In addition, 3D reconstruction by triangulation also requires a resolved correspondence of important points on the scene, e.g., the laser spot, the light spot of the projector, or the detected edges captured by the camera. The 3D position can be calculated by a known translation and relative rotation of the camera to the projector. This parameter defines the external calibration of the triangulation system. Thus, the 3D measurement result quality depends on the external calibration. Depending on the hardware, an already calibrated system may degenerate due to physical stress or temperature shifts, i.e., relative position and rotation changes in time. This may result in erroneous measurement results. Obviously, the system may be repaired by an additional new calibration process. Depending on the application, this may be time consuming and impractical. The calibration process may be based on capturing a static scene with a defined target at a known distance.

[0003] The concept of recalibration algorithms already exists, e.g., E. Rehder et al., "Online Stereo Camera Calibration From Scratch", June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI: 10.1109 / IVS.2017.7995952, and T. Dang, "Continuous Stereo Self-Calibration by Camera Parameter Tracking", August 2009 IEEE Transactions on Image Processing 8(7):1536-50, DOI: 10.1109 / TIP.2009.2017824.

[0004] Those recalibration approaches are based on finding feature correspondences that should satisfy a system of equations (e.g. epipolar conditions) with respect to external parameters. A famous example is the 8-point algorithm. For example, in the case of a 3D measurement system with one camera and a laser dot projector, it can be assumed that the laser spots on the captured camera image are assigned exactly to the laser grid. That means that the correspondences are found exactly. The relationship of the positions of the laser spots on the camera image and the reference laser grid can be used to obtain a system of linear equations. This linear equation requires at least 8 corresponding laser spots to the reference grid. This equation can be solved by a least squares fit. Adept usage of singular value decomposition can determine the rotation and translation of the camera and the laser projector. However, if the correspondences are incorrect, the results can be very poor. Proper outlier detection of incorrect correspondences can be very important for that kind of method.

[0005] This procedure can also work for stereo measurements, however it may be necessary to find corresponding features in both camera images, e.g. edges or corners in the images, whose location can then be used to determine, for example, the 8-point method.

[0006] Generally, the third reconstruction method for structured light or stereo may require an externally calibrated system. It means that the search for correspondence is based on epipolar lines. However, this kind of search is only one-dimensional. In the case of an uncalibrated system, the search for correspondence can no longer work on one-dimensional epipolar lines. Conventional recalibration methods would search for correspondence in the two-dimensional image domain. This may be followed by an additional search algorithm for correspondence on top of the three-dimensional reconstruction. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] E. Rehder et al., "Online Stereo Camera Calibration From Scratch", June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI: 10.1109 / IVS.2017.7995952 [Non-Patent Document 2] T. Dang, "Continuous Stereo Self-Calibration by Camera Parameter Tracking", August 2009 IEEE Transactions on Image Processing8(7):1536-50, DOI:10.1109 / TIP.2009.2017824 Summary of the Invention [Problem to be solved by the invention]

[0008] It is therefore an object of the present invention to provide a device and a method that faces the above mentioned technical problems of known devices and methods, in particular to provide a device and a method that allows an automatic recalibration algorithm for a three-dimensional camera, preferably allowing to correct erroneous measurements on the fly, without starting a new static calibration process. [Means for solving the problem]

[0009] This problem is solved by the invention with the features of the independent claims. Advantageous developments of the invention, which can be realized individually or in combination, are set out in the dependent claims and / or in the following description and detailed embodiments.

[0010] In a first aspect of the present invention, a detector for determining a position of at least one object is disclosed.

[0011] As used herein, the term "object" refers to any object, particularly a surface or area configured to at least partially reflect at least one light beam impinging on the object. The light beam may originate from a projector of a detector that illuminates the object, where the light beam is reflected or scattered by the object.

[0012] As used herein, the term "position" refers to at least one item of information regarding the location and / or orientation of an object and / or at least a part of an object in space. Thus, the at least one item of information may imply at least one distance between at least one point of the object and at least one detector. The distance may be an ordinate or may contribute to determining an ordinate of a point of the object. Additionally or alternatively, one or more other items of information regarding the location and / or orientation of the object and / or at least a part of the object may be determined. As an example, additionally, at least one abscissa of the object and / or at least a part of the object may be determined. Thus, the position of an object may imply at least one ordinate of the object and / or at least a part of the object. Additionally or alternatively, the position of an object may imply at least one abscissa of the object and / or at least a part of the object. Additionally or alternatively, the position of an object may imply at least one orientation information of the object, indicating the orientation of the object in space.

[0013] The detector is at least one projector for illuminating an object with at least one illumination pattern, the illumination pattern including a plurality of illumination features; and - at least one sensor element having a matrix of optical sensors, each optical sensor having a photosensitive area, each optical sensor designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by a reflected light beam propagating from the object to a detector, the sensor element being configured to determine at least one reflection image comprising a plurality of reflection features, each of the reflection features comprising a beam profile; at least one evaluation device configured to determine initial distance information of the reflection features by analysis of a beam profile of each of the reflection features, the analysis of the beam profile comprising evaluating a combined signal Q from each of the sensor signals, the evaluation device comprising: a) matching reflectance features to reference features of a reference image taking into account initial distance information, thereby determining matched reflectance and reference feature pairs; b) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; c) determining the epipolar line distance d of the matched reflection feature to said epipolar line; d) estimating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; e) determining at least one correction to the rotation and / or translation of the reflected image as a function of the geometric pattern; an evaluation device configured to perform a calibration method comprising: Equipped with.

[0014] As used herein, the term "projector", also referred to as light projector, refers to an optical device configured to project at least one illumination pattern onto an object, in particular onto a surface of the object. The projector may include at least one light source, also referred to as illumination device or illumination source, configured to generate at least one light beam. The projector may be configured to generate at least one pattern and project the pattern towards at least one surface or scene including the object. The projector may be configured such that the illumination pattern propagates from the projector, in particular from at least one opening in the projector housing towards the object. The projector may be configured to generate and / or project a point cloud, for example, the projector may include at least one digital light processing (DLP) projector, at least one LCoS projector, at least one laser source, at least one array of laser sources; at least one light emitting diode; at least one array of light emitting diodes. In addition, additional illumination patterns may be generated by at least one ambient light source.

[0015] As used herein, the term "pattern" refers to any known or predefined arrangement that includes at least one arbitrary shaped feature. The pattern may include at least one feature, such as a point or a symbol. The pattern may include multiple features. The pattern may include an arrangement of periodic or non-periodic features. As used herein, the term "illumination pattern" refers to a pattern generated and projected by a projector that is used specifically to illuminate an object. As used herein, the term "illumination feature" refers to at least one arbitrary shaped feature of the illumination pattern. The illumination pattern may include at least one periodic regular pattern selected from the group consisting of: at least one periodic regular point pattern; at least one hexagonal pattern; at least one rectangular pattern. For example, the projector may be configured to generate and / or project a point cloud. For example, the projector may be configured to generate a point cloud such that the illumination pattern may include multiple point features.

[0016] The projector may be configured to generate multiple illumination patterns, each including multiple illumination features. The projector may be configured to project two, three, four, five, or more illumination patterns, each including multiple illumination features. The illumination patterns may differ in one or more of the number of illumination features, the arrangement of the illumination features, the shape of the illumination features, the wavelength of the illumination features, the intensity of the illumination features, and the opening angle, among others.

[0017] The projector may include at least one transfer device, in particular at least one diffractive optical element, configured to generate an illumination pattern from at least one light beam generated by a laser source. The term "transfer device", also referred to as "transfer system", may generally refer to one or more optical elements configured to modify the light beam, such as by modifying one or more of the beam parameters of the light beam, the width of the light beam, or the direction of the light beam. The transfer device may in particular include one or more of at least one lens selected from the group consisting of at least one lens, for example at least one adjustable focus lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system.

[0018] The transfer device may have an optical axis. In particular, the detector and the transfer device have a common optical axis. As used herein, the term "optical axis of the transfer device" generally refers to an axis of mirror symmetry or rotational symmetry of a lens or lens system. The optical axis of the detector may be a line of symmetry of the optical setup of the detector. The detector includes at least one transfer device, preferably at least one transfer system having at least one lens. The transfer system may include, by way of example, at least one beam path, and the elements of the transfer system in the beam path are located in a rotationally symmetrical manner with respect to the optical axis. Still, as will be further outlined below, one or more optical elements located in the beam path may also be off-centered or tilted with respect to the optical axis. However, in this case, the optical axis may be defined in series by interconnecting the centers of the optical elements in the beam path, such as by interconnecting the centers of the lenses, and in this context the optical sensor is not counted as an optical element. The optical axis may generally represent the beam path. There, the detector may have a single beam path along which the light beam can travel from the object to the optical sensor, or may have multiple beam paths. As an example, a single beam path may be provided, or the beam path may be divided into two or more partial beam paths. In the latter case, each partial beam path may have its own optical axis. The optical sensors may be located in one and the same beam path or partial beam paths. However, alternatively, the optical sensors may also be located in different partial beam paths.

[0019] The transfer device may form a coordinate system, where ordinate l is the coordinate along the optical axis and d is the spatial offset from the optical axis. The coordinate system may be a polar coordinate system, where the optical axis of the transfer device forms the z-axis, and the distance and polar angle from the z-axis may be used as additional coordinates. Directions parallel or anti-parallel to the z-axis may be considered as longitudinal directions, and the coordinate along the z-axis may be considered as ordinate z. Any direction perpendicular to the z-axis may be considered as transverse directions, and the polar coordinate and / or polar angle may be considered as transverse coordinates.

[0020] As used herein, the term "beam" generally refers to a collection of light rays. In the following, the terms "light rays" and "beam" are used synonymously. As further used herein, the term "light beam" generally refers to a quantity of light, particularly a quantity of light traveling in essentially the same direction, including the possibility of light beams having wide or divergent angles.

[0021] The light beam may include at least one beam profile. The light beam may have a spatial extension. In particular, the light beam may have a non-Gaussian beam profile. The beam profile may be selected from the group consisting of a trapezoidal beam profile; a triangular beam profile; a conical beam profile. The trapezoidal beam profile may have a plateau region and at least one edge region. The light beam may be a Gaussian light beam or a linear combination of Gaussian light beams. As used herein, the term "beam profile" refers to the spatial distribution of the intensity of the light beam, in particular in at least one plane perpendicular to the propagation of the light beam. The beam profile may be a transverse intensity profile of the light beam. The beam profile may be a cross-section of the light beam. The beam profile may be selected from the group consisting of a trapezoidal beam profile; a triangular beam profile; a conical beam profile, and a linear combination of Gaussian beam profiles. However, other embodiments are possible. The projector may include at least one transmission device, which may be configured for one or more of adjusting, defining, and determining the beam profile, in particular the shape of the beam profile.

[0022] For example, the projector may include at least one illumination source, such as a single light source, in particular a single laser source, configured to generate at least one light beam, also referred to as a laser beam. The projector may include at least one transfer device for diffracting and replicating the laser beam generated by the single laser source to generate an illumination pattern including patterned illumination features. In particular, the projector includes at least one diffractive optical element for diffracting and replicating the light beam. The diffractive optical element may be configured for beam forming and / or beam splitting. As used herein, the term "replicate" may refer to generating multiple light beams from one light beam, in particular multiplying the light beam.

[0023] Additionally or alternatively, for example, the projector may include at least one array of densely packed light sources, in particular laser sources, following a pattern configured to generate a cluster of light beams. As used herein, the term "densely packed" light sources may refer to a plurality of light sources arranged in a cluster. The density of the light sources may depend on the elongation of the housing of the individual light sources and the distinguishability of the light beams. The projector may include at least one transfer device for diffracting and replicating the cluster of light beams to generate an illumination pattern including a patterned illumination feature.

[0024] As used herein, the term "sensor element" generally refers to a device or a combination of devices configured to sense at least one parameter. In this case, the parameter may in particular be an optical parameter and the sensor element may in particular be an optical sensor element. The sensor element may be formed unitary, as a single device, or as a combination of several devices. As further used herein, the term "matrix" generally refers to an arrangement of a plurality of elements in a predefined geometric order. The matrix may in particular be or include a rectangular matrix having one or more rows and one or more columns, as will be outlined in more detail below. The rows and columns may in particular be arranged in a rectangular format. However, it should be outlined that other arrangements are feasible, such as non-rectangular arrangements. By way of example, a circular arrangement is also feasible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, the matrix may be a single row of pixels. Other arrangements are feasible.

[0025] The optical sensors of the matrix may in particular be equal in size, sensitivity and one or more of other optical, electrical and mechanical properties. The light-sensitive areas of all the optical sensors of the matrix may in particular be located in a common plane, which preferably faces the object, so that a light beam propagating from the object to the detector may generate a light spot on the common plane.

[0026] As used herein, "optical sensor" generally refers to a photosensitive device for detecting a light beam, such as for detecting illumination and / or light spots generated by at least one light beam. As further used herein, "photosensitive area" generally refers to an area of ​​an optical sensor that can be externally illuminated by at least one light beam in response to the illumination, in which at least one sensor signal is generated. The photosensitive area may in particular be located on the surface of the respective optical sensor. However, other embodiments are feasible. As used herein, the term "optical sensor having at least one photosensitive area" refers to an arrangement having multiple single optical sensors, each having one photosensitive area, and an arrangement having one combined optical sensor having multiple photosensitive areas. Thus, the term "optical sensor" further refers to a photosensitive device configured to generate one output signal, whereas in the present specification, a photosensitive device configured to generate two or more output signals, e.g., at least one CCD and / or CMOS device, is referred to as two or more optical sensors. As outlined in more detail below, each optical sensor may be embodied such that there is exactly one photosensitive area in each optical sensor, such as by providing exactly one photosensitive area that can be illuminated in response to illumination whereby exactly one uniform sensor signal is created for the entire optical sensor. Thus, each optical sensor may be a single area optical sensor. However, the use of a single area optical sensor makes the setup of the detector particularly simple and efficient. Thus, by way of example, commercially available photo sensors, such as commercially available silicon photodiodes, each having exactly one photosensitive area, may be used in the setup. However, other embodiments are feasible. Thus, by way of example, an optical device including two, three, four or more than four photosensitive areas, which are considered as two, three, four or more than four optical sensors, may be used in the context of the present invention. As outlined above, a sensor element includes a matrix of optical sensors. Thus, by way of example, an optical sensor may be part of or constitute a pixelated optical device.By way of example, the optical sensor may be part of or may constitute at least one CCD and / or CMOS device having a matrix of pixels, each pixel forming a light-sensitive area.

[0027] As outlined above, the optical sensors may in particular be or include photodetectors, preferably inorganic photodetectors, more preferably inorganic semiconductor photodetectors, most preferably silicon photodetectors. In particular, the optical sensors may be sensitive in the infrared spectral range. All of the optical sensors of the matrix or at least a group of the optical sensors of the matrix may in particular be identical. A group of identical optical sensors of the matrix may in particular be provided for different spectral ranges or all the optical sensors may be identical with respect to their spectral sensitivity. Furthermore, the optical sensors may be identical in size and / or with respect to their electronic or optoelectronic properties.

[0028] In particular, the optical sensor may be or may include an inorganic photodiode that is sensitive in the infrared spectral range, preferably in the range of 780 nanometers to 3.0 micrometers. In particular, the optical sensor may be sensitive in a part of the near infrared range, where silicon photodiodes are particularly applicable in the range of 700 nanometers to 1000 nanometers. Infrared optical sensors that may be used for the optical sensor may be commercially available infrared optical sensors, such as the infrared optical sensors commercially available under the brand name Hertzstueck™ from trinamiX GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, by way of example, the optical sensor may include at least one optical sensor of essentially photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, HgCdTe photodiodes. Additionally or alternatively, the optical sensor may comprise at least one optical sensor of the extrinsic photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge:Au photodiode, Ge:Hg photodiode, Ge:Cu photodiode, Ge:Zn photodiode, Si:Ga photodiode, Si:As photodiode, Additionally or alternatively, the optical sensor may comprise at least one bolometer, preferably selected from the group consisting of VO bolometers and amorphous Si bolometers.

[0029] The matrix may consist of independent optical sensors, thus consisting of inorganic photodiodes, however, alternatively, commercially available matrices may be used, such as one or more of CCD detectors, such as CCD detector chips, and / or CMOS detectors, such as CMOS detector chips.

[0030] Thus, generally, the optical sensors of the detector may form or be part of a sensor array, such as the matrix mentioned above. Thus, by way of example, the detector may include an array of optical sensors, such as a rectangular array with m rows and n columns, m, n being independently positive integers. Preferably, more than one column and more than one row are provided, i.e. n>1, m>1. Thus, by way of example, n may be 2 to 16 or more, or m may be 2 to 16 or more. Preferably, the ratio of the number of rows and the number of columns is close to 1. By way of example, n and m may be selected such that 0.3≦m / n≦3, such as by choosing m / n=1:1, 4:3, 16:9, or similar. By way of example, the array may be a square array with an equal number of rows and columns, such as by choosing m=2, n=2, or m=3, n=3, or similar.

[0031] The matrix may in particular be a rectangular matrix having at least one row, preferably several rows, and several columns. By way of example, the rows and columns may be essentially vertically oriented, reference being made to the definition given above for the term "essentially vertically". Thus, by way of example, a tolerance of less than 20 degrees, in particular a tolerance of less than 10 degrees or even less than 5 degrees, may be acceptable. In order to provide a wide range of views, the matrix may in particular have at least 10 rows, preferably at least 50 rows, more preferably at least 100 rows. Likewise, the matrix may have at least 10 columns, preferably at least 50 columns, more preferably at least 100 columns. The matrix may include at least 50 optical sensors, preferably at least 100 optical sensors, more preferably at least 500 optical sensors. The matrix may include some pixels in the multi-mega pixel range. However, other embodiments are feasible. Thus, in settings where axial rotational symmetry is to be expected, a circular or concentric arrangement of the optical sensors of the matrix, which may also be referred to as pixels, may be preferred.

[0032] Preferably, the sensor element may be oriented essentially perpendicular to the optical axis of the detector. Again, with regard to the term "essentially perpendicular", reference may be made to the definitions and tolerances given above. The optical axis may be a straight optical axis and may be curved or further split, such as by using one or more deflection elements and / or by using one or more beam splitters, and the essentially perpendicular orientation may in the latter case refer to the local optical axis within each branch or beam path of the optical setup.

[0033] The reflected light beam may propagate from the object towards the detector. The projector may illuminate the object with an illumination pattern, and the light is reflected or scattered by the object and thereby directed at least in part as a reflected light beam towards the detector.

[0034] Each optical sensor is designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by a reflected light beam propagating from the object to the detector. The reflected light beam may in particular completely illuminate the sensor element, so that the sensor element is completely located within the light beam, which has a width of the light beam larger than the matrix. Conversely, preferably, the reflected light beam may in particular create a light spot on the entire matrix, which is smaller than the matrix, so that the light spot is completely located within the matrix. This situation can be easily adjusted by a person skilled in the art of optical systems by choosing one or more suitable lenses or elements that have a focusing or defocusing effect on the light beam, such as by using a suitable transfer device.

[0035] As further used herein, a "sensor signal" generally refers to a signal generated by an optical sensor in response to illumination by a light beam. In particular, the sensor signal may be or include at least one electrical signal, such as at least one analog electrical signal and / or at least one digital electrical signal. More specifically, the sensor signal may be or include at least one voltage signal and / or at least one current signal. More specifically, the sensor signal may include at least one photocurrent. Furthermore, either the raw sensor signal may be used, or the detector, the optical sensor, or any other element may be configured to process or pre-process the sensor signal, thereby generating a secondary sensor signal that may also be used as a sensor signal, such as pre-processing by filtering or the like.

[0036] The raw sensor signals of the optical sensors or secondary sensor signals derived therefrom may be used for the evaluation. As used herein, the term "secondary sensor signal" generally refers to a signal, such as an electronic signal, more preferably an analog and / or digital signal, obtained by processing one or more raw signals, such as by filtering, averaging or demodulation. Thus, image processing algorithms may be used to generate secondary sensor signals from the totality of the sensor signals of the matrix or from an area of ​​interest within the matrix. In particular, a detector, such as an evaluation device, may be configured to convert the sensor signals of the optical sensors and thereby generate secondary optical sensor signals, the evaluation device being configured to perform the determination of the initial distance information by using the secondary optical sensor signals. The transformation of the sensor signal may in particular include at least one transformation selected from the group consisting of: filtering; selecting at least one region of interest; forming a difference image between an image produced by the sensor signal and at least one offset; inverting the sensor signal by inverting the image produced by the sensor signal; forming a difference image between images produced by the sensor signal at different times; background correction; decomposition into color channels; decomposition into tone; saturation; and intensity channels; frequency decomposition; singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high pass filter; applying a low pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; creating a binary image. The region of interest may be determined manually by a user or automatically, such as by recognizing an object in the image produced by the optical sensor.By way of example, a vehicle, a human or another type of predefined object may be determined by automatic image recognition in the image, i.e. in the totality of the sensor signals generated by the optical sensor, and a region of interest may be chosen such that the object is located within the region of interest. In this case, an evaluation such as a determination of the ordinate may be performed only for the region of interest. However, other implementations are feasible.

[0037] The photosensitive areas may in particular be oriented towards the object. As used herein, the term "oriented towards the object" generally refers to a situation in which the respective surfaces of the photosensitive areas are fully or partially visible from the object. In particular, at least one intersecting line between at least one point of the object and at least one point of the respective photosensitive area may form an angle with the surface elements of the photosensitive area different from 0 degrees, such as an angle within the range of 20 degrees to 90 degrees, preferably 80 degrees to 90 degrees, such as 90 degrees. Thus, when the object is located on or near the optical axis, the light beam propagating from the object towards the detector may be essentially parallel to the optical axis. As used herein, the term "essentially vertical" refers to a state of vertical orientation, for example with a tolerance of ±20 degrees or less, preferably with a tolerance of ±10 degrees or less, more preferably with a tolerance of ±5 degrees or less. Similarly, the term "essentially parallel" refers to a state of parallel orientation, for example, with a tolerance of ±20 degrees or less, preferably, a tolerance of ±10 degrees or less, and more preferably, a tolerance of ±5 degrees or less.

[0038] The optical sensors may be sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. In particular, the optical sensors may be sensitive in the visible spectral range of 500 nanometers to 780 nanometers, most preferably in the visible spectral range of 650 nanometers to 750 nanometers or 690 nanometers to 700 nanometers. In particular, the optical sensors may be sensitive in the near infrared range. In particular, the optical sensors may be sensitive in a portion of the near infrared range where silicon photodiodes are particularly applicable in the range of 700 nanometers to 1000 nanometers. The optical sensors may be particularly sensitive in the infrared spectral range, particularly in the range of 780 nanometers to 3.0 micrometers. For example, each optical sensor may independently be or include at least one element selected from the group consisting of a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. For example, the optical sensor may be or may include at least one element selected from the group consisting of a CCD sensor element, a CMOS sensor element, a photodiode, a photocell, a photoconductor, a phototransistor, or any combination thereof. Any other type of photosensitive element may be used. As outlined in more detail below, the photosensitive element may generally be made entirely or partially from inorganic materials and / or entirely or partially from organic materials. Most commonly, one or more photodiodes may be used, such as commercially available photodiodes, e.g., inorganic semiconductor photodiodes, as outlined in more detail below.

[0039] As used herein, the term "reflected image" refers to an image determined by an optical sensor that includes at least one reflective feature, and / or evaluation of the image of the optical sensor with respect to at least one feature, and / or transformation of external parameters such as rotation and translation. Each of the reflective features includes a beam profile. As used herein, the term "reflective feature" refers to a feature in an image plane that is generated by an object in response to illumination by at least one illumination feature, for example. The reflected image may include at least one reflection pattern that includes at least one reflective feature. As used herein, the term "determining at least one reflected image" refers to one or more of imaging, recording, and generating a reflected image.

[0040] The sensor element may be configured to determine at least one reflection pattern. As used herein, the term "reflection pattern" refers to a response pattern generated by reflection or scattering of light at the surface of an object, in particular a response pattern generated by an object in response to illumination by an illumination pattern. The reflection pattern may include at least one feature corresponding to at least one feature of the illumination pattern. The reflection pattern may include at least one distortion pattern compared to the illumination pattern, the distortion depending on the distance of the object, such as the surface characteristics of the object. The evaluation device may be configured to select at least one feature of the reflection pattern and determine the longitudinal area of ​​the selected feature of the reflection pattern by evaluating the combined signal Q from the sensor signals as described above and in more detail below.

[0041] As further used herein, the term "evaluation device" generally refers to any device configured to perform the so-called operations, preferably by using at least one data processing device, more preferably by using at least one processor and / or at least one application specific integrated circuit. Thus, by way of example, the at least one evaluation device may comprise at least one data processing device having a software code stored therein, comprising several computer commands. The evaluation device may provide one or more hardware elements for performing one or more of the so-called operations and / or provide one or more processors with software running thereon for performing one or more of the so-called operations. The above-mentioned operations, including determining at least one ordinate of the object, are performed by the at least one evaluation device. Thus, by way of example, one or more of the relations may be implemented in software and / or hardware, such as by implementing one or more look-up tables, as outlined below. Thus, by way of example, the evaluation device may include one or more programmable devices, such as one or more computers, application specific integrated circuits (ASICs), digital signal processors (DSPs) or field programmable gate arrays (FPGAs), configured to perform the above-mentioned evaluations to determine at least one ordinate coordinate of the object. However, in addition or instead, the evaluation device may also be embodied fully or partly in hardware.

[0042] The evaluation device may be configured to select at least one reflection feature of the reflection image. The evaluation device may be configured to sequentially select reflection features of the reflection image. The evaluation device may be configured to perform image analysis of the reflection image, thereby identifying reflection features of the reflection image. As used herein, the term "selecting at least one reflection feature" refers to one or more of identifying, determining, and choosing at least one reflection feature of the reflection image. The evaluation device may be configured to perform at least one image analysis and / or image processing to identify the reflection feature. The image analysis and / or image processing may use at least one feature detection algorithm. The image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image produced by the sensor signal and at least one offset; inverting the sensor signal by inverting the image produced by the sensor signal; forming a difference image between images produced by the sensor signal at different times; background correction; decomposition into color channels; decomposition into tone; saturation; and intensity channels; frequency decomposition; singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high pass filter; applying a low pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; creating a binary image. The region of interest may be determined manually by a user or automatically, such as by recognizing an object in an image produced by an optical sensor.

[0043] As used herein, the term "initial distance information" may refer to an ordinate determined by using the combined signal Q. The detector may be configured to determine an ordinate of an object point for at least one reflection feature of the reflection image from the combined signal Q. Thus, the detector may be configured to pre-classify at least one reflection feature of the reflection image and / or provide a distance estimate for the reflection feature. In particular, the detector may be configured to determine at least one more accurate distance information of the object by using triangulation taking into account the initial distance information, in particular the pre-classification and / or the distance estimate.

[0044] The evaluation device may be configured to determine at least one initial distance information, i.e. the ordinate z of a selected reflection feature of the reflection image, by analysis of a respective beam profile of the reflection feature. This technique is called beam profile analysis or depth ratio from photons technique and involves determining the ordinate by evaluating a combined signal Q from the sensor signals. Beam profile analysis, in particular the determination of the ordinate using the combined signal Q, is generally known to the skilled person, e.g. from WO018 / 091649A1, WO2018 / 091638A1 and WO2018 / 091640A1, the contents of which are incorporated by reference.

[0045] The analysis of the beam profile includes evaluating a combined signal Q from the respective sensor signals, in particular the combined signal Q of the sensor signals generated by the optical sensors detecting the reflected light beam on their photosensitive areas. Each optical sensor is designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by the reflected light beam propagating from the object to the detector.

[0046] As used herein, the term "combined signal Q" refers to a signal generated by combining the sensor signals, in particular by one or more of dividing the sensor signals, dividing a multiple of the sensor signals, or dividing a linear combination of the sensor signals. The evaluation device may be configured to derive the combined signal Q by one or more of dividing the sensor signals, dividing a multiple of the sensor signals, or dividing a linear combination of the sensor signals. The evaluation device may be configured to use at least one predetermined relationship between the combined signal Q and the longitudinal area for determining the longitudinal area.

[0047] For example, the evaluation device

[0048]

number

[0049] In general, the beam profile is o ) and beam shape S(x,y;z o ), E(x,y;z o )=L·S. Thus, deriving the combined signal may allow one to determine the ordinate independent of the luminance. In addition, using the combined signal allows one to determine the distance z o Thus, the combined signal allows the determination of the distance z that is independent of the material and / or reflective and / or scattering properties of the object and independent of modifications of the light source, such as due to manufacturing inaccuracies, heat, water, dust, or damage to the lens. oThis enables the determination of

[0050] Each of the sensor signals may include at least one information of at least one area of ​​the beam profile of the light beam. As used herein, the term "area of ​​the beam profile" generally refers to any region of the beam profile at the sensor location that is used to determine the combined signal Q.

[0051] The photosensitive areas may be arranged such that the first sensor signal includes information of the first area of ​​the beam profile and the second sensor signal includes information of the second area of ​​the beam profile. The first area of ​​the beam profile and the second area of ​​the beam profile may be one or both of adjacent areas or overlapping areas. The first area of ​​the beam profile and the second area of ​​the beam profile may not coincide in area.

[0052] The evaluation device may be configured to determine and / or select a first area of ​​the beam profile and a second area of ​​the beam profile. The first area of ​​the beam profile may essentially include edge information of the beam profile and the second area of ​​the beam profile may essentially include center information of the beam profile. The beam profile may have a center, i.e. a maximum of the beam profile and / or a center point of the plateau of the beam profile and / or a geometric center of the light spot, and a falling edge extending from the center. The second area may include an inner area of ​​the cross section and the first area may include an outer area of ​​the cross section. As used herein, the term "essentially center information" generally refers to a low ratio of edge information, i.e. a ratio of the intensity distribution corresponding to the edge, i.e. a ratio of the intensity distribution corresponding to the center, compared to a ratio of the center information. Preferably, the center information has a ratio of edge information less than 10%, more preferably less than 5%, and most preferably, the center information does not include edge content. As used herein, the term "essentially edge information" generally refers to a low ratio of center information compared to a ratio of edge information. The edge information may include information of the entire beam profile, especially from the center and edge regions. The edge information has a ratio of center information less than 10%, preferably less than 5%, and more preferably the edge information does not include center content. At least one area of ​​the beam profile may be determined and / or selected as the second area of ​​the beam profile if it is close to or around the center and essentially includes center information. At least one area of ​​the beam profile may be determined and / or selected as the first area of ​​the beam profile if it includes at least a part of the falling edge of the cross section. For example, the entire area of ​​the cross section may be determined as the first region. The first area of ​​the beam profile may be area A2 and the second area of ​​the beam profile may be area A1.

[0053] Other choices of the first area A1 and the second area A2 may be feasible. For example, the first area may essentially include the outer region of the beam profile, and the second area may essentially include the inner region of the beam profile. For example, in the case of a two-dimensional beam profile, the beam profile may be divided into a left part and a right part, and the first area may essentially include the area of ​​the left part of the beam profile, and the second area may essentially include the area of ​​the right part of the beam profile.

[0054] The edge information may include information related to a number of photons in a first area of ​​the beam profile, and the center information may include information related to a number of photons in a second area of ​​the beam profile. The evaluation device may be configured to determine an area integral of the beam profile. The evaluation device may be configured to determine the edge information by integrating and / or summing the first area. The evaluation device may be configured to determine the center information by integrating and / or summing the second area. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the integral of the trapezoid. Furthermore, when a trapezoidal beam profile may be assumed, the determination of the edge and center signals may be replaced with an equivalent evaluation that utilizes the properties of the trapezoidal beam profile, such as the determination of the slope and position of the edges and the height of the central plateau, and derives the edge and center signals according to geometrical considerations.

[0055] Additionally or alternatively, the evaluation device may be configured to determine one or both of center and edge information from at least one slice or cut of the light spot. This may be achieved, for example, by replacing the area integral in the combined signal Q with a line integral along the slice or cut. For improved accuracy, several slices or cuts through the light spot may be used and averaged. In case of an elliptical spot profile, averaging over several slices or cuts may result in improved distance information.

[0056] In one embodiment, the light beam propagating from the object to the detector may illuminate the sensor element with at least one reflective pattern comprising a plurality of feature points. As used herein, the term "feature point" refers to at least one at least partially elongated feature of the pattern. The feature point may be selected from the group consisting of at least one point, at least one line, at least one edge. The reflective pattern may be generated by the object in response to illumination by at least one light source with an illumination pattern comprising at least one pattern, for example. A1 may correspond to the total area or complete area of ​​the feature point on the optical sensor. A2 may be the central area of ​​the feature point on the optical sensor. The central area may be a constant value. The central area may be small compared to the total area of ​​the feature point. For example, in the case of a circular feature point, the central area may have a radius of 0.1 to 0.9 of the total radius of the feature point, preferably 0.4 to 0.6 of the total radius.

[0057] The evaluation device may be configured to derive the combined signal Q by one or more of dividing the edge and center information, dividing a multiple of the edge and center information, dividing a linear combination of the edge and center information, etc. Thus, essentially, photon ratios may be used as the physical basis of the technique.

[0058] For example, the evaluation device - determining at least one optical sensor having a highest sensor signal and forming at least one center signal; - evaluating the sensor signals of the matrix of optical sensors and forming at least one sum signal; - determining at least one combined signal by combining the center signal and the sum signal; - determining an ordinate z of at least one of the selected features by evaluating the combined signal; The sensor signal may be evaluated by

[0059] As a result, according to the present invention, the term "central signal" generally refers to at least one sensor signal that essentially contains central information of the beam profile. For example, the central signal may be the signal of at least one optical sensor having the highest sensor signal from a plurality of sensor signals generated by the optical sensors of the entire matrix or of a region of interest within the matrix, the region of interest may be predefined or discernible within the image generated by the optical sensors of the matrix. As used herein, the term "highest sensor signal" refers to one or both of a local maximum or a maximum within the region of interest. The central signal may originate from a single optical sensor or from a group of optical sensors, as outlined in more detail below, in the latter case, by way of example, the sensor signals of the group of optical sensors may be added, integrated or averaged to determine the central signal. The group of optical sensors from which the central signal originates may be a group of adjacent optical sensors, such as optical sensors having less than a predefined distance from the actual optical sensor having the highest sensor signal, or may be a group of optical sensors that generate sensor signals that are within a predefined range from the highest sensor signal. The group of optical sensors from which the central signal originates may be chosen as large as possible to allow a maximum dynamic range. The evaluation device may be configured to determine the central signal by integration of a plurality of sensor signals, for example a plurality of optical sensors around an optical sensor having a highest sensor signal. For example, the beam profile may be a trapezoidal beam profile and the evaluation device may be configured to determine the integral of the trapezoid, in particular the integral of the plateau of the trapezoid.

[0060] As outlined above, the center signal may generally be a single sensor signal, such as a sensor signal from an optical sensor in the center of the light spot, or may be a combination of multiple sensor signals, such as a combination of sensor signals originating from optical sensors in the center of the light spot, or may be a secondary sensor signal derived by processing sensor signals derived by one or more of the possibilities described above. The determination of the center signal may be performed electronically, as the comparison of sensor signals is fairly straightforward to implement by conventional electronics, or may be performed fully or partially by software. In particular, the center signal may be selected from the group consisting of: the highest sensor signal; the average of the group of sensor signals that are within a predetermined range of tolerance from the highest sensor signal; the average of the sensor signals from the group of optical sensors including the optical sensor with the highest sensor signal and the predetermined group of adjacent optical sensors; the sum of the sensor signals from the group of optical sensors including the optical sensor with the highest sensor signal and the predetermined group of adjacent optical sensors; the sum of the group of sensor signals that are within a predetermined range of tolerance from the highest sensor signal; the average of the group of sensor signals that are above a predetermined threshold; the sum of the group of sensor signals that are above a predetermined threshold; the integral of the sensor signals from the group of optical sensors including the optical sensor with the highest sensor signal and the predetermined group of adjacent optical sensors; the integral of the group of sensor signals that are within a predetermined range of tolerance from the highest sensor signal; the integral of the group of sensor signals that are above a predetermined threshold.

[0061] Similarly, the term "sum signal" generally refers to a signal that essentially includes edge information of the beam profile. For example, the sum signal may be derived by adding, integrating over, or averaging over the sensor signals of the entire matrix or of a region of interest within the matrix, the region of interest being predefined or identifiable within the image generated by the optical sensor of the matrix. When adding, integrating over, or averaging over the sensor signals, the actual optical sensor from which the sensor signal is generated may be excluded from the summing, integrating, or averaging, or may instead be included in the summing, integrating, or averaging. The evaluation device may be configured to determine the sum signal by integrating the signal of the entire matrix or of the region of interest within the matrix. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the integral of the entire trapezoid. Furthermore, when a trapezoidal beam profile may be assumed, the determination of the edge and center signals may be replaced with an equivalent evaluation that utilizes the characteristics of the trapezoidal beam profile, such as the determination of the slope and position of the edges and the height of the central plateau, and derives the edge and center signals according to geometrical considerations.

[0062] Similarly, the center signal and edge signal can be determined by using a section of the beam profile, such as a circular section of the beam profile. For example, the beam profile can be divided into two sections by a secant or chord that does not pass through the center of the beam profile. Thus, one section essentially contains edge information and the other section essentially contains center information. For example, the edge signal can be further subtracted from the center signal to further reduce the amount of edge information in the center signal.

[0063] Additionally or alternatively, the evaluation device may be configured to determine one or both of center information or edge information from at least one slice or cut of the light spot. This may be achieved, for example, by replacing the area integral in the combined signal Q with a line integral along the slice or cut. For improved accuracy, several slices or cuts through the light spot may be used and averaged. In case of an elliptical spot profile, averaging over several slices or cuts may result in improved distance information.

[0064] The combined signal may be a signal generated by combining the center signal and the sum signal. In particular, the combining may include one or more of: forming a quotient or inverse quotient of the center signal and the sum signal; forming a quotient or inverse quotient of a multiple of the center signal and a multiple of the sum signal; forming a quotient or inverse quotient of a linear combination of the center signal and a linear combination of the sum signal. Additionally or alternatively, the combined signal may include any signal or combination of signals that includes at least one item of information regarding the comparison between the center signal and the sum signal.

[0065] The detection of the center of the light spot, i.e. the detection of the center signal and / or the detection of the at least one optical sensor from which the center signal originates, may be fully or partly performed electronically or fully or partly performed by using one or more software algorithms. In particular, the evaluation device may comprise at least one center detector for detecting at least one highest sensor signal and / or forming a center signal. The center detector may in particular be fully or partly embodied in software and / or fully or partly embodied in hardware. The center detector may be fully or partly integrated in at least one sensor element and / or fully or partly embodied independently of the sensor element.

[0066] The sum signal can be derived from all sensor signals of the matrix, from sensor signals within the region of interest, or from one of these possibilities with sensor signals resulting from optical sensors contributing to the excluded central signal. In all cases, a reliable sum signal that can be reliably compared to the central signal can be generated to determine the ordinate. In general, the sum signal can be selected from the group consisting of: average over all sensor signals of the matrix; sum of all sensor signals of the matrix; integral of all sensor signals of the matrix; average over all sensor signals of the matrix except for sensor signals from those optical sensors that contribute to the central signal; sum of all sensor signals of the matrix except for sensor signals from those optical sensors that contribute to the central signal; integral of all sensor signals of the matrix except for sensor signals from those optical sensors that contribute to the central signal; sum of sensor signals of optical sensors within a predetermined range from the optical sensor with the highest sensor signal; integral of sensor signals of optical sensors within a predetermined range from the optical sensor with the highest sensor signal; sum of sensor signals above a certain threshold of optical sensors located within a predetermined range from the optical sensor with the highest sensor signal; integral of sensor signals above a certain threshold of optical sensors located within a predetermined range from the optical sensor with the highest sensor signal. However, other options exist.

[0067] The summing can be fully or partially performed in software and / or fully or partially performed in hardware. The summing is generally possible by purely electronic means, which can typically be easily implemented in the detector. Thus, in the electronic field, summing devices are generally known for summing two or more electrical signals, both analog and digital signals. Thus, the evaluation device may include at least one summing device for forming a sum signal. The summing device may be fully or partially integrated in the sensor element or may be embodied fully or partially independent of the sensor element. The summing device may be embodied fully or partially in hardware or software or both.

[0068] The comparison between the center signal and the sum signal may in particular be performed by forming one or more quotient signals. Thus, in general, the combined signal Q may be a quotient signal derived from one or more of: forming a quotient or inverse quotient of the center signal and the sum signal; forming a quotient or inverse quotient of a multiple of the center signal and a multiple of the sum signal; forming a quotient or inverse quotient of a linear combination of the center signal and a linear combination of the sum signal; forming a quotient or inverse quotient of the center signal and a linear combination of the sum signal and the center signal; forming a quotient or inverse quotient of the sum signal and a linear combination of the sum signal and the center signal; forming a quotient or inverse quotient of a power of the center signal and a power of the sum signal. However, other options exist. The evaluation device may be configured to form one or more quotient signals. The evaluation device may further be configured to determine at least one ordinate by evaluating at least one quotient signal.

[0069] The evaluation device is configured to use at least one predetermined relationship between the combined signal Q and an ordinate for determining initial distance information, in particular by using at least one known, discriminable or predetermined relationship between the sensor signals. In particular, the evaluation device is configured to determine at least one coordinate of the object by using at least one known, discriminable or predetermined relationship between a quotient signal derived from the sensor signals and an ordinate. The predetermined relationship may be one or more of an empirical relationship, a semi-empirical relationship and an analytically derived relationship. The evaluation device may include at least one data storage device for storing the predetermined relationships, such as a look-up list or a look-up table.

[0070] Thus, due to the reasons disclosed above and due to the dependence of the properties of the light spot on the ordinate, the combined signal Q is typically a monotonic function of the ordinate of the object and / or of the size of the light spot, such as the diameter or equal diameter of the light spot. Thus, by way of example, and particularly in the case where a linear optical sensor is used, the sensor signal scenter and the total signal s sum A simple quotient of s center / s sum can be a monotonically decreasing function of distance. Without wishing to be bound by this theory, in the preferred setup described above, the central signal s center and the total signal s sum This is believed to be due to the fact that both s and s decrease as a square function with increasing distance to the light source. However, in the optical setup used in the experiment, the light spot in the image plane grows and thus spreads over a larger area, so that the central signal s center is the total signal s sum , which decreases faster than the center signal and the total signal. Thus, the quotient of the center signal and the total signal decreases continuously with increasing diameter of the light beam or the diameter of the light spot on the photosensitive area of ​​the optical sensor of the matrix. The quotient is also typically independent of the total power of the light beam, since the total power of the light beam forms a factor in both the center signal and the total sensor signal. As a result, the combined signal Q may form a unique and ambiguous relationship between the center signal and the total signal, and a secondary signal providing the size or diameter of the light beam. There may be a unique and ambiguous relationship between the center signal and the total signal, on the one hand, and the ordinate, on the other hand, since the size or diameter of the light beam depends on the distance between the object from which the light beam propagates towards the detector, and the detector itself, i.e., on the ordinate of the object. The predetermined relationship may be determined by analytical considerations, such as by assuming a linear combination of Gaussian light beams, by empirical measurements, such as measurements measuring the combined signal and / or the center signal and the total signal or its derived secondary signal as a function of the ordinate of the object, or both.

[0071] The combined signal Q can be determined by using various means. For example, software means for deriving a quotient signal, hardware means for deriving a quotient signal, or both can be used and implemented in the evaluation device. Thus, the evaluation device can include, for example, at least one divider, the divider being configured to derive the quotient signal. The divider can be fully or partially embodied as one or both of a software divider or a hardware divider. The divider can be fully or partially integrated into the sensor element answer, or can be fully or partially embodied independently of the sensor element.

[0072] Depth measurement using beam profile analysis may enable reliable distance determination even in environments that cause multiple reflections by biasing the light source or in the case of reflection measurement objects by reducing the computational requirements, in particular by reducing the processing power. Beam profile analysis may enable estimating a depth map from the image of the sensor element. In particular, the distances determined by the beam profile analysis may provide distance estimates for each illumination feature, which can be refined by triangulation methods with respect to known, in particular fixed, positions of the sensor element and the projector. In order to calculate the refined ordinates using triangulation, a so-called correspondence problem needs to be solved. In general, three-dimensional reconstruction methods using triangulation require an externally calibrated system. In the case of an externally calibrated system, each reflection feature can be matched with a reference grid point, i.e., a reference feature, together with the estimated initial distance information of the beam profile analysis. Thus, if the epipolar condition is met and the beam profile analysis obtains a reliable depth estimate, each detected reflection feature can be matched with a corresponding reference grid point. However, if the correspondence is determined to be incorrect, the distance measurement based on triangulation becomes very poor. Depending on the hardware, a pre-calibrated detector may degrade due to physical stress or temperature shifts, resulting in a change in relative position and rotation over time. Changes in the relative position of the sensor element and the projector result in changes in the reflected image and in an incorrectly determined correspondence of the reference feature and the reflected feature, and thus in an incorrect distance measurement result. The invention proposes to perform a calibration method, in particular a recalibration, for determining the extrinsic parameters of the detector. The calibration method according to the invention may allow to directly correct an incorrect distance measurement on the fly, without starting a new static calibration process. In particular, the calibration method may be performed automatically, such as without any user intervention. The term "automatically", as used herein, is a broad term and is to be given its original and customary meaning for a person skilled in the art, and is not limited to a special or customized meaning.The term may particularly, but without limitation, refer to a process that is performed entirely by means of at least one computer and / or computer network and / or machine, in particular without manual action and / or intervention by a user.

[0073] As used herein, the term "calibration" is a broad term and is to be given its original and customary meaning to those skilled in the art and is not limited to a special or customized meaning. The term calibration may refer to at least one process for determining at least one extrinsic parameter of the detector and / or determining, in particular, a correction to the detector's measurements of the position of the reflection features in the reflection image. The evaluation device may be configured to determine at least one extrinsic parameter of the detector. The extrinsic parameter may include at least one parameter selected from the group consisting of a rotation angle between the projector and the coordinates of the sensor element, a translation component between the projector and the coordinates of the sensor element, an aperture angle, a center of the sensor element, an aperture, a focal length.

[0074] The calibration method involves matching the reflectance features to reference features of a reference image, taking into account initial distance information, thereby determining a matched reflectance and reference feature pair.

[0075] As used herein, the term "reference image" refers to an image different from the reflected image determined at a different spatial location compared to the reflected image. The reference image is determined by one or more of recording at least one reference feature, imaging at least one reference feature, and calculating the reference image. In particular, the reference image includes at least one reference pattern, also represented as a reference grid, that includes a plurality of reference features. As used herein, the term "reference feature" refers to at least one feature of the reference image. The reference image and the reflected image may be images of objects determined at different spatial locations with a fixed distance. The distance may be a relative distance, also referred to as a baseline.

[0076] For example, the reference image may be a reference grid, such as an image of the illumination pattern in an image plane at the projector's position. The projector and the sensor elements may be separated by a fixed distance.

[0077] For example, the detector may include at least two sensor elements each having a matrix of optical sensors. The at least one first sensor element and the at least one second sensor element may be located at different spatial positions. The relative distance between the first sensor element and the second element may be fixed. The at least one first sensor element may be configured to determine at least one first reflection pattern, in particular at least one first reflection feature, and the at least one second sensor element may be configured to determine at least one second reflection pattern, in particular at least one second reflection feature. The evaluation device may be configured to select at least one image determined by the first or the second sensor element as a reflection image and to select at least one image determined by the other of the first or the second sensor element as a reference image.

[0078] The evaluation device may be configured to match each one of the reflection features with each one of the reference features in the displacement region by using at least one linear scaling algorithm. A beam profile analysis may make it possible to reduce the number of possibilities.

[0079] The evaluation device may be configured to determine at least one reference feature in at least one reference image that corresponds to the at least one reflection feature. The evaluation device may be configured to perform an image analysis and to identify features of the reflection image. The evaluation device may be configured to identify at least one reference feature in the reference image that has essentially the same ordinate as the selected reflection feature. The term "essentially the same" refers to being identical within 10%, preferably within 5%, most preferably within 1%. The reference feature that corresponds to the reflection feature may be determined using epipolar geometry. For the description of epipolar geometry, reference is made, for example, to X. Jiang, H. Bunke, Chapter 2: "Dreidimensionales Computersehen", Springer, Berlin Heidelberg, 1997. The epipolar geometry may assume that the reference image and the reflection image may be images of an object determined at different spatial positions and / or spatial orientations with a fixed distance. The reference image and the reflection image may be images of an object determined at different spatial positions with a fixed distance. The evaluation device may be configured to determine an epipolar line in the reference image. The assumed relative position of the reference image and the reflected image may be known. For example, the assumed relative position of the reference image may be determined in a previous or historical calibration, in which steps a) to e) have been performed. For example, the assumed relative position of the reference image and the reflected image may be a manufacturer's value. For example, the assumed relative position of the reference image and the reflected image may be stored in at least one storage unit of the evaluation device. The evaluation device may be configured to determine a straight line extending from a selected reflection feature of the reflected image. The straight line may include possible reflection features corresponding to the selected reflection feature. The straight line and the baseline span the epipolar plane. In order for the reference image to be determined in a different relative position from the reflected image, the corresponding possible reflection features may be imaged on a straight line in the reference image, a so-called epipolar line. Thus, a reference feature of the reference image corresponding to a selected reflection feature of the reflected image is assumed to be on the epipolar line.However, as outlined above, due to image distortions or changes in external parameters, such as due to aging, temperature changes, mechanical stress, etc., the epipolar lines may cross each other or be very close to each other and / or the correspondence between reference features and reflected features may become unclear.

[0080] The evaluation device may be configured to determine, for each reflection feature, a longitudinal area of ​​the reflection feature. The longitudinal area may be given by the initial distance information of the reflection feature determined from the combined signal Q and the error interval ±ε. The evaluation device may be configured to determine at least one displacement area in the reference image corresponding to the longitudinal area. As used herein, the term "displacement area" refers to an area in the reference image in which a reference feature corresponding to a selected reflection feature may be imaged. In particular, the displacement area may be an area in the reference image in which a reference feature corresponding to a selected reflection feature is expected to be located in the reference image. Depending on the distance to the object, the image position of the reference feature corresponding to the reflection feature may be displaced in the reference image compared to the image position of the reflection feature in the reflection image. The displacement area may include only one reference feature. The displacement area may also include more than one reference feature.

[0081] The displacement area may include the epipolar line or a section of the epipolar line. The displacement area may include more than one epipolar line or more sections of more than one epipolar line. The displacement area may extend along the epipolar line, may be orthogonal to the epipolar line, or both. The evaluation device may be configured to determine a reference feature along the epipolar line corresponding to the initial distance information and to determine a degree of the displacement area along the epipolar line or orthogonal to the epipolar line corresponding to the error interval ±ε. The measurement uncertainty of the distance measurement using the combined signal Q may result in a displacement area that is non-circular, since the measurement uncertainty may be different for different directions. In particular, the measurement uncertainty along the epipolar line or epipolar line(s) may be larger than the measurement uncertainty in the orthogonal direction to the epipolar line or epipolar line(s). The displacement area may include a degree in the orthogonal direction to the epipolar line or epipolar line(s). The evaluation device may be configured to match the selected reflection feature with at least one reference feature in the displacement region. As used herein, the term "matching" refers to determining and / or evaluating corresponding reference and reflection features. The evaluation device may be configured to match the selected feature of the reflection image with the reference feature in the displacement region by using at least one evaluation algorithm taking into account the determined initial distance information. The evaluation algorithm may be a linear scaling algorithm. The evaluation device may be configured to determine an epipolar line closest to and / or within the displacement region. The evaluation device may be configured to determine an epipolar line closest to an image location of the reflection feature. The extent of the displacement region along the epipolar line may be greater than the extent of the displacement region orthogonal to the epipolar line. The evaluation device may be configured to determine the epipolar line prior to determining the corresponding reference feature. The evaluation device may determine a displacement region around an image location of each reflection feature. The evaluation device may be configured to assign an epipolar line to each displacement region for each image position of the reflective feature, such as by assigning an epipolar line closest to the displacement region and / or within the displacement region and / or closest to the displacement region along a direction perpendicular to the epipolar line.The evaluation device may be configured to determine a reference feature corresponding to an image position of the reflection feature by determining a reference feature closest to the assigned displacement region and / or within the assigned displacement region and / or closest to the assigned displacement region along the assigned epipolar line and / or within the assigned displacement region along the assigned epipolar line.

[0082] Additionally or alternatively, the evaluation device may include the following steps: - determining a displacement area for an image location of each reflective feature; - assigning an epipolar line to the displacement area of ​​each reflection feature by assigning the epipolar line closest to the displacement area and / or within the displacement area and / or along a direction orthogonal to the epipolar line; - assigning at least one reference feature to each reflection feature and / or determining at least one reference feature to each reflection feature by assigning a reference feature closest to the assigned displacement area and / or within the assigned displacement area and / or closest to the assigned displacement area and / or within the assigned displacement area along the assigned epipolar line. The method may be configured to perform the following steps:

[0083] Additionally or alternatively, the evaluation device may be configured to discriminate between more than one epipolar line and / or reference feature to be assigned to a reflection feature, such as by comparing distances of epipolar lines in the reflection feature and / or reference image and / or by comparing error-weighted distances, such as ε-weighted distances of epipolar lines in the reflection feature and / or reference image, and assigning the epipolar line and / or reference feature within a shorter distance and / or ε-weighted distance to the reference feature and / or reflection feature.

[0084] The detector, in particular the evaluation device, can be configured to use the combined signal Q to pre-classify selected reflection features, so that an ambiguous assignment to one reference feature is possible. In particular, the illumination features of the illumination pattern can be arranged such that corresponding reference features of the reference image can have a relative distance from each other on the epipolar line as long as possible. The illumination features of the illumination pattern can be arranged such that only a small number of reference features are located on the epipolar line.

[0085] Using the beam profile analysis may allow to estimate initial distance information, such as an ordinate in the error interval. By determining the displacement area corresponding to the initial distance information, the corresponding error interval may allow to reduce the possible number of solutions along the epipolar line for significantly matching the reference feature and the reflected feature. The number of possible solutions may even be reduced to one. The determination of the initial distance information may be performed during a pre-evaluation before matching the reflected feature and the reference feature. This may allow to reduce the computational requirements, which may result in significantly reduced costs and allow for use in mobile or outdoor devices.

[0086] The calibration method includes determining, for each matched reflection and reference feature pair, an epipolar line of the matched reference feature in the reference image, in particular, the epipolar line used to match the respective matched reference and reflection feature may be used as the epipolar line of said pair of matched reflection and reference features.

[0087] The calibration method further includes determining the epipolar line distance d of the matched reflection feature to the aforementioned epipolar line. As used herein, the term "epipolar line distance" may refer to the distance of the reflection feature and the distance of the epipolar line used to match the matched reference feature, represented as the corresponding epipolar line. The distance may be determined by determining the image coordinates of the reflection image and the image coordinates of the corresponding epipolar line and comparing the image coordinates. The minimum distance to the corresponding epipolar line may be used as the epipolar line distance. In the case of a good external calibration, the epipolar line distance is close to zero. This is exactly the epipolar condition. In the case of a decalibrated detector, the reflection feature may have a good chance of matching with the reference feature. However, it may always be possible to determine the epipolar line distance to the incorrect corresponding reference feature or the true corresponding reference feature. In the matching step, the reconstruction algorithm used may match the reflection feature to the reference feature if the epipolar line distance is within a tolerance range. It may therefore be possible that a reflection feature is matched with an incorrect reference feature, resulting in a non-zero epipolar line distance. If an incorrect pair of matched reflection and reference features is determined, the evaluation device may be configured to perform steps b) to e). A calibration method may be based on evaluating the resulting epipolar line distance. The calibration method may take into account the epipolar distance independently of the fact if the correspondence between the reflection feature and the reference feature is correct or incorrect. The epipolar distance may be appropriate for the proposed calibration strategy even if the reflection feature is matched with an incorrect reference feature.

[0088] Step d) includes evaluating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern. As used herein, the term "geometric pattern" may refer to a distribution of epipolar line distances. The epipolar line distance may be defined as a function d(x,y) on the location (x,y) of the reference image. The epipolar line distance function d can be analyzed to calculate a correction for rotation and / or translation. In the case of a decalibrated system, the function d may generate a geometric pattern. The shape of this geometric pattern of the epipolar line distance function may uniquely indicate the degree of decalibration. The geometric pattern, such as repetition, steepness, discontinuity, and curvature in the function d(x,y) can be used to restore the calibration. When the rotation and / or translation of the projector and / or sensor elements is changed, the result can be observed in the function d as a geometric pattern. The evaluation device may be configured to execute an algorithm designed to analyze d(x,y) and calculate a correction for rotation and / or translation. The evaluation device may be configured to determine a correction for the reflected image by evaluating one or more of the shape, repetition, steepness, discontinuities, and curvature of the geometric pattern.

[0089] The evaluation device may be configured to correct the reflected image based on the determined correction. As outlined above, the reflected image refers to the evaluation of the image of the optical sensor with respect to at least one feature and / or the transformation of external parameters such as rotation and translation. The correction for the rotation and / or translation of the reflected image may be at least one correction factor applied to the image position of the reflected image. The evaluation device may be configured to determine whether the geometric pattern matches within a predefined tolerance to the geometric pattern of the calibrated detector or whether the geometric pattern deviates from the geometric pattern of the calibration by more than a predefined tolerance. The determined correction may give information about the degree of decalibration. If the detector is already optimally calibrated, the correction step may be very small and the correction may also have a very small effect. In case of a match within the tolerance, the evaluation device may keep the initial external parameters and / or the correction of the reflected image may be discarded. Otherwise, the correction may be applied to the reflected image. The determined correction may be used to correct the rotation and / or translation. The rotation and translation describe the spatial context between the sensor elements and the projector. The rotation and translation may include a relationship between a "reflection feature" and a "reference feature". From this relationship, triangulation distance information can be calculated by using triangulation techniques. Thus, in case of a determined change in rotation and / or translation, the resulting triangulation information can be corrected.

[0090] The evaluation device may be configured to determine at least one triangulation distance information of the reflected feature by using triangulation taking into account the determined correction. As used herein, the term "triangulation distance" may refer to an ordinate determined by using triangulation. The evaluation device may be configured to determine a displacement of the matched reference feature and the reflected feature. As used herein, the term "displacement" may refer to a difference between a position in the reference image and a position in the reflected image. The evaluation device may be configured to determine the triangulation distance of the matched reference feature using a predetermined relationship between the ordinate and the displacement. The evaluation device is configured to perform a calibration method on the fly while determining the triangulation distance information.

[0091] As outlined above, the detector may be configured to determine at least one ordinate of the object, including the option of determining the ordinate of the entire object or of one or more parts thereof. However, in addition, other coordinates of the object, including one or more abscissas and / or rotational coordinates, may be determined by the detector, in particular by the evaluation device. Thus, by way of example, one or more lateral sensors for determining at least one lateral coordinate of the object may be used. In general, various lateral sensors are known in the art, such as the lateral sensors disclosed in WO2014 / 097181A1 and / or other position sensitive devices (PSDs), such as four-quadrant diodes, CCD chips, or CMOS chips. Additionally or alternatively, by way of example, the detector according to the invention may include one or more PSDs disclosed in RAStreet (Ed.): Technology and Applications of Amorphous Silicon, Springer-Verlag Heidelberg, 2010, pages 346-349. Other embodiments are possible. In general, these devices may also be implemented in the detector according to the invention. By way of example, a part of the light beam can be separated in the detector by at least one beam splitting element. The separated part can be guided towards a lateral sensor, for example a CCD chip or a CMOS chip or a camera sensor, and the lateral position of the light spot generated by the separated part on the lateral sensor can be determined, thereby determining at least one lateral coordinate of the object. The detector according to the invention can then either be a one-dimensional detector, such as a simple distance measuring device, or can be embodied as a two-dimensional detector or even as a three-dimensional detector. Furthermore, as outlined above or outlined in more detail below, a three-dimensional image can also be created by scanning the scenery or the environment in a one-dimensional format. The detector according to the invention can then in particular be one of a one-dimensional detector, a two-dimensional detector or a three-dimensional detector. The evaluation device can further be configured to determine at least one lateral coordinate x,y of the object. The evaluation device can then be configured to combine the ordinate information and the abscissa information and to determine the position of the object in space.

[0092] In a further aspect, the present invention discloses a method of calibrating at least one detector according to the present invention, such as according to one or more of the embodiments with reference to the detector as disclosed above or as disclosed in more detail below. The method comprises the following method steps, which may be performed in a given order or in a different order. Furthermore, there may be one or more additional method steps not listed. Furthermore, one of the method steps, two or more of the method steps, or even all of the method steps may be performed repeatedly.

[0093] The method comprises the following steps: i) initial distance information, - illuminating the object with at least one illumination pattern generated by at least one projector of the detector, the illumination pattern including a plurality of illumination features; - generating, in response to the illumination, at least one sensor signal for each reflected light beam impinging on a photosensitive area of ​​an optical sensor of a sensor element having a matrix of optical sensors; - determining at least one reflected image by using a sensor element including a plurality of reflected features, each of the reflected features including a beam profile; - evaluating the sensor signals by using at least one evaluation device, thereby determining a combined signal Q and determining initial distance information of the reflective features by analysis of a beam profile of each of the reflective features, the analysis of the beam profile comprising evaluating the combined signal Q from each of the sensor signals. and ii) matching the reflectance features to reference features of a reference image taking into account the initial distance information, thereby determining matched reflectance and reference feature pairs; iii) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; iv) determining an epipolar line distance d of the matched reflection feature to the epipolar line; v) estimating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; vi) determining at least one correction to the rotation and / or translation of the reflected image as a function of the geometric pattern; Includes.

[0094] The method may include correcting the reflected image based on the determined correction. The method may further include determining at least one triangulation distance information of the reflected feature by using triangulation in consideration of the determined correction.

[0095] For details, options and definitions, reference may be made to the detector as discussed above. Thus, in particular, as outlined above, the method may comprise using a detector according to the invention, such as according to one or more of the embodiments given above or given in more detail below.

[0096] In a further aspect of the invention, a method of using the detector according to the invention is proposed for the purpose of use, such as according to one or more of the embodiments given above or given in more detail below, selected from the group consisting of: position measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photography applications; imaging or camera applications; mapping applications for generating a map of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robot applications; quality control applications; manufacturing applications. For further uses of the detector and device of the invention, reference is made to WO2018 / 091649A1, WO2018 / 091638A1 and WO2018 / 091640A1, the contents of which are incorporated by reference.

[0097] In a further aspect, a computer program is proposed which, when executed on a computer or a computer network, causes the computer or the computer network to fully or partially execute the method of calibrating at least one detector according to the invention, the computer program being configured to perform and / or execute at least steps i) to vi) of the method of calibrating at least one detector according to the invention. Similarly, a computer readable storage medium is disclosed which, when the program is executed by a computer or a computer network, causes the computer or the computer network to execute the method according to the invention, such as according to any one of the embodiments disclosed above and / or any one of the embodiments disclosed in more detail below. As used herein, the term "computer readable storage medium" may in particular refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer executable instructions. The computer readable data carrier or storage medium may in particular be or include a storage medium, such as a random access memory (RAM) and / or a read only memory (ROM).

[0098] Thus, in particular, one of the method steps, two or more of the method steps or even all of the method steps may be performed by using a computer or a computer network, preferably by using a computer program.

[0099] Further disclosed and proposed herein is a computer program product having program code means for carrying out the method according to the invention in one or more of the embodiments contained herein when said program is executed on a computer or a computer network. In particular, the program code means may be stored on a computer readable data carrier and / or a computer readable storage medium.

[0100] Further disclosed and proposed in this specification is a data carrier having a data structure stored thereon, such as in the working memory or main memory of a computer or computer network, which, after being loaded into the computer or computer network, is capable of performing the methods according to one or more of the embodiments disclosed in this specification.

[0101] Further disclosed and proposed herein is a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to one or more of the embodiments disclosed herein.

[0102] Further disclosed and proposed herein is a computer program product having program code means stored on a machine-readable carrier for performing the method according to one or more of the embodiments disclosed herein when the program is executed on a computer or a computer network. As used herein, a computer program product refers to a program as a tradeable product. The product may generally be present in any format, such as in paper format or on a computer-readable data carrier and / or a computer-readable storage medium. In particular, the computer program product may be distributed over a data network.

[0103] Also disclosed and suggested herein is a modulated data signal containing instructions readable by a computer system or computer network for carrying out a method according to one or more of the embodiments disclosed herein.

[0104] In particular, further disclosed herein are a computer or a computer network including at least one processor, the processor being adapted to execute a method according to one of the embodiments described in the present description; - a computer-loadable data structure adapted to carry out a method according to one of the embodiments described in the present description while the data structure is executed on a computer; - a computer program adapted to carry out a method according to one of the embodiments described in the present description while the program is run on a computer; a computer program comprising program means for carrying out a method according to one of the embodiments described in the present description while said computer program is being executed on a computer or a computer network; a computer program comprising program means according to the above-mentioned embodiments, the program means being stored on a computer readable storage medium; a storage medium on which a data structure is stored, the data structure being adapted to execute, after being loaded into a main and / or working storage of a computer or a computer network, a method according to one of the embodiments described in the present description; a computer program product comprising program code means, which can be or are stored on a storage medium for performing a method according to one of the embodiments described in the present description when said program code means are executed on a computer or a computer network; It is.

[0105] As used herein, the terms "have", "comprise", or "include", or any grammatical variants thereof, are used non-exclusively. Thus, they may refer both to the situation where, in addition to the features introduced by them, no further features are present in the entity described in this context, and to the situation where one or more further features are present. As an example, the expressions "A has B", "A comprises B", and "A includes B" may refer both to the situation where, in addition to B, no other elements are present in A (i.e., A solely and exclusively comprises B), and to the situation where, in addition to B, one or more further elements are present in entity A, such as element C, elements C and D, or further elements.

[0106] Furthermore, it should be noted that the terms "at least one," "one or more," or similar phrases indicating that a feature or element may be present one or more times are typically used only once when introducing each feature or element. In most cases, when referring to each feature or element, the phrases "at least one" or "one or more" will not be repeated, regardless of the fact that each feature or element may be present one or more times.

[0107] Furthermore, as used below, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically", or similar terms are used with any feature without limiting alternative possibilities. Thus, the features introduced by those terms are optional features and are not intended to limit the scope of the claims in any way. The invention can be practiced by using alternative features, as a person skilled in the art will recognize. Similarly, features introduced "in an embodiment of the invention" or similar expressions are intended to be optional features, without any limitations on alternative embodiments of the invention, without any limitations on the scope of the invention, and without any limitations on the possibility of combining features introduced in such a way with other optional or non-optional features of the invention.

[0108] Overall, in the context of the present invention, the following embodiments are considered preferred:

[0109] EMBODIMENT 1 1. A detector for determining a position of at least one object, comprising: at least one projector for illuminating an object with at least one illumination pattern, the illumination pattern including a plurality of illumination features; and - at least one sensor element having a matrix of optical sensors, each optical sensor having a photosensitive area, each optical sensor designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by a reflected light beam propagating from the object to a detector, the sensor element being configured to determine at least one reflection image comprising a plurality of reflection features, each of the reflection features comprising a beam profile; at least one evaluation device configured to determine initial distance information of the reflection features by analysis of a beam profile of each of the reflection features, the analysis of the beam profile comprising evaluating a combined signal Q from each of the sensor signals, the evaluation device comprising: a) matching reflectance features to reference features of a reference image taking into account initial distance information, thereby determining matched reflectance and reference feature pairs; b) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; c) determining an epipolar line distance d of the matched reflection feature to the epipolar line; d) estimating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; e) determining at least one correction to the rotation and / or translation of the reflected image as a function of the geometric pattern; an evaluation device configured to perform a calibration method comprising: A detector comprising:

[0110] EMBODIMENT 2 4. A detector according to any preceding embodiment, wherein the evaluation device is configured to correct the reflected image based on the determined correction.

[0111] EMBODIMENT 3 4. A detector according to any one of the preceding embodiments, wherein the evaluation device is configured to determine triangulation distance information of at least one of the reflection features by using triangulation taking into account the determined correction.

[0112] EMBODIMENT 4 A detector according to any preceding embodiment, wherein the evaluation device is configured to perform the calibration method on-the-fly while determining the triangulation distance information.

[0113] EMBODIMENT 5 A detector according to the previous embodiment, wherein the evaluation device is configured to determine at least one extrinsic parameter of the detector, the extrinsic parameter comprising at least one parameter selected from the group consisting of a rotation angle between the projector and the coordinates of the sensor element, a translation component between the projector and the coordinates of the sensor element, an aperture angle, a center of the sensor element, an aperture, a focal length.

[0114] EMBODIMENT 6 A detector according to any preceding embodiment, wherein the evaluation device is configured to perform steps b) to e) if an erroneous pair of matched reflection features and reference features is also determined.

[0115] EMBODIMENT 7 4. The detector of claim 1, wherein the evaluation device is configured to determine a correction for the reflected image by evaluating one or more of the shape, repetition, steepness, discontinuity, and curvature of the geometric pattern.

[0116] EMBODIMENT 8 A detector as described in the previous embodiment, wherein the illumination pattern includes at least one periodic regular pattern selected from the group consisting of at least one periodic regular point pattern, at least one hexagonal pattern, and at least one rectangular pattern.

[0117] EMBODIMENT 9 A detector according to any one of the preceding embodiments, wherein the evaluation device is configured to derive the combined signal Q by one or more of: dividing the sensor signal, dividing a multiple of the sensor signal, or dividing a linear combination of the sensor signals, and the evaluation device is configured to use at least one predetermined relationship between the combined signal Q and an ordinate for determining initial distance information.

[0118] EMBODIMENT 10 13. A detector according to any one of the preceding embodiments, wherein the evaluation device is configured to perform an image analysis of the reflected image, thereby identifying reflective features of the reflected image.

[0119] EMBODIMENT 11 A detector according to any one of the preceding embodiments, wherein the evaluation device is configured to determine, for each reflection feature, a longitudinal area of ​​the reflection feature, the longitudinal area being given by initial distance information of the reflection feature determined from the combined signal Q and an error interval ±ε, and the evaluation device is configured to determine at least one displacement area in the reference image corresponding to the longitudinal area.

[0120] EMBODIMENT 12 A detector according to any preceding embodiment, wherein the evaluation device is configured to match each one of the reflection features with a respective one of the reference features in the displacement region by using at least one linear scaling algorithm.

[0121] EMBODIMENT 13 A method of calibrating at least one detector according to any one of the preceding embodiments, the method comprising the steps of: vii) initial distance information, - illuminating the object with at least one illumination pattern generated by at least one projector of the detector, the illumination pattern including a plurality of illumination features; - generating, in response to the illumination, at least one sensor signal for each reflected light beam impinging on a photosensitive area of ​​an optical sensor of a sensor element having a matrix of optical sensors; - determining at least one reflected image by using a sensor element including a plurality of reflected features, each of the reflected features including a beam profile; - evaluating the sensor signals by using at least one evaluation device, thereby determining a combined signal Q and determining initial distance information of the reflective features by analysis of a beam profile of each of the reflective features, the analysis of the beam profile comprising evaluating the combined signal Q from each of the sensor signals. and viii) matching the reflectance features to reference features of a reference image taking into account the initial distance information, thereby determining a matched reflectance feature and reference feature pair; ix) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; x) determining an epipolar line distance d of the matched reflection feature to the epipolar line; xi) estimating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; xii) determining at least one correction to the rotation and / or translation of the reflected image as a function of the geometric pattern; A method comprising:

[0122] EMBODIMENT 14 The method according to the previous embodiment, comprising: correcting the reflection image based on the determined correction; and determining at least one triangulation distance information of the reflection feature by using triangulation taking into account the determined correction.

[0123] EMBODIMENT 15 A method of using a detector according to any one of the preceding embodiments with reference to the detector for a purpose of use selected from the group consisting of: position measurement in traffic technology, entertainment applications, security applications, surveillance applications, safety applications, human-machine interface applications, logistics applications, tracking applications, outdoor applications, mobile applications, communication applications, photography applications, machine vision applications, robotics applications, quality control applications, manufacturing applications.

[0124] Further optional details and features of the invention are evident from the following description of preferred exemplary embodiments together with the dependent claims. In this context, certain features may be implemented in isolation or in combination with other features. The invention is not limited to the exemplary embodiments. The exemplary embodiments are illustrated diagrammatically in the drawings. The same reference signs in the individual drawings refer to identical elements or elements having the same function or elements which correspond to each other in terms of their function. [Brief description of the drawings]

[0125] [Figure 1] 1 shows an embodiment of a detector according to the present invention. [Diagram 2] 1 shows the reflectance pattern of a calibrated detector matched to a reference pattern. [Figure 3A] 1 shows an embodiment of the reflectance pattern of a decalibrated detector (rotated) matched to a reference pattern. [Figure 3B] 1 shows an embodiment of the reflectance pattern of a decalibrated detector (rotated) matched to a reference pattern. [Figure 4] 13 shows a further embodiment of a reflected pattern and a matched reference pattern and an estimated epipolar line distance function d(x,y). [Diagram 5] 2 shows an exemplary flow chart of an embodiment of a method for calibrating at least one detector according to the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0126] 1 shows in highly schematic form an embodiment of a detector 110 for determining the position of at least one object 112 in accordance with the present invention. The detector 110 includes at least one sensor element 114 having a matrix 116 of optical sensors 118. Each of the optical sensors 118 has a light-sensitive area 120.

[0127] The sensor element 114 may be formed unitary, as a single device, or as a combination of several devices. The matrix 116 may in particular be or include a rectangular matrix with one or more rows and one or more columns. The rows and columns may in particular be arranged in a rectangular format. However, other arrangements are also possible, such as non-rectangular arrangements. By way of example, a circular arrangement is also possible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, the matrix 116 may be a single row of pixels. Other arrangements are possible.

[0128] The optical sensors 118 of the matrix 116 may in particular be equal in size, sensitivity and one or more of other optical, electrical and mechanical properties. The photosensitive areas 120 of all the optical sensors 118 of the matrix 116 may in particular be located in a common plane, which preferably faces the object 112, so that a light beam propagating from the object to the detector 110 may generate a light spot on the common plane. The photosensitive areas 120 may in particular be located on a surface of each optical sensor 118. However, other embodiments are feasible.

[0129] The optical sensor 118 may include, for example, at least one CCD device and / or CMOS device. By way of example, the optical sensor 118 may be part of or may constitute a pixelated optical device. By way of example, the optical sensor may be part of or may constitute at least one CCD device and / or CMOS device having a matrix of pixels, each pixel forming a photosensitive area 120. Preferably, the detectors are configured such that the optical sensor 118 is exposed simultaneously within a certain period of time, represented as a frame or an imaging frame. For example, the optical sensor 118 may be part of or may constitute at least one global shutter CMOS.

[0130] The optical sensors 118 may in particular be or include photodetectors, preferably inorganic photodetectors, more preferably inorganic semiconductor photodetectors, most preferably silicon photodetectors. In particular, the optical sensors 118 may be sensitive in the infrared spectral range. All of the optical sensors 118 of the matrix 116 or at least a group of the optical sensors 118 of the matrix 116 may in particular be identical. A group of identical optical sensors 118 of the matrix 116 may in particular be provided for different spectral ranges or all optical sensors may be identical with respect to their spectral sensitivity. Furthermore, the optical sensors 118 may be identical in size and / or with respect to their electronic or optoelectronic properties. The matrix 116 may be composed of independent optical sensors 118. Thus, the matrix 116 may be composed of inorganic photodiodes. Alternatively, however, commercially available matrices may be used, such as one or more of CCD detectors, such as CCD detector chips, and / or CMOS detectors, such as CMOS detector chips.

[0131] The optical sensors 118 may form or be part of a sensor array, such as the matrix mentioned above. Thus, by way of example, the detector 110 may include an array of optical sensors 118, such as a rectangular array with m rows and n columns, where m, n are independently positive integers. Preferably, more than one column and more than one row are provided, i.e. n>1, m>1. Thus, by way of example, n may be 2 to 16 or more, or m may be 2 to 16 or more. Preferably, the ratio of the number of rows and the number of columns is close to 1. By way of example, n and m may be selected such that 0.3≦m / n≦3, such as by choosing m / n=1:1, 4:3, 16:9, or similar. By way of example, the array may be a square array with equal numbers of rows and columns, such as by choosing m=2, n=2, or m=3, n=3, or similar.

[0132] The matrix 116 may in particular be a rectangular matrix having at least one row, preferably a number of rows, and a number of columns. By way of example, the rows and columns may be essentially vertically oriented. To provide a wide range of views, the matrix 116 may in particular have at least 10 rows, preferably at least 50 rows, more preferably at least 100 rows. Similarly, the matrix may have at least 10 columns, preferably at least 50 columns, more preferably at least 100 columns. The matrix 116 may include at least 50 optical sensors 118, preferably at least 100 optical sensors 118, more preferably at least 500 optical sensors 118. The matrix 116 may include some pixels in the multi-mega pixel range. However, other embodiments are feasible.

[0133] The detector 110 may further include a projector 122 for illuminating the object 112 with at least one illumination pattern 124. The projector 122 includes, among other things, at least one laser source 126 for generating at least one light beam. The projector 122 includes, among other things, at least one diffractive optical element 128 for generating and / or forming the illumination pattern 124 from the light beam of the laser source 126. The projector 122 may be configured such that the illumination pattern 124 propagates from the projector 122, among other things, from at least one opening 130 in a housing of the projector 122 towards the object 112. The projector 122 may be configured to generate and / or project a point cloud, for example, the projector 122 may include at least one digital light processing (DLP) projector, at least one LCoS projector, at least one laser source, at least one array of laser sources; at least one light emitting diode; at least one array of light emitting diodes. The laser source 126 may include focusing optics 134. The projector 122 may include multiple laser sources 126. In addition, additional illumination patterns may be generated by at least one ambient light source.

[0134] The projector 122 may include at least one control unit 136. The control unit 136 may be configured to control the laser source 126. The control unit 136 may include at least one processing device, in particular at least one processor and / or at least one application specific integrated circuit (ASIC). The control unit 136 may include one or more programmable devices, such as one or more computers, application specific integrated circuits (ASICs), digital signal processors (DSPs) or field programmable gate arrays (FPGAs), configured to perform the control of the laser source 126. The control unit 136 may include at least one processing device having a software code stored therein, including several computer commands. The control unit 136 may provide one or more hardware elements for performing the controlling of the laser source 126 and / or one or more processors having software running thereon for performing the control of the laser source. The control unit 136 may be configured to issue and / or generate at least one electronic signal for controlling the laser source. The control unit 136 may have one or more wireless and / or wired interfaces and / or other types of control connections for controlling the laser source 126. The control unit 136 and the laser source may be interconnected by one or more connectors and / or by one or more interfaces.

[0135] The illumination pattern 124 includes a plurality of illumination features 125. The illumination pattern 124 may include at least one periodic regular pattern selected from the group consisting of: at least one periodic regular point pattern; at least one hexagonal pattern; and at least one rectangular pattern.

[0136] 1 may include a single light source, specifically a single laser source 126, configured to generate at least one light beam, also referred to as a laser beam. Projector 122 may include at least one transfer device, specifically a DOE 128, for diffracting and replicating the laser beam generated by the single laser source to generate illumination pattern 124 that includes patterned illumination features. Diffractive optical element 128 may be configured for beam forming and / or beam splitting.

[0137] For example, the projector 122 may include at least one array of densely packed light sources, particularly laser sources 126, according to a configured pattern to generate clusters of light beams. The density of the laser sources 126 may depend on the elongation of the housings of the individual light sources and the distinguishability of the light beams. The projector 122 may include at least one transmission device, particularly a DOE 128, for diffracting and replicating the clusters of light beams to generate an illumination pattern 124 that includes patterned illumination features.

[0138] Each optical sensor 118 is designed to generate at least one sensor signal in response to illumination of its respective photosensitive area 120 by a reflected light beam propagating from the object 112 to the detector 110. Furthermore, the sensor elements 114 are configured to determine at least one reflected image 142 including at least one reflected pattern 138. The reflected image 142 may include points as reflected features. These points result from the reflected light beam originating from the object 112. The sensor elements 114 may be configured to determine the reflected pattern 138. The reflected pattern 138 may include at least one feature corresponding to at least one illumination feature 125 of the illumination pattern 124. The reflected pattern 138 may include at least one distortion pattern compared to the illumination pattern 124, the distortion depending on the distance of the object 112, such as the surface characteristics of the object 112.

[0139] The detector 110 may include at least one transfer device 140, which may include one or more of at least one lens, for example at least one lens selected from the group consisting of at least one adjustable focus lens, at least one aspheric lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflecting element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or a beam splitting mirror; at least one multi-lens system. In particular, the transfer device 140 may include at least one collimating lens configured to focus at least one object point in the image plane.

[0140] The detector 110 includes at least one evaluation device 144. The evaluation device 144 may be configured to select at least one reflection feature of the reflection image 142. The evaluation device 144 may be configured to select at least one feature of the reflection pattern 138 and determine initial distance information, i.e. the ordinate of the selected feature of the reflection pattern, by evaluating the combined signal Q from the sensor signals as described above. Thus, the detector 110 may be configured to pre-classify at least one reflection feature of the reflection image 142.

[0141] The evaluation device 144 may be configured to perform at least one image analysis and / or image processing to identify reflective features. The image analysis and / or image processing may use at least one feature detection algorithm. The image analysis and / or image processing may include one or more of the following: filtering; selecting at least one region of interest; forming a difference image between an image produced by the sensor signal and at least one offset; inverting the sensor signal by inverting the image produced by the sensor signal; forming a difference image between images produced by the sensor signal at different times; background correction; decomposition into color channels; decomposition into tone; saturation; and intensity channels; frequency decomposition; singular value decomposition; applying a Canny edge detector; applying a Laplacian of Gaussian filter; applying a Difference of Gaussian filter; applying a Sobel operator; applying a Laplace operator; applying a Scharr operator; applying a Prewitt operator; applying a Roberts operator; applying a Kirsch operator; applying a high pass filter; applying a low pass filter; applying a Fourier transform; applying a Radon transform; applying a Hough transform; applying a wavelet transform; thresholding; creating a binary image. The region of interest may be determined manually by a user, or may be determined automatically, such as by recognizing objects in images produced by the optical sensor 118.

[0142] The evaluation device 144 is configured to determine initial distance information, i.e., at least one ordinate z of a selected reflection feature of the reflection image 142, by evaluating a combined signal Q from the sensor signals. The evaluation device 144 may be configured to derive the combined signal Q by one or more of dividing the sensor signal, dividing a multiple of the sensor signal, dividing a linear combination of the sensor signals. The evaluation device 144 may be configured to use at least one predetermined relationship between the combined signal Q and the longitudinal area for determining the longitudinal area. For example, the evaluation device 144 may:

[0143]

number

[0144] Each of the sensor signals may include at least one information of at least one area of ​​the beam profile of the light beam. The photosensitive areas 120 may be arranged such that the first sensor signal includes information of the first area of ​​the beam profile and the second sensor signal includes information of the second area of ​​the beam profile. The first area of ​​the beam profile and the second area of ​​the beam profile may be one or both of adjacent areas or overlapping areas. The first area of ​​the beam profile and the second area of ​​the beam profile may not coincide in area.

[0145] The evaluation device 144 may be configured to determine and / or select a first area of ​​the beam profile and a second area of ​​the beam profile. The first area of ​​the beam profile may essentially include edge information of the beam profile and the second area of ​​the beam profile may essentially include center information of the beam profile. The beam profile may have a center, i.e. a maximum of the beam profile and / or a center point of the plateau of the beam profile and / or a geometric center of the light spot, and a falling edge extending from the center. The second area may include an inner area of ​​the cross section and the first area may include an outer area of ​​the cross section. Preferably, the center information has a ratio of edge information of less than 10%, more preferably less than 5%, and most preferably the center information does not include edge content. The edge information may include information of the entire beam profile, in particular from the center and edge areas. The edge information has a ratio of center information of less than 10%, preferably less than 5%, and more preferably the edge information does not include center content. At least one area of ​​the beam profile may be determined and / or selected as the second area of ​​the beam profile if it is near or around the center and essentially includes the center information. At least one area of ​​the beam profile may be determined and / or selected as the first area of ​​the beam profile if it includes at least a portion of the falling edge of the cross section. For example, the entire area of ​​the cross section may be determined as the first region. The first area of ​​the beam profile may be area A2, and the second area of ​​the beam profile may be area A1. Similarly, the center signal and the edge signal may be determined by using a section of the beam profile, such as a circular section of the beam profile. For example, the beam profile may be divided into two sections by a secant or chord that does not pass through the center of the beam profile. Thus, one section essentially includes the edge information and the other section essentially includes the center information. For example, the edge signal may be further subtracted from the center signal to further reduce the amount of edge information in the center signal.

[0146] The edge information may include information related to a number of photons in a first area of ​​the beam profile, and the center information may include information related to a number of photons in a second area of ​​the beam profile. The evaluation device 144 may be configured to determine an area integral of the beam profile. The evaluation device 144 may be configured to determine the edge information by integrating and / or summing the first area. The evaluation device 144 may be configured to determine the center information by integrating and / or summing the second area. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the integral of the trapezoid. Furthermore, when a trapezoidal beam profile may be assumed, the determination of the edge and center signals may be replaced with an equivalent evaluation that utilizes the characteristics of the trapezoidal beam profile, such as the determination of the slope and position of the edges and the height of the central plateau, and derives the edge and center signals according to geometrical considerations.

[0147] The evaluation device 144 may be configured to use at least one predetermined relationship between the combined signal and the ordinate. The predetermined relationship may be one or more of an empirical relationship, a semi-empirical relationship, and an analytically derived relationship. The evaluation device 144 may include at least one data storage device for storing the predetermined relationships, such as a look-up list or a look-up table.

[0148] Depth measurement using beam profile analysis using the combined signal Q may enable reliable distance determination even in environments that cause multiple reflections by biasing the light source or in the case of reflective measurement objects by reducing the computational requirements, in particular the processing power. Beam profile analysis may enable estimating a depth map from the image of the sensor element 114. In particular, the distances determined by the beam profile analysis may provide distance estimates for each illumination feature 125, which can be refined by triangulation methods with respect to known positions, in particular fixed positions, of the sensor element 114 and the projector 122. To calculate the refined ordinates using triangulation, a so-called correspondence problem needs to be solved. Generally, three-dimensional reconstruction methods using triangulation require an externally calibrated system. In the case of an externally calibrated system, each reflective feature can be matched with a reference grid point, i.e. a reference feature, together with the estimated initial distance information of the beam profile analysis. Thus, if the epipolar condition is satisfied and the beam profile analysis obtains a reliable depth estimate, each detected reflection feature can be matched to a corresponding reference grid point. However, if the correspondence is determined to be incorrect, the distance measurement based on triangulation becomes very poor. Depending on the hardware, a pre-calibrated detector may degrade due to physical stress or temperature shifts, resulting in a change in relative position and rotation in time. Changes in the relative positions of the sensor elements 114 and the projector 122 result in changes in the reflected image and an incorrectly determined correspondence of the reference and reflection features and thus an incorrect distance measurement result. The invention proposes to perform a calibration method, in particular a recalibration, for determining the extrinsic parameters of the detector 110. The calibration method according to the invention may allow to directly correct the incorrect distance measurement on the fly, without starting a new static calibration process. In particular, the calibration method may be performed automatically, such as without any user intervention.

[0149] The calibration may comprise at least one process for determining at least one extrinsic parameter of the detector 110 and / or for determining corrections to measurements of the detector 110, in particular to measurements of the positions of reflective features in the reflective image. The evaluation device 144 may be configured to determine at least one extrinsic parameter of the detector 110. The extrinsic parameter may comprise at least one parameter selected from the group consisting of a rotation angle between the coordinates of the projector 122 and the sensor element 114, a translation component between the coordinates of the projector 122 and the sensor element 114, an aperture angle, a center of the sensor element 114, an aperture, a focal length.

[0150] The calibration method includes matching the reflection features with reference features of a reference image, taking into account the initial distance information, thereby determining pairs of matched reflection and reference features. The evaluation device 144 may be configured to match each one of the reflection features with each one of the reference features in the displacement region by using at least one linear scaling algorithm. A beam profile analysis may allow the number of possibilities to be reduced.

[0151] The evaluation device 144 may be configured to determine at least one reference feature in at least one reference image that corresponds to at least one reflection feature. The evaluation device 144 may be configured to perform an image analysis and to identify features of the reflection image. The evaluation device 144 may be configured to identify at least one reference feature in the reference image that has essentially the same ordinate as the selected reflection feature. The reference feature that corresponds to the reflection feature may be determined using epipolar geometry. For a description of epipolar geometry, reference is made, for example, to X. Jiang, H. Bunke, Chapter 2: "Dreidimensionales Computersehen", Springer, Berlin Heidelberg, 1997. The epipolar geometry may assume that the reference image and the reflection image may be images of an object determined at different spatial positions and / or spatial orientations with a fixed distance. The reference image and the reflection image may be images of an object determined at different spatial positions with a fixed distance. The evaluation device 144 may be configured to determine an epipolar line in the reference image. The assumed relative positions of the reference image and the reflected image may be known. For example, the assumed relative positions of the reference image may be determined in a previous or historical calibration, in which steps a) to e) have been performed. For example, the assumed relative positions of the reference image and the reflected image may be manufacturer values. For example, the assumed relative positions of the reference image and the reflected image may be stored in at least one storage unit of the evaluation device. The evaluation device 144 may be configured to determine a straight line extending from a selected reflection feature of the reflected image. The straight line may include possible reflection features corresponding to the selected reflection feature. The straight line and the baseline span the epipolar plane. So that the reference image is determined in a different relative position from the reflected image, the corresponding possible reflection features may be imaged on a straight line in the reference image, a so-called epipolar line. Thus, the reference feature of the reference image corresponding to the selected reflection feature of the reflected image is assumed to be on the epipolar line.However, as outlined above, due to image distortions or changes in external parameters, such as due to aging, temperature changes, mechanical stress, etc., the epipolar lines may cross each other or be very close to each other and / or the correspondence between reference features and reflected features may become unclear.

[0152] The evaluation device 144 may be configured to determine, for each reflection feature, a longitudinal area of ​​the reflection feature. The longitudinal area may be given by the initial distance information of the reflection feature determined from the combined signal Q and the error interval ±ε. The evaluation device 144 may be configured to determine at least one displacement area in the reference image corresponding to the longitudinal area. The displacement area may be an area in the reference image in which a reference feature corresponding to the selected reflection feature may be imaged. In particular, the displacement area may be an area in the reference image in which a reference feature corresponding to the selected reflection feature is expected to be located in the reference image. Depending on the distance to the object, the image position of the reference feature corresponding to the reflection feature may be displaced in the reference image compared to the image position of the reflection feature in the reflection image. The displacement area may include only one reference feature. The displacement area may also include more than one reference feature.

[0153] The displacement area may include the epipolar line or a section of the epipolar line. The displacement area may include more than one epipolar line or more sections of more than one epipolar line. The displacement area may extend along the epipolar line, may be orthogonal to the epipolar line, or both. The evaluation device 144 may be configured to determine a reference feature along the epipolar line corresponding to the initial distance information and determine a degree of the displacement area along the epipolar line or orthogonal to the epipolar line corresponding to the error interval ±ε. The measurement uncertainty of the distance measurement using the combined signal Q may result in a displacement area that is non-circular, since the measurement uncertainty may be different for different directions. In particular, the measurement uncertainty along the epipolar line or epipolar line(s) may be larger than the measurement uncertainty in the orthogonal direction to the epipolar line or epipolar line(s). The displacement area may include a degree in the orthogonal direction to the epipolar line or epipolar line(s). The evaluation device may be configured to match the selected reflection feature with at least one reference feature in the displacement region. The evaluation device 144 may be configured to match the selected feature of the reflection image with the reference feature in the displacement region by using at least one evaluation algorithm taking into account the determined initial distance information. The evaluation algorithm may be a linear scaling algorithm. The evaluation device 144 may be configured to determine an epipolar line closest to and / or within the displacement region. The evaluation device may be configured to determine an epipolar line closest to the image location of the reflection feature. The extent of the displacement region along the epipolar line may be greater than the extent of the displacement region perpendicular to the epipolar line. The evaluation device 144 may be configured to determine the epipolar line before determining the corresponding reference feature. The evaluation device 144 may determine a displacement region around the image location of each reflection feature. The evaluation device 144 may be configured to assign an epipolar line to each displacement region at each image position of the reflective feature, such as by assigning an epipolar line closest to the displacement region and / or within the displacement region and / or closest to the displacement region along a direction perpendicular to the epipolar line.The evaluation device 144 may be configured to determine a reference feature corresponding to an image position of the reflection feature by determining a reference feature closest to the assigned displacement region and / or within the assigned displacement region and / or closest to the assigned displacement region along the assigned epipolar line and / or within the assigned displacement region along the assigned epipolar line.

[0154] Additionally or alternatively, the evaluation device 144 may be configured to perform the following steps: - determining a displacement area for an image location of each reflective feature; - assigning an epipolar line to the displacement area of ​​each reflection feature by assigning the epipolar line closest to the displacement area and / or within the displacement area and / or along a direction orthogonal to the epipolar line; - assigning at least one reference feature to each reflection feature and / or determining at least one reference feature to each reflection feature by assigning a reference feature closest to the assigned displacement area and / or within the assigned displacement area and / or closest to the assigned displacement area along the assigned epipolar line and / or within the assigned displacement area along the assigned epipolar line.

[0155] Additionally or alternatively, the evaluation device 144 may be configured to discriminate between more than one epipolar line and / or reference feature to be assigned to a reflection feature, such as by comparing distances of the reflection features and / or epipolar lines in the reference image and / or comparing error-weighted distances, such as ε-weighted distances, of the reflection features and / or epipolar lines in the reference image, and assigning the epipolar line and / or reference feature within a shorter distance and / or ε-weighted distance to the reference feature and / or reflection feature.

[0156] As outlined above, the detector 110, in particular the evaluation device 144, may be configured to use the combined signal Q to pre-classify selected reflection features, so that an ambiguous assignment to one reference feature is possible. In particular, the illumination features of the illumination pattern may be arranged such that corresponding reference features of the reference image may have a relative distance from one another on the epipolar line as long as possible. The illumination features of the illumination pattern may be arranged such that only a small number of reference features are located on the epipolar line.

[0157] Using the beam profile analysis may allow to estimate initial distance information, such as an ordinate in the error interval. By determining the displacement area corresponding to the initial distance information, the corresponding error interval may allow to reduce the possible number of solutions along the epipolar line for significantly matching the reference feature and the reflected feature. The number of possible solutions may even be reduced to one. The determination of the initial distance information may be performed during a pre-evaluation before matching the reflected feature and the reference feature. This may allow to reduce the computational requirements, which may result in significantly reduced costs and allow for use in mobile or outdoor devices.

[0158] The calibration method includes determining, for each matched reflection and reference feature pair, an epipolar line of the matched reference feature in the reference image, in particular, the epipolar line used to match the respective matched reference and reflection feature may be used as the epipolar line of said pair of matched reflection and reference features.

[0159] The calibration method further includes determining an epipolar line distance d of the matched reflection feature to the aforementioned epipolar line. The epipolar line distance may be the distance of the reflection feature and the epipolar line used to match the matched reference feature, represented as the corresponding epipolar line. The distance may be determined by determining the image coordinates of the reflection image and the image coordinates of the corresponding epipolar line and comparing the image coordinates. The minimum distance to the corresponding epipolar line may be used as the epipolar line distance.

[0160] In the case of a good external calibration, the epipolar line distance is close to zero. Such a reflection pattern 138 (circles) matching a reference pattern 146 (squares) of a calibrated detector 110 is shown in FIG.

[0161] In the case of a decalibrated detector, the reflection feature may be likely to match the reference feature. Figures 3A and 3B show two embodiments of the reflection pattern 138 (circle) of a decalibrated detector 110 in the case of rotated sensor elements 114 and / or projector 122 matching a reference pattern 146 (square). However, it may always be possible to determine the epipolar line distance to the incorrect corresponding reference feature or to the true corresponding reference feature. In the matching step, the reconstruction algorithm used may match the reflection feature to the reference feature if the epipolar line distance is within a tolerance range. It may therefore be possible that the reflection feature matches the incorrect reference feature, resulting in a non-zero epipolar line distance. If an incorrect pair of matched reflection and reference features is determined, the evaluation device 144 may be configured to perform steps b) to e). The calibration method may be based on evaluating the resulting epipolar line distance. The calibration method may take into account the epipolar distance independent of the fact if the correspondence between the reflection features and the reference features is correct or incorrect: the epipolar distance may be appropriate for the proposed calibration strategy even when the reflection features match the incorrect reference features.

[0162] Step d) includes evaluating the epipolar line distance d as a function of image position (x,y) in the reference image, thereby determining a geometric pattern. The geometric pattern may be a distribution of epipolar line distances. The epipolar line distance may be defined as a function d(x,y) on the reference image position (x,y). The epipolar line distance function d can be analyzed to calculate a correction for rotation and / or translation. In case of a decalibrated system, the function d may generate a geometric pattern. The shape of this geometric pattern of the epipolar line distance function may uniquely indicate the degree of decalibration. The geometric pattern, such as repetition, steepness, discontinuity, and curvature in the function d(x,y) can be used to restore the calibration. If the rotation and / or translation of the projector and / or sensor elements is changed, this result can be observed in the function d as a geometric pattern. The evaluation device 144 may be configured to execute an algorithm designed to analyze d(x,y) and calculate a correction for rotation and / or translation. The evaluation device 144 may be configured to determine a correction for the reflected image by evaluating one or more of the shape, repetition, steepness, discontinuity, and curvature of the geometric pattern. Figure 4 shows on the left a further embodiment of a reflected pattern and a matched reference pattern and on the right a further embodiment of an evaluated epipolar line distance function d(x,y).

[0163] The evaluation device 144 may be configured to correct the reflected image based on the determined correction. As outlined above, the reflected image refers to the evaluation of the image of the optical sensor with respect to at least one feature and / or the transformation of external parameters such as rotation and translation. The correction for the rotation and / or translation of the reflected image may be at least one correction factor applied to the image position of the reflected image. The evaluation device 144 may be configured to determine whether the geometric pattern matches within a predefined tolerance to the geometric pattern of the calibrated detector or whether the geometric pattern deviates from the geometric pattern of the calibration by more than a predefined tolerance. The determined correction may give information about the degree of decalibration. If the detector is already optimally calibrated, the correction step may be very small and the correction may also have a very small effect. In case of a match within the tolerance, the evaluation device 144 may keep the initial external parameters and / or the correction of the reflected image may be discarded. Otherwise, the correction may be applied to the reflected image. The determined correction may be used to correct the rotation and / or translation. The rotation and translation describe the spatial context between the sensor elements and the projector. The rotation and translation may include a relationship between a "reflection feature" and a "reference feature". From this relationship, triangulation distance information can be calculated by using triangulation techniques. Thus, in case of a determined change in rotation and / or translation, the resulting triangulation information can be corrected.

[0164] The evaluation device 144 may be configured to determine at least one triangulation distance information of the reflected feature by using triangulation taking into account the determined correction. The evaluation device 144 may be configured to determine a displacement of the matched reference feature and the reflected feature. The displacement may be a difference between a position in the reference image and a position in the reflected image. The evaluation device may be configured to determine the triangulation distance of the matched reference feature using a predetermined relationship between the ordinate and the displacement. The evaluation device 144 is configured to perform a calibration method on the fly while determining the triangulation distance information.

[0165] 5 shows an exemplary flow chart of an embodiment of a method for calibrating at least one detector according to the present invention. The method comprises the following steps: i) initial distance information (represented by reference numeral 148); - illuminating the object with at least one illumination pattern generated by at least one projector of the detector, the illumination pattern including a plurality of illumination features; - generating at least one sensor signal for each reflected light beam impinging on a light-sensitive area of ​​an optical sensor of a sensor element having a matrix of optical sensors in response to the illumination; - determining at least one reflected image by using a sensor element including a plurality of reflected features, each of the reflected features including a beam profile; - evaluating the sensor signals by using at least one evaluation device, thereby determining a combined signal Q, and determining initial distance information of the reflective features by analysis of a beam profile of each of the reflective features, the analysis of the beam profile comprising evaluating the combined signal Q from each of the sensor signals; and ii) (represented by reference numeral 150) matching reflectance features to reference features of a reference image taking into account the initial distance information, thereby determining matched reflectance and reference feature pairs; iii) for each matched reflection feature and reference feature pair (represented by reference numeral 152), determining an epipolar line of the matched reference feature in the reference image; iv) determining the epipolar line distance d of the matched reflection feature to the aforementioned epipolar line (represented by reference numeral 154); v) estimating the epipolar line distance d as a function of image position (x,y) in the reference image (represented by reference numeral 156) to thereby determine a geometric pattern; vi) determining at least one correction to the rotation and / or translation of the reflected image as a function of the geometric pattern (represented by reference numeral 158); Includes. [Explanation of symbols]

[0166] 110 Detector 112 Object 114 Sensor element 116 Matrix 118 Optical Sensor 120 Photosensitive Area 122 Projector 124 Irradiation Pattern 125 Irradiation Characteristics 126 Laser Source 128 DOE 130 Opening 132 Case 134 Optical system 136 Control Unit 138 Reflective Pattern 140 Transfer Device 142 Reflection Images 144 Evaluation Devices 146 Reference Patterns 148 Determining Initial Distance Information 150 matches 152 Determining the epipolar line 154 Determining the Epipolar Line Distance 156 Epipolar Line Distance Evaluation 158 Decision on at least one amendment

Claims

1. A detector (110) for determining the position of at least one object (112), - at least one projector (122) for irradiating the object (112) with at least one irradiation pattern (124), the irradiation pattern (124) including a plurality of irradiation features (125), at least one projector (122); - at least one sensor element (114) having a matrix (116) of optical sensors (118), each of the optical sensors having a photosensitive area (120), and each optical sensor (118) being designed to generate at least one sensor signal in response to irradiation of its respective photosensitive area (120) by a reflected light beam propagating from the object (112) to the detector (110), the sensor element being configured to determine at least one reflected image (142) including a plurality of reflection features, each of the reflection features including a beam profile, at least one sensor element (114); - at least one evaluation device (144) configured to determine initial distance information of the reflection features by analysis of the respective beam profiles of the reflection features, the analysis of the beam profile including evaluating a combined signal Q from the respective sensor signals, the evaluation device comprising: a) (150) matching the reflection features to reference features of a reference image taking into account the initial distance information, thereby determining pairs of matching reflection features and reference features; b) (152) for each of the pairs of matching reflection features and reference features, determining an epipolar line of the matching reference feature in the reference image; c) (154) determining an epipolar line distance d of the matching reflection feature to the epipolar line; d) (156) evaluating the epipolar line distance d as a function of the image position (x, y) in the reference image, thereby determining a geometric pattern; e) (158) determining at least one correction for rotation and / or translational movement of the reflected image (142) according to the geometric pattern; An evaluation device (144) configured to execute a calibration method including the above, A detector (110) comprising the above.

2. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to correct the reflected image (142) based on the determined correction.

3. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine at least one triangulation distance information of the reflection features by using triangulation in consideration of the determined correction.

4. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to execute the calibration method on-the-fly while determining the triangulation distance information.

5. The evaluation device (144) is configured to determine at least one external parameter of the detector, and the external parameter is a rotation angle between the coordinates of the projector (122) and the sensor element (114), a translational movement component between the coordinates of the projector (122) and the sensor element (114), an opening angle, the center of the sensor element, an opening, a focal length, and at least one parameter selected from the group consisting of the above, and the detector (110) according to claim 1.

6. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to execute steps b) to e) when incorrect pairs of matched reflection features and reference features are also determined.

7. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine the correction of the reflected image by evaluating one or more of the shape, repetition, sharpness, discontinuity, and curvature of the geometric pattern.

8. The detector (110) according to claim 1, wherein the irradiation pattern includes at least one periodic regular point pattern selected from the group consisting of at least one periodic regular point pattern, at least one hexagonal pattern, and at least one rectangular pattern.

9. The evaluation device (144) is configured to derive the combined signal Q by one or more of dividing the sensor signal, dividing multiples of the sensor signal, and dividing a linear combination of the sensor signals, and the evaluation device (144) is configured to use at least one predetermined relationship between the combined signal Q and the ordinate to determine the initial distance information. The detector (110) according to claim 1.

10. The evaluation device (144) is configured to perform image analysis of the reflection image (142), thereby identifying reflection features of the reflection image (142). The detector (110) according to claim 1.

11. The evaluation device (144) is configured to determine a longitudinal region of the reflection feature for each reflection feature, the longitudinal region being given by the initial distance information and the error interval ±ε of the reflection feature determined from the combined signal Q, and the evaluation device (144) is configured to determine at least one displacement region in the reference image corresponding to the longitudinal region. The detector (110) according to claim 1.

12. The evaluation device (144) is configured to match each one of the reflection features with each one of the reference features in the displacement region by using at least one linear scaling algorithm. The detector (110) according to claim 11.

13. A method of calibrating at least one detector (110) according to any one of claims 1 to 12, the method comprising the following steps: i) (148) Initial distance information is - irradiating the object (112) with at least one irradiation pattern (124) generated by the at least one projector (122) of the detector (110), the irradiation pattern (124) including a plurality of irradiation features (125), - generating at least one sensor signal for each reflected light beam impinging on the photosensitive area (120) of the optical sensor (118) of the sensor element (114) having a matrix (116) of optical sensors (118) in response to the irradiation. - Determining at least one reflected image (142) by using the sensor element (114) including a plurality of reflection features, each of the reflection features including a beam profile, - Evaluating the sensor signal by using at least one evaluation device (144), thereby determining a combined signal Q, and determining initial distance information of the reflection features by analyzing the respective beam profiles of the reflection features, the analysis of the beam profile including evaluating the combined signal Q from the respective sensor signals, Determining by: ii) (150) Matching the reflection features to reference features of a reference image in consideration of the initial distance information, thereby determining pairs of matched reflection features and reference features, iii) (152) For each of the pairs of matched reflection features and reference features, determining an epipolar line of the matched reference feature in the reference image, iv) (154) Determining an epipolar line distance d of the matched reflection feature to the epipolar line, v) (156) Evaluating the epipolar line distance d as a function of an image position (x, y) in the reference image, thereby determining a geometric pattern, vi) (158) Determining at least one correction for rotation and / or translational movement of the reflected image (142) according to the geometric pattern, A method comprising.

14. Correcting the reflected image (142) based on the determined correction, and determining at least one triangulation distance information of the reflection features by using triangulation in consideration of the determined correction, the method according to claim 13.

15. A method of using the detector (110) according to any one of claims 1 to 12 for a purpose of use selected from the group consisting of position measurement in traffic technology, entertainment applications, security applications, surveillance applications, safety applications, human-machine interface applications, logistics applications, tracking applications, outdoor applications, mobile applications, communication applications, photographic applications, machine vision applications, robotic applications, quality control applications, manufacturing applications, with reference to the detector.

Images