Automatic calibration from epipolar line distance within the projection pattern

Abstract

A detector (110) is provided for determining a position of at least one object (112). The detector (110) comprises: at least one projector (122) for illuminating the object (112) with at least one illumination pattern (124), the illumination pattern (124) including a plurality of illumination features (125); at least one sensor element (114) having a matrix (116) of optical sensors (118), each having a photosensitive area (120), each optical sensor (118) designed to generate at least one sensor signal in response to illumination of its respective photosensitive area (120) by a reflected light beam propagating from the object (112) to the detector (110), the sensor element being configured to determine at least one reflection image (142) comprising a plurality of reflection features, each of the reflection features comprising a beam profile; at least one evaluation device (144) configured to determine initial distance information of the reflection features by analysis of a beam profile of each of said reflection features, the analysis of the beam profile comprising evaluating a combined signal Q from each of the sensor signals, the evaluation device comprising: a) (150) matching reflectance features to reference features of a reference image, taking into account initial distance information, thereby determining pairs of matched reflectance features and reference features; b) (152) for each matched reflection feature and reference feature pair, determining an epipolar line of the matched reference feature in the reference image; c) determining the epipolar line distance d of the matched reflection feature to the (154) epipolar line; and d) (156) Evaluating the epipolar line distance d as a function of image location (x,y) in the reference image, thereby determining a geometric pattern; and e) (158) determining at least one correction to a rotation and / or translation of the reflected image (142) in response to the geometric pattern; and an evaluation device (144) configured to perform a calibration method including:

Description

[Technical Field]

[0001] The present invention relates to a detector for determining the position of at least one object and a method for calibrating the aforementioned detector. The present invention further relates to various uses of the detector. The devices, methods, and uses according to the present invention may be employed in various areas, for example, in daily life, gaming, traffic technology, production technology, security technology, photography such as digital photography or video photography for art, documentation or technical purposes, medical technology, or in science. Furthermore, the present invention may be used to scan one or more objects and / or scan scenery, for example, in the fields of architecture, metrology, archaeology, art, medicine, engineering, or manufacturing, such as to generate depth profiles of objects or scenery. However, other applications are also possible. [Background technology]

[0002] An active triangulation system typically includes at least one camera and at least one optical projector, e.g., a structured optical system. Other triangulation systems, such as stereo cameras, may include at least two cameras. Knowledge of the position and rotation of components such as cameras and projectors is essential for proper three-dimensional reconstruction by triangulation. In addition, three-dimensional reconstruction by triangulation also requires resolved correspondences of key points on the scene, e.g., laser spots, projector light spots, or detected edges captured by the cameras. The three-dimensional position can be calculated from the known translational movement and relative rotation of the camera to the projector. This parameter defines the external calibration of the triangulation system. Therefore, the quality of the three-dimensional measurement results depends on the external calibration. Depending on the hardware, an already calibrated system may degrade due to physical stress or temperature shifts, i.e., rotational changes in relative position and time. This can lead to erroneous measurement results. Obviously, the system may be restored by additional new calibration processes. Depending on the application, this can be time-consuming and impractical. Calibration can be based on capturing a static scene with a defined target at a known distance.

[0003] The concept of recalibration algorithms already exists, for example, E. Rehder et al., "Online Stereo Camera Calibration From Scratch," June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI:10.1109 / IVS.2017.7995952, and T. Dang, "Continuous Stereo Self-Calibration by Camera Parameter Tracking," August 2009, IEEE Transactions on Image Processing 8(7):1536-50, DOI:10.1109 / TIP.2009.2017824.

[0004] These recalibration approaches are based on discovering feature correspondences that should satisfy a system of equations (e.g., an epipolar condition) with respect to external parameters. A well-known example is the 8-point algorithm. For example, in the case of a three-dimensional measurement system with one camera and a laser dot projector, it can be assumed that the laser spots on the captured camera image are precisely assigned to the laser grid. This means that the correspondences can be precisely found. The relationship between the position of the laser spots on the camera image and the reference laser grid can be used to obtain a system of linear equations. These linear equations require at least eight corresponding laser spots on the reference grid. These equations can be solved by least-squares fitting. The adept utilization rate of the singular value decomposition can determine the rotation and translation of the camera and laser projector. However, if the correspondences are incorrect, the results can be very poor. Proper outlier detection of incorrect correspondences can be very important for methods of this type.

[0005] This procedure can also be used for stereo measurement. However, it may be necessary to find corresponding features in both camera images, such as edges or corners in the images. The location of these features can then be used, for example, to determine an 8-point method.

[0006] Generally, a third reconstruction method for structured light or stereo may require an externally calibrated system. This means that the search for correspondences is based on epipolar lines. However, this type of search is only one-dimensional. In the case of an uncalibrated system, the search for correspondences can no longer operate on one-dimensional epipolar lines. Conventional recalibration methods are supposed to search for correspondences within a two-dimensional image domain. This may follow an additional search algorithm for correspondences at the top of the three-dimensional reconstruction. [Prior art documents] [Non-patent literature]

[0007] [Non-Patent Document 1] E. Rehder et al., "Online Stereo Camera Calibration From Scratch," June 2017, Conference: 2017 IEEE Intelligent Vehicles Symposium, DOI: 10.1109 / IVS.2017.7995952 [Non-Patent Document 2] T. Dang, "Continuous Stereo Self-Calibration by Camera Parameter Tracking", August 2009 IEEE Transactions on Image Processing8(7):1536-50, DOI:10.1109 / TIP.2009.2017824 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] Therefore, an object of the present invention is to provide devices and methods that address the technical problems of known devices and methods mentioned above. In particular, an object of the present invention is to provide devices and methods that enable an automatic recalibration algorithm for a three-dimensional camera, preferably enabling the correction of erroneous measurements on the fly without initiating a new static calibration process. [Means for solving the problem]

[0009] This problem is solved by the present invention having the features of the independent claims. Advantageous developments of the present invention, which can be realized individually or in combination, are presented in the dependent claims and / or in the following specification and detailed embodiments.

[0010] A first aspect of the present invention discloses a detector for determining the position of at least one object.

[0011] As used herein, the term “object” means any object, in particular a surface or region configured to at least partially reflect at least one light beam that strikes the object. The light beam may originate from a projector of a detector illuminating the object, and the light beam is reflected or scattered by the object.

[0012] As used herein, the term “location” refers to at least one item of information relating to the location and / or orientation of an object and / or at least a portion of an object in space. Thus, at least one item of information may implicitly mean at least one distance between at least one point of the object and at least one detector. The distance may be a y-coordinate or may contribute to determining the y-coordinate of a point of the object. In addition or alternatively, one or more other items of information relating to the location and / or orientation of the object and / or at least a portion of an object may be determined. For example, in addition, at least one ax-coordinate of the object and / or at least a portion of an object may be determined. Thus, the location of an object may implicitly mean at least one y-coordinate of the object and / or at least a portion of an object. In addition or alternatively, the location of an object may implicitly mean at least one azimuthal information of the object indicating the orientation of the object in space.

[0013] The detector is - At least one projector for illuminating an object with at least one illumination pattern, wherein the illumination pattern includes a plurality of illumination features, - A sensor element having an optical sensor matrix, each optical sensor having a photosensitive area, each optical sensor designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by a reflected light beam propagating from an object to a detector, the sensor element configured to determine at least one reflection image comprising a plurality of reflection features, each reflection feature comprising a beam profile, and the at least one sensor element, - At least one evaluation device configured to determine initial distance information of reflection features by analyzing respective beam profiles of the reflection features, wherein the analysis of the beam profiles includes evaluating a combined signal Q from respective sensor signals, and the evaluation device is a) matching the reflection features to reference features in a reference image taking into account the initial distance information, thereby determining pairs of matched reflection and reference features; b) for each pair of matched reflection and reference features, determining an epipolar line of the matched reference feature in the reference image; c) determining an epipolar line distance d of the matched reflection feature to the aforementioned epipolar line; d) evaluating the epipolar line distance d as a function of the image position (x, y) in the reference image, thereby determining a geometric pattern; e) determining at least one correction for rotation and / or translational movement of the reflection image according to the geometric pattern, and configured to execute a calibration method including the above, an evaluation device, comprising.

[0014] As used herein, the term “projector,” also expressed as “optical projector,” refers to an optical device configured to project at least one illumination pattern onto an object, particularly onto the surface of an object. A projector may include at least one light source, also expressed as an illumination device or illumination source, configured to produce at least one light beam. A projector may be configured to produce at least one pattern and project the pattern toward at least one surface or scene, including an object. A projector may be configured for the illumination pattern to propagate from the projector toward an object, particularly from at least one aperture of the projector housing. A projector may be configured to produce and / or project a point cloud; for example, a projector may include at least one digital light processing (DLP) projector, at least one LCoS projector, at least one laser source, at least one array of laser sources; at least one light-emitting diode; at least one array of light-emitting diodes. In addition, additional illumination patterns may be produced by at least one ambient light source.

[0015] As used herein, the term "pattern" refers to any known or predefined arrangement that includes at least one arbitrarily shaped feature. A pattern can include at least one feature such as a point or symbol. A pattern can include a plurality of features. A pattern can include an array of periodic or non-periodic features. As used herein, the term "irradiation pattern" refers to a pattern generated and projected by a projector specifically used to irradiate an object. As used herein, the term "irradiation feature" refers to at least one arbitrarily shaped feature of an irradiation pattern. An irradiation pattern can include at least one periodic regular pattern selected from the group consisting of: at least one periodic regular point pattern; at least one hexagonal pattern; at least one rectangular pattern. For example, a projector can be configured to generate and / or project a point cloud. For example, a projector can be configured to generate a point cloud such that the irradiation pattern can include a plurality of point features.

[0016] A projector can be configured to generate a plurality of irradiation patterns each including a plurality of irradiation features. A projector can be configured to project two, three, four, five, or more irradiation patterns each including a plurality of irradiation features. The irradiation patterns can differ in particular in one or more of the number of irradiation features, the arrangement of irradiation features, the shape of irradiation features, the wavelength of irradiation features, the intensity of irradiation features, and the aperture angle.

[0017] A projector may include at least one transfer device, in particular at least one diffractive optical element, configured to generate an illumination pattern from at least one light beam produced by a laser source. The term “transfer device,” also expressed as “transfer system,” can generally refer to one or more optical elements, which are configured to modify the light beam, for example, by modifying one or more of the beam parameters of the light beam, the width of the light beam, or the direction of the light beam. A transfer device may particularly include one or more lenses selected from the group consisting of at least one lens, e.g., at least one focusable lens, at least one aspherical lens, at least one spherical lens, at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one of a beam splitting cube or beam splitting mirror; and at least one multi-lens system.

[0018] A transfer device may have an optical axis. In particular, a detector and a transfer device have a common optical axis. As used herein, the term “optical axis of a transfer device” generally refers to the axis of mirror symmetry or rotational symmetry of a lens or lens system. The optical axis of a detector may be the line of symmetry of the optical setup of the detector. A detector includes at least one transfer device, preferably at least one transfer system having at least one lens. The transfer system may include, for example, at least one beam path, and the elements of the transfer system in the beam path are positioned in a rotationally symmetrical manner with respect to the optical axis. Furthermore, as will be further outlined below, one or more optical elements located in the beam path may also be off-center or tilted with respect to the optical axis. However, in this case, the optical axis may be defined in series by interconnecting the centers of the optical elements in the beam path, for example by interconnecting the centers of the lenses, and in this context, optical sensors are not counted as optical elements. The optical axis may generally represent a beam path. There, the detector may have a single beampath through which the light beam can travel from the object to the optical sensor, or it may have multiple beampaths. For example, a single beampath may be given, or the beampath may be divided into two or more partial beampaths. In the latter case, each partial beampath may have its own optical axis. The optical sensor may be located in one and the same beampath or partial beampath. However, instead, the optical sensor may also be located in different partial beampaths.

[0019] The transfer device may constitute a coordinate system, where the vertex coordinate l is a coordinate along the optical axis, and d is a spatial offset from the optical axis. The coordinate system may be a polar coordinate system where the optical axis of the transfer device forms the z-axis, and the distance from the z-axis and the polar angle can be used as additional coordinates. A direction parallel or antiparallel to the z-axis may be considered a longitudinal direction, and a coordinate along the z-axis may be considered a vertex coordinate z. Any direction perpendicular to the z-axis may be considered a transverse direction, and the polar coordinate and / or polar angle may be considered a transverse coordinate.

[0020] As used herein, the term “beam” generally refers to a collection of light rays. Hereafter, the terms “ray” and “beam” are used synonymously. As further used herein, the term “light beam” generally refers to a quantity of light, in particular a quantity of light traveling in essentially the same direction, including the possibility of a light beam having a wide angle or aperture angle.

[0021] A light beam may include at least one beam profile. A light beam may have spatial extension. In particular, a light beam may have a non-Gaussian beam profile. The beam profile may be selected from a group consisting of trapezoidal beam profiles; triangular beam profiles; and conical beam profiles. A trapezoidal beam profile may have a plateau region and at least one edge region. A light beam may be a Gaussian light beam or a linear combination of Gaussian light beams. As used herein, the term “beam profile” refers to the spatial distribution of the intensity of a light beam, in particular, in at least one plane perpendicular to the propagation of the light beam. A beam profile may be a transverse intensity profile of a light beam. A beam profile may be a cross-section of a light beam. The beam profile may be selected from a group consisting of trapezoidal beam profiles; triangular beam profiles; conical beam profiles; and a linear combination of Gaussian beam profiles. However, other embodiments are possible. The projector may include at least one transfer device, the at least one transfer device may be configured for one or more of the following: adjusting, defining, and determining the shape of the beam profile.

[0022] For example, a projector may include at least one illumination source, such as a single light source, in particular a single laser source, configured to produce at least one light beam, also referred to as a laser beam. A projector may include at least one transfer device for diffracting and replicating the laser beam produced by the single laser source for producing an illumination pattern including patterned illumination features. In particular, a projector may include at least one diffractive optical element for diffracting and replicating the light beam. The diffractive optical element may be configured for beamforming and / or beam splitting. As used herein, the term “replicate” may mean producing multiple light beams from one light beam, in particular augmenting the light beams.

[0023] In addition, or instead, for example, the projector may include at least one array of tightly packed light sources, in particular laser sources, arranged according to a certain pattern configured to generate clusters of light beams. As used herein, the term “tightly packed” light sources may refer to multiple light sources arranged within a cluster. The density of the light sources may depend on the elongation of the housings of the individual light sources and the identifiability of the light beams. The projector may include at least one transfer device for diffracting and replicating clusters of light beams to generate an illumination pattern containing patterned illumination features.

[0024] As used herein, the term “sensor element” generally refers to a device or combination of devices configured to detect at least one parameter. In this case, the parameter may in particular be an optical parameter, and the sensor element may in particular be an optical sensor element. The sensor element may be formed unitarily, as a single device, or as a combination of several devices. As used further herein, the term “matrix” generally refers to an arrangement of elements in a predetermined geometric order. The matrix may in particular be a rectangular matrix having one or more rows and one or more columns, or may include a rectangular matrix, as outlined in more detail below. The rows and columns may in particular be arranged in a rectangular form. However, it should be outlined that other arrangements, such as non-rectangular arrangements, are possible. For example, a circular arrangement is also possible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, the matrix may be a single row of pixels. Other arrangements are possible.

[0025] The optical sensors in the matrix may be identical in size, sensitivity, and one or more of the other optical, electrical, and mechanical properties. The photosensitive areas of all the optical sensors in the matrix may be located in a common plane, which preferably faces an object, so that the light beam propagating from the object to the detector can produce a light spot on the common plane.

[0026] As used herein, “optical sensor” generally refers to a photosensitive device for detecting a light beam, such as for detecting an illumination and / or light spot generated by at least one light beam. As further used herein, “photosensitive area” generally refers to an area of ​​an optical sensor that can be illuminated externally by at least one light beam in response to illumination that generates at least one sensor signal. The photosensitive area may be located on the surface of each optical sensor. However, other embodiments are possible. As used herein, the term “optical sensor having at least one photosensitive area each” refers to a configuration having a plurality of single optical sensors, each having one photosensitive area, and a configuration having a single coupled optical sensor having a plurality of photosensitive areas. Thus, the term “optical sensor” further refers to a photosensitive device configured to generate one output signal, whereas in this specification, photosensitive devices configured to generate two or more output signals, such as at least one CCD and / or CMOS device, are referred to as two or more optical sensors. As outlined in more detail below, each optical sensor may be embodied such that each optical sensor has exactly one photosensitive area, for example, by providing exactly one photosensitive area that can be illuminated in response to illumination which creates exactly one uniform sensor signal for the entire optical sensor. Thus, each optical sensor may be a single-area optical sensor. However, the use of single-area optical sensors makes the detector setup particularly simple and efficient. Thus, commercially available photosensors, such as commercially available silicon photodiodes each having exactly one photosensitive area, may be used in the setup, for example. However, other embodiments are possible. Thus, for example, optical devices including two, three, four, or more photosensitive areas, which are considered as two, three, four, or more optical sensors, may be used in the context of the present invention. As outlined above, the sensor element includes a matrix of optical sensors. Thus, for example, an optical sensor may be part of or constitute a pixelated optical device.For example, an optical sensor may be part of or constitute at least one CCD device and / or CMOS device having a matrix of pixels, where each pixel forms a photosensitive area.

[0027] As outlined above, the optical sensors may be, or include, photodetectors, preferably inorganic photodetectors, more preferably inorganic semiconductor photodetectors, and most preferably silicon photodetectors. In particular, the optical sensors may be highly sensitive in the infrared spectral range. All or at least a group of optical sensors in the matrix may be identical. A group of identical optical sensors in the matrix may be provided for different spectral ranges, or all optical sensors may be identical with respect to spectral sensitivity. Furthermore, the optical sensors may be identical in size and / or with respect to their electronic or optical-electronic properties.

[0028] In particular, the optical sensor may be or may include an inorganic photodiode that is highly sensitive in the infrared spectral range, preferably in the range of 780 nanometers to 3.0 micrometers. In particular, the optical sensor may be highly sensitive in a part of the near-infrared region in which silicon photodiodes are particularly applicable in the range of 700 nanometers to 1000 nanometers. Infrared optical sensors that can be used for the optical sensor may be commercially available infrared optical sensors, such as the infrared optical sensor commercially available under the brand name Hertzstueck® from trinamX GmbH, D-67056 Ludwigshafen am Rhein, Germany. Thus, as an example, the optical sensor may include at least one optical sensor of the essential photovoltaic type, more preferably at least one semiconductor photodiode selected from the group consisting of Ge photodiodes, InGaAs photodiodes, extended InGaAs photodiodes, InAs photodiodes, InSb photodiodes, and HgCdTe photodiodes. In addition or alternatively, the optical sensor may include at least one externally photovoltaic optical sensor, more preferably at least one semiconductor photodiode selected from the group consisting of Ge:Au photodiode, Ge:Hg photodiode, Ge:Cu photodiode, Ge:Zn photodiode, Si:Ga photodiode, and Si:As photodiode. In addition or alternatively, the optical sensor may include at least one bolometer, preferably a bolometer selected from the group consisting of VO bolometers and amorphous Si bolometers.

[0029] The matrix may consist of independent optical sensors. Therefore, the matrix may consist of inorganic photodiodes. However, commercially available matrices may be used instead, such as one or more CCD detectors, such as a CCD detector chip, and / or CMOS detectors, such as a CMOS detector chip.

[0030] Therefore, generally speaking, the optical sensors of a detector may form or be part of a sensor array, such as the matrix mentioned above. Thus, for example, a detector may include an array of optical sensors, such as a rectangular array having m rows and n columns, where m and n are independently positive integers. Preferably, there are more than one column and more than one row, i.e., n>1, m>1. Thus, for example, n may be 2 to 16 or more, or m may be 2 to 16 or more. Preferably, the ratio of the number of rows to the number of columns is close to 1. For example, n and m may be chosen such that 0.3 ≤ m / n ≤ 3, such as by choosing m / n = 1:1, 4:3, 16:9, or similar. For example, the array may be a square array having an equal number of rows and columns, such as by choosing m=2, n=2 or m=3, n=3, or similar.

[0031] The matrix may be a rectangular matrix having at least one row, preferably more rows, and more columns. For example, the rows and columns may be oriented essentially perpendicularly, and with respect to the term “essentially perpendicular,” a reference to the definition given above may be made. Thus, for example, tolerances of less than 20 degrees, in particular less than 10 degrees or even less than 5 degrees may be acceptable. To provide a broad view, the matrix may have at least 10 rows, preferably at least 50 rows, more preferably at least 100 rows. Similarly, the matrix may have at least 10 columns, preferably at least 50 columns, more preferably at least 100 columns. The matrix may include at least 50 optical sensors, preferably at least 100 optical sensors, more preferably at least 500 optical sensors. The matrix may include some pixels within a multi-megapixel range. However, other embodiments are possible. Thus, in settings where axial rotational symmetry is expected, a circular or concentric arrangement of optical sensors in the matrix, also referred to as pixels, may be preferred.

[0032] Preferably, the sensor element may be oriented essentially perpendicular to the optical axis of the detector. Again, with respect to the term “essentially perpendicular,” reference may be made to the definitions and tolerances given above. The optical axis may be a straight optical axis, or it may be curved, or it may be further divided by using one or more deflection elements and / or by using one or more beam splitters, and the essentially perpendicular orientation in the latter case may refer to the local optical axis within each branch or beam path of the optical setup.

[0033] The reflected light beam can propagate from the object towards the detector. A projector can illuminate an object with an illumination pattern, and the light is reflected or scattered by the object, thereby being directed at least partially as a reflected light beam towards the detector.

[0034] Each optical sensor is designed to generate at least one sensor signal in response to the illumination of its respective photosensitive area by a reflected light beam propagating from an object to the detector. The reflected light beam may, in particular, completely illuminate the sensor element, resulting in the sensor element being entirely located within a light beam having a beam width greater than the matrix. Conversely, preferably, the reflected light beam may, in particular, create a light spot on the entire matrix that is smaller than the matrix, resulting in the light spot being entirely located within the matrix. This situation can be easily regulated by a person skilled in optics by selecting one or more suitable lenses or elements having a focusing or defocusing effect on the light beam, such as by using a suitable transfer device.

[0035] As used further herein, “sensor signal” generally refers to a signal generated by an optical sensor in response to irradiation by a light beam. In particular, a sensor signal may be or may include at least one electrical signal, such as at least one analog electrical signal and / or at least one digital electrical signal. More specifically, a sensor signal may be or may include at least one voltage signal and / or at least one current signal. More specifically, a sensor signal may include at least one photocurrent. Furthermore, an unprocessed sensor signal may be used, or a detector, optical sensor, or any other element may be configured to process or preprocess the sensor signal, thereby generating a secondary sensor signal that can also be used as a sensor signal, such as a preprocessed signal by filtering or the like.

[0036] The raw sensor signal of an optical sensor or a secondary sensor signal derived therefrom may be used for evaluation. As used herein, the term “secondary sensor signal” generally refers to a signal such as an electronic signal, more preferably an analog signal and / or a digital signal, obtained by processing one or more raw signals, such as by filtering, averaging, or demodulation. Thus, an image processing algorithm may be used to generate a secondary sensor signal from the entirety of the sensor signals in a matrix, or from a region of interest within the matrix. In particular, a detector such as an evaluation device may be configured to transform the sensor signal of an optical sensor, thereby generating a secondary optical sensor signal, and the evaluation device may be configured to perform the determination of initial distance information by using the secondary optical sensor signal. The transformation of the sensor signal may include at least one transformation selected from a group consisting of: filtering; selection of at least one region of interest; formation of a difference image between the image created by the sensor signal and at least one offset; inversion of the sensor signal by inverting the image created by the sensor signal; formation of a difference image between images created by the sensor signal at different times; background correction; decomposition into color channels; decomposition into tones; saturation; and brightness channels; frequency decomposition; singular value decomposition; application of a Canny edge detector; application of a Laplacian to a Gaussian filter; application of a Difference to a Gaussian filter; application of a Sobel operator; application of a Laplace operator; application of a Scherr operator; application of a Prewitt operator; application of a Roberts operator; application of a Kirsch operator; application of a high-pass filter; application of a low-pass filter; application of a Fourier transform; application of a Radon transform; application of a Hough transform; application of a wavelet transform; thresholding; and creation of a binary image. The region of interest may be determined manually by the user or automatically by recognizing objects in the image generated by the optical sensor.For example, a vehicle, a person, or another predetermined object can be determined by automatic image recognition within an image, i.e., within the entire sensor signal generated by an optical sensor, and the region of interest can be selected such that the object is located within the region of interest. In this case, evaluations such as determining the vertical coordinate can be performed only on the region of interest. However, other implementations are possible.

[0037] The photosensitive areas can be oriented toward an object. As used herein, the term “oriented toward an object” generally refers to a situation in which each surface of the photosensitive area is fully or partially visible from the object. In particular, at least one intersecting line between at least one point on the object and at least one point on each photosensitive area can form an angle by the surface elements of the photosensitive area that is different from 0 degrees, such as an angle in the range of 20 to 90 degrees, preferably 80 to 90 degrees, such as 90 degrees. Thus, when the object is located on or near the optical axis, the light beam propagating from the object toward the detector can be essentially parallel to the optical axis. As used herein, the term “essentially perpendicular” refers to a perpendicular orientation having, for example, a tolerance of ±20 degrees or less, preferably ±10 degrees or less, more preferably ±5 degrees or less. Similarly, the term “essentially parallel” refers to a state of parallel orientation having a tolerance of ±20 degrees or less, preferably ±10 degrees or less, and more preferably ±5 degrees or less.

[0038] Optical sensors may be highly sensitive in one or more of the ultraviolet, visible, or infrared spectral ranges. In particular, optical sensors may be highly sensitive in the visible spectral range between 500 nanometers and 780 nanometers, most preferably in the visible spectral range between 650 nanometers and 750 nanometers or between 690 nanometers and 700 nanometers. In particular, optical sensors may be highly sensitive in the near-infrared region. In particular, optical sensors may be highly sensitive in a portion of the near-infrared region in which silicon photodiodes are particularly applicable within the range of 700 nanometers to 1000 nanometers. Optical sensors may be particularly highly sensitive in the infrared spectral range, especially within the range of 780 nanometers to 3.0 micrometers. For example, each optical sensor may be, or include, at least one element selected from a group consisting of photodiodes, photocells, photoconductors, phototransistors, or any combination thereof. For example, an optical sensor may be or may include at least one element selected from a group consisting of CCD sensor elements, CMOS sensor elements, photodiodes, photocells, photoconductors, phototransistors, or any combination thereof. Any other type of photosensitive element may be used. As outlined in more detail below, photosensitive elements may generally be made entirely or partially from inorganic materials and / or entirely or partially from organic materials. Most commonly, as outlined in more detail below, one or more photodiodes, such as commercially available photodiodes, e.g., inorganic semiconductor photodiodes, may be used.

[0039] As used herein, the term “reflected image” refers to an image determined by an optical sensor including at least one reflectance feature, and / or an evaluation of the optical sensor’s image with respect to at least one feature, and / or a transformation of external parameters such as rotation and translation. Each reflectance feature includes a beam profile. As used herein, the term “reflectance feature” refers, for example, to a feature in the image plane generated by an object in response to illumination by at least one illumination feature. A reflected image may include at least one reflection pattern including at least one reflectance feature. As used herein, the term “determining at least one reflected image” refers to one or more of the imaging, recording, and generation of a reflected image.

[0040] The sensor element may be configured to determine at least one reflection pattern. As used herein, the term “reflection pattern” refers to a response pattern produced by the reflection or scattering of light on the surface of an object, in particular a response pattern produced by the object in response to illumination by an illumination pattern. The reflection pattern may include at least one feature corresponding to at least one feature of the illumination pattern. The reflection pattern may include at least one strain pattern compared to the illumination pattern, the strain being distance-dependent, such as the surface properties of the object. The evaluation device may be configured to select at least one feature of the reflection pattern and determine the longitudinal region of the selected feature of the reflection pattern by evaluating the coupled signal Q from the sensor signal, as described above and in further detail below.

[0041] As used further herein, the term “evaluation device” generally refers to any device configured to perform so-called operations, preferably by using at least one data processing device, and more preferably by using at least one processor and / or at least one application-specific integrated circuit. Thus, as an example, at least one evaluation device may include at least one data processing device having software code stored therein, which includes some computer commands. The evaluation device may provide one or more hardware elements for performing one or more so-called operations, and / or provide one or more processors with software running therein for performing one or more so-called operations. The operations mentioned above, including determining at least one vertex coordinate of an object, are performed by at least one evaluation device. Thus, as an example, one or more relationships may be implemented in software and / or hardware, such as by implementing one or more lookup tables, as outlined below. Therefore, for example, an evaluation device may include one or more programmable devices such as computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs) configured to perform the evaluations mentioned above in order to determine at least one vertex coordinate of an object. However, in addition to or instead of the evaluation device, the evaluation device may also be fully or partially embodied by hardware.

[0042] The evaluation device may be configured to select at least one reflection feature of a reflected image. The evaluation device may be configured to sequentially select reflection features of a reflected image. The evaluation device may be configured to perform image analysis of a reflected image, thereby identifying reflection features of the reflected image. As used herein, the term “selecting at least one reflection feature” means one or more of identifying, determining, and choosing at least one reflection feature of a reflected image. The evaluation device may be configured to perform at least one image analysis and / or image processing to identify reflection features. The image analysis and / or image processing may use at least one feature detection algorithm. Image analysis and / or image processing may include one or more of the following: filtering; selection of at least one region of interest; formation of a difference image between an image created by a sensor signal and at least one offset; inversion of the sensor signal by inverting an image created by a sensor signal; formation of a difference image between images created by a sensor signal at different times; background correction; decomposition into color channels; decomposition into tones; saturation; and brightness channels; frequency decomposition; singular value decomposition; application of a Canny edge detector; application of a Laplacian to a Gaussian filter; application of a Difference to a Gaussian filter; application of a Sobel operator; application of a Laplace operator; application of a Scharr operator; application of a Prewitt operator; application of a Roberts operator; application of a Kirsch operator; application of a high-pass filter; application of a low-pass filter; application of a Fourier transform; application of a Radon transform; application of a Hough transform; application of a wavelet transform; thresholding; and creation of a binary image. The region of interest may be determined manually by the user or automatically by recognizing objects in an image generated by an optical sensor.

[0043] As used herein, the term “initial distance information” may refer to the vertex coordinates determined by using the combined signal Q. The detector may be configured to determine the vertex coordinates of an object point for at least one reflection feature in the reflected image from the combined signal Q. Thus, the detector may be configured to pre-classify at least one reflection feature in the reflected image and / or provide a distance estimate for the reflection feature. In particular, the detector may be configured to determine at least one more accurate distance information for an object by using triangulation, taking into account the initial distance information, in particular the pre-classification and / or distance estimate.

[0044] The evaluation device may be configured to determine at least one initial distance piece of information, i.e., the y-coordinate z of a selected reflection feature in a reflection image, by analyzing the beam profile of each reflection feature. This technique is called beam profile analysis or depth ratio from photons and involves determining the y-coordinate by evaluating a combined signal Q from a sensor signal. Beam profile analysis, in particular the determination of the y-coordinate using a combined signal Q, is generally known to those skilled in the art from WO018 / 091649A1, WO2018 / 091638A1, and WO2018 / 091640A1, the contents of which are included by reference.

[0045] The beam profile analysis involves evaluating the combined signal Q from each sensor signal, in particular the combined signal Q of the sensor signals generated by optical sensors that detect reflected light beams on their photosensitive areas. Each optical sensor is designed to generate at least one sensor signal in response to the illumination of its respective photosensitive area by reflected light beams propagating from an object to the detector.

[0046] As used herein, the term “combined signal Q” refers to a signal generated by combining sensor signals, in particular by one or more of the following: splitting a sensor signal, splitting multiples of a sensor signal, or splitting a linear combination of sensor signals. An evaluation device may be configured to derive a combined signal Q by one or more of the following: splitting a sensor signal, splitting multiples of a sensor signal, or splitting a linear combination of sensor signals. An evaluation device may be configured to use at least one predetermined relationship between the combined signal Q and a longitudinal region for determining the longitudinal region.

[0047] For example, an evaluation device is

[0048]

number

[0049] Generally, the beam profile is luminance L(z o ) and beam shape S(x,y;z o ), E(x,y;z o ) = L·S depends. Therefore, by deriving the combined signal, it may be possible to determine the vertical coordinate independent of luminance. In addition, using the combined signal allows for the determination of a distance z independent of object size. o This enables the determination of distance z. Therefore, the combined signal is independent of the material properties and / or reflective and / or scattering properties of the object, and independent of alterations to the light source such as manufacturing precision, heat, water, dust, or damage to the lens. oThis enables the decision.

[0050] Each sensor signal may contain information from at least one area of ​​the beam profile of the optical beam. As used herein, the term “area of ​​beam profile” generally refers to any region of the beam profile at the sensor location used to determine the combined signal Q.

[0051] The photosensitive areas may be arranged such that the first sensor signal contains information about the first area of ​​the beam profile, and the second sensor signal contains information about the second area of ​​the beam profile. The first area and the second area of ​​the beam profile may be one or both adjacent or overlapping areas. The first area and the second area of ​​the beam profile may not coincide in terms of area.

[0052] The evaluation device may be configured to determine and / or select a first area of ​​the beam profile and a second area of ​​the beam profile. The first area of ​​the beam profile may essentially contain edge information of the beam profile, and the second area of ​​the beam profile may essentially contain central information of the beam profile. The beam profile may have a center, i.e., the center point of the maximum value and / or plateau of the beam profile and / or the geometric center of the light spot, and a falling edge extending from the center. The second area may include an internal area of ​​the cross-section, and the first area may include an external area of ​​the cross-section. As used herein, the term “essentially central information” generally refers to a low ratio of edge information, i.e., the ratio of intensity distributions corresponding to edges, i.e., the ratio of intensity distributions corresponding to the center, compared to a low ratio of edge information, i.e., the ratio of central information. Preferably, the central information has a ratio of less than 10% edge information, more preferably less than 5% edge information, and most preferably, the central information does not contain edge content. As used herein, the term “essentially central information” generally refers to a low ratio of central information compared to a low ratio of edge information. Edge information may include information about the entire beam profile, particularly from the central and edge regions. The edge information has a ratio of less than 10% central information, preferably less than 5% central information, and more preferably, the edge information does not include central content. At least one area of ​​the beam profile may be determined and / or selected as a second area of ​​the beam profile if it is close to or around the center and essentially contains central information. At least one area of ​​the beam profile may be determined and / or selected as a first area of ​​the beam profile if it includes at least a portion of the falling edge of the cross section. For example, the entire area of ​​the cross section may be determined as the first area. The first area of ​​the beam profile may be area A2, and the second area of ​​the beam profile may be area A1.

[0053] Other selections for the first area A1 and the second area A2 may be feasible. For example, the first area may essentially include the external region of the beam profile, and the second area may essentially include the internal region of the beam profile. For example, in the case of a two-dimensional beam profile, the beam profile may be divided into a left portion and a right portion, the first area may essentially include the area of ​​the left portion of the beam profile, and the second area may essentially include the area of ​​the right portion of the beam profile.

[0054] Edge information may include information relating to several photons in a first area of ​​the beam profile, and central information may include information relating to several photons in a second area of ​​the beam profile. The evaluation device may be configured to determine the area integral of the beam profile. The evaluation device may be configured to determine edge information by integrating and / or summing the first area. The evaluation device may be configured to determine central information by integrating and / or summing the second area. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the trapezoidal integral. Furthermore, when a trapezoidal beam profile is assumed, the determination of edge and central signals may be replaced by an equivalent evaluation that derives edge and central signals by geometric considerations, utilizing the characteristics of the trapezoidal beam profile, such as determining the slope and position of the edges and the height of the central plateau.

[0055] In addition, or alternatively, the evaluation device may be configured to determine either or both central information or edge information from at least one slice or cut of the optical spot. This can be achieved, for example, by replacing the area integral in the coupled signal Q with a line integral along the slice or cut. For improved accuracy, several slices or cuts through the optical spot may be used and averaged. In the case of an elliptic spot profile, averaging across several slices or cuts may result in improved distance information.

[0056] In one embodiment, a light beam propagating from an object to a detector may illuminate a sensor element with at least one reflection pattern comprising a plurality of feature points. As used herein, the term “feature point” refers to at least one partially extended feature of the pattern. Feature points may be selected from a group consisting of at least one point, at least one line, and at least one edge. The reflection pattern may be generated by an object in response to illumination by at least one light source with an illumination pattern comprising at least one pattern. A1 may correspond to the entire or complete area of ​​the feature point on the optical sensor. A2 may be the central area of ​​the feature point on the optical sensor. The central area may be a constant value. The central area may be smaller than the entire area of ​​the feature point. For example, in the case of a circular feature point, the central area may have a radius of 0.1 to 0.9, preferably 0.4 to 0.6, of the total radius of the feature point.

[0057] The evaluation device may be configured to derive a combined signal Q by one or more of the following: dividing edge information and central information, dividing multiples of edge information and central information, or dividing a linear combination of edge information and central information. Thus, essentially, the photon ratio can be used as the physical basis of the technique.

[0058] For example, an evaluation device is - Determine at least one optical sensor having the best sensor signal and form at least one central signal, - Evaluate the sensor signals of the matrix optical sensors and form at least one summation signal, - To determine at least one combined signal by combining the center signal and the total signal, - By evaluating the combined signal, determine the vertex z of at least one of the selected features, This can be configured to evaluate the sensor signal.

[0059] As a result, according to the present invention, the term “center signal” generally refers to at least one sensor signal that essentially contains central information of a beam profile. For example, the center signal may be the signal of at least one optical sensor having the best sensor signal from a plurality of sensor signals generated by optical sensors in the entire matrix or in a region of interest within the matrix, the region of interest may be predetermined or discriminable within the image generated by the optical sensors of the matrix. As used herein, the term “best sensor signal” refers to either or both a local maximum value or a maximum value in the region of interest. The center signal may arise from a single optical sensor or from a group of optical sensors, as outlined in more detail below, in the latter case, for example, the sensor signals of the group of optical sensors may be added, integrated, or averaged to determine the center signal. The group of optical sensors from which the center signal arises may be a group of adjacent optical sensors, such as optical sensors located less than a predetermined distance from the actual optical sensor having the best sensor signal, or a group of optical sensors that produce sensor signals within a predetermined range from the best sensor signal. The group of optical sensors from which the central signal originates can be selected as large as possible to allow for the maximum dynamic range. The evaluation device may be configured to determine the central signal by integrating multiple sensor signals, for example, multiple optical sensors around the optical sensor with the best sensor signal. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the trapezoidal integral, in particular the integral of the trapezoidal plateau.

[0060] As outlined above, the central signal can generally be a single sensor signal, such as a sensor signal from an optical sensor within the center of a light spot; or a combination of multiple sensor signals, such as a combination of sensor signals originating from optical sensors within the center of a light spot; or a secondary sensor signal derived by processing sensor signals derived from one or more of the above possibilities. The determination of the central signal can be performed electronically, or entirely or partially by software, since the comparison of sensor signals is adequately and simply implemented by conventional electronic equipment. In particular, the central signal may be selected from a group consisting of: the best sensor signal; the average of a group of sensor signals that are within a predetermined tolerance range from the best sensor signal; the average of sensor signals from a predetermined group of optical sensors that includes the optical sensor with the best sensor signal and adjacent optical sensors; the sum of sensor signals from a predetermined group of optical sensors that includes the optical sensor with the best sensor signal and adjacent optical sensors; the sum of sensor signals from a group of sensor signals that are within a predetermined tolerance range from the best sensor signal; the average of a group of sensor signals that exceed a predetermined threshold; the sum of a group of sensor signals that exceed a predetermined threshold; the integral of sensor signals from a predetermined group of optical sensors that includes the optical sensor with the best sensor signal and adjacent optical sensors; the integral of a group of sensor signals that are within a predetermined tolerance range from the best sensor signal; and the integral of a group of sensor signals that exceed a predetermined threshold.

[0061] Similarly, the term “total signal” generally refers to a signal that essentially contains edge information of the beam profile. For example, the total signal may be derived by adding sensor signals, integrating with respect to sensor signals, or averaging with respect to sensor signals of the entire matrix or a region of interest within the matrix, the region of interest may be predetermined or discriminable within the image generated by the optical sensors of the matrix. When adding, integrating with respect to, or averaging with respect to sensor signals, the actual optical sensors from which the sensor signals are generated may be excluded from adding, integrating, or averaging, or instead may be included in adding, integrating, or averaging. The evaluation device may be configured to determine the total signal by integrating the signals of the entire matrix or the signals of a region of interest within the matrix. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the integral of the entire trapezoid. Furthermore, when a trapezoidal beam profile is assumed, the determination of edge and center signals may be replaced by an equivalent evaluation that derives edge and center signals by geometric considerations, utilizing the characteristics of the trapezoidal beam profile, such as determining the slope and position of the edges and the height of the central plateau.

[0062] Similarly, the center signal and edge signal can also be determined by using sections of the beam profile, such as circular sections of the beam profile. For example, a beam profile can be divided into two sections by a secant or code that does not pass through the center of the beam profile. Thus, one section essentially contains edge information, while the other section essentially contains center information. For example, to further reduce the amount of edge information in the center signal, the edge signal can be further subtracted from the center signal.

[0063] In addition, or alternatively, the evaluation device may be configured to determine either or both central information or edge information from at least one slice or cut of the optical spot. This can be achieved, for example, by replacing the area integral in the coupled signal Q with a line integral along the slice or cut. For improved accuracy, several slices or cuts through the optical spot may be used and averaged. In the case of an elliptic spot profile, averaging across several slices or cuts may result in improved distance information.

[0064] The combined signal may be a signal produced by combining the center signal and the sum signal. In particular, the combination may include one or more of the following: forming a quotient or inverse quotient of the center signal and the sum signal; forming a quotient or inverse quotient of multiples of the center signal and multiples of the sum signal; or forming a quotient or inverse quotient of a linear combination of the center signals and a linear combination of the sum signal. In addition or alternatively, the combined signal may include any signal or combination of signals that includes at least one item of information relating to the comparison between the center signal and the sum signal.

[0065] The detection of the center of a light spot, i.e., the detection of a center signal and / or the detection of at least one optical sensor from which the center signal originates, may be performed entirely or partially electronically, or entirely or partially by using one or more software algorithms. In particular, the evaluation device may include at least one center detector for detecting at least one best sensor signal and / or for forming a center signal. The center detector may be fully or partially embodied in software and / or fully or partially embodied in hardware. The center detector may be fully or partially integrated into at least one sensor element and / or fully or partially embodied independently of the sensor element.

[0066] The total signal can be derived from all sensor signals in the matrix, from sensor signals within the region of interest, or from one of those possibilities resulting from sensor signals from optical sensors contributing to the excluded central signal. In all cases, a reliable total signal, which can be reliably compared to the central signal, can be generated to determine the y-coordinate. Generally, the total signal can be selected from a group consisting of: the average of all sensor signals in the matrix; the sum of all sensor signals in the matrix; the integral of all sensor signals in the matrix; the average of all sensor signals in the matrix excluding the sensor signals from those optical sensors contributing to the central signal; the sum of all sensor signals in the matrix excluding the sensor signals from those optical sensors contributing to the central signal; the integral of all sensor signals in the matrix excluding the sensor signals from those optical sensors contributing to the central signal; the sum of sensor signals from optical sensors within a predetermined range from the optical sensor with the best sensor signal; the integral of sensor signals from optical sensors within a predetermined range from the optical sensor with the best sensor signal; the sum of sensor signals above a certain threshold from optical sensors located within a predetermined range from the optical sensor with the best sensor signal; and the integral of sensor signals above a certain threshold from optical sensors located within a predetermined range from the optical sensor with the best sensor signal. However, other options exist.

[0067] Summarization can be performed entirely or partially in software and / or entirely or partially in hardware. Summarization is generally possible by purely electronic means, which can typically be easily implemented in detectors. Thus, in the field of electronics, summing devices are generally known for summing two or more electrical signals, analog signals and digital signals. Thus, an evaluation device may include at least one summing device for forming a summed signal. The summing device may be fully or partially integrated into a sensor element, or may be embodied entirely or partially independently of the sensor element. The summing device may be fully or partially embodied in either hardware or software, or both.

[0068] The comparison between the center signal and the sum signal can be performed, in particular, by forming one or more quotient signals. Thus, generally, the combined signal Q may be a quotient signal derived from one or more of the following: forming the quotient or inverse quotient of the center signal and the sum signal; forming the quotient or inverse quotient of multiples of the center signal and multiples of the sum signal; forming the quotient or inverse quotient of a linear combination of the center signal and a linear combination of the sum signal; forming the quotient or inverse quotient of the center signal and a linear combination of the sum signal and the center signal; forming the quotient or inverse quotient of the sum signal and a linear combination of the sum signal and the center signal; forming the quotient or inverse quotient of a power of the center signal and a power of the sum signal. However, other options exist. The evaluation device may be configured to form one or more quotient signals. The evaluation device may further be configured to determine at least one vertex coordinate by evaluating at least one quotient signal.

[0069] The evaluation device is configured to use at least one predetermined relationship between a combined signal Q and a y-coordinate to determine initial distance information, in particular by using at least one known, discriminable, or predetermined relationship between sensor signals. Specifically, the evaluation device is configured to determine at least one coordinate of an object by using at least one known, discriminable, or predetermined relationship between a quotient signal derived from sensor signals and a y-coordinate. The predetermined relationship may be one or more of empirical relationships, semi-empirical relationships, and analytically derived relationships. The evaluation device may include at least one data storage device for storing the predetermined relationships, such as a lookup list or lookup table.

[0070] Therefore, due to the reasons disclosed above and the dependence on the characteristics of the light spot on the y-coordinate, the combined signal Q is typically a monotonic function of the y-coordinate of the object and / or a monotonic function of the size of the light spot, such as the diameter of the light spot or an equal diameter. Therefore, as an example, particularly in cases where a linear optical sensor is used, the sensor signal scenter and the total signal s sum a simple quotient, Q = s center / s sum can be a monotonically decreasing function of distance. Without wishing to be bound by this theory, in the preferred setting described above, as the amount of light reaching the detector decreases, the central signal s center and the total signal s sum both decrease as quadratic functions, which is believed to be due to the fact that the distance to the light source increases. However, there, in the optical setting used in the experiment, the light spot in the image plane grows and thus spreads over a larger area, so the central signal s center decreases faster than the total signal s sum . Thus, the quotient of the central signal and the total signal decreases continuously while increasing the diameter of the light beam or the diameter of the light spot on the photosensitive area of the matrix optical sensor. Since the total output of the light beam forms a factor in both the central signal and the total sensor signal, the quotient is also typically independent of the total output of the light beam. As a result, the combined signal Q can form a unique and ambiguous relationship between the central signal and the total signal, and a secondary signal providing the size or diameter of the light beam. On the other hand, since the size or diameter of the light beam depends on the distance between the object from which the light beam propagates towards the detector and the detector itself depends on, i.e., the ordinate of the object, there can be a unique and ambiguous relationship between the central signal and the total signal on the one hand and the ordinate on the other hand. The predefined relationship can be determined by empirical measurements such as measuring the derived secondary signal as a function of the combined signal and / or the central signal and the total signal or the ordinate of the object, by assuming a linear combination of Gaussian light beams, or by both, etc., by analytical considerations.

[0071] The combined signal Q can be determined by various means. For example, software means for deriving the quotient signal, hardware means for deriving the quotient signal, or both may be used and implemented in the evaluation device. Thus, the evaluation device may include, for example, at least one divider, which is configured to derive the quotient signal. The divider may be fully or partially embodied as either a software divider or a hardware divider, or both. The divider may be fully or partially integrated into the sensor element answer, or may be embodied fully or partially independently of the sensor element.

[0072] Depth measurements using beam profile analysis can enable reliable distance determination even in environments that produce multiple reflections due to biasing the light source, or in the case of reflectance measurement objects by reducing computational requirements, particularly by reducing processing power. Beam profile analysis can enable the estimation of depth maps from images of sensor elements. In particular, the distances determined by beam profile analysis can provide distance estimates for each illumination feature and can be refined by triangulation methods with respect to known positions of sensor elements and projectors, especially fixed positions. To calculate the refined vertex coordinates using triangulation, the so-called correspondence problem must be solved. Generally, three-dimensional reconstruction methods using triangulation require an externally calibrated system. In the case of an externally calibrated system, each reflection feature can be matched to a reference grid point, i.e., a reference feature, along with the initial distance information estimated from beam profile analysis. Therefore, if the epipolar condition is met and beam profile analysis yields a reliable depth estimate, each detected reflection feature can be matched to a corresponding reference grid point. However, if the correspondence is determined to be incorrect, distance measurements based on triangulation become very poor. Depending on the hardware, a pre-calibrated detector may degrade due to physical stress or temperature shifts, resulting in changes in relative position and rotation over time. Changes in the relative positions of the sensor element and projector result in changes in the reflected image, as well as incorrectly determined correspondences between reference and reflected features, thus leading to erroneous distance measurement results. The present invention proposes a calibration method, in particular, to perform recalibration, for determining the external parameters of a detector. The calibration method according to the present invention may enable the direct correction of erroneous distance measurements on the fly without initiating a new static calibration process. In particular, the calibration method may be performed automatically, such as without any user intervention. The term “automatically” is a broad term as used herein and will be given its original and customary meaning to those skilled in the art, and is not limited to any special or customized meaning.This term may, in particular, without limitation, refer to a process that is entirely performed by means of at least one computer and / or computer network and / or machine, without manual action and / or user intervention.

[0073] As used herein, the term “calibration” is a broad term and will be given its original and customary meaning to those skilled in the art, and is not limited to any special or customized meaning. The term calibration may refer to at least one process for determining at least one external parameter of a detector and / or, in particular, for determining a correction to the detector’s measurements of the position of reflective features in a reflected image. An evaluation device may be configured to determine at least one external parameter of a detector. The external parameter may include at least one parameter selected from the group consisting of rotation angle between the coordinates of the projector and the sensor element, translation component between the coordinates of the projector and the sensor element, aperture angle, center of the sensor element, aperture, and focal length.

[0074] The calibration method includes matching the reflection features to the reference features of a reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features.

[0075] As used herein, the term “reference image” refers to an image distinct from the reflected image, determined at a different spatial location compared to the reflected image. A reference image is determined by one or more of the following: recording at least one reference feature, imaging at least one reference feature, or calculating the reference image. In particular, a reference image includes at least one reference pattern, also represented as a reference grid, which contains multiple reference features. As used herein, the term “reference feature” refers to at least one feature of the reference image. The reference image and the reflected image may be images of an object determined at different spatial locations with a fixed distance between them. The distance may be a relative distance, also called a baseline.

[0076] For example, the reference image could be a reference grid, such as an image of the illumination pattern on the image plane at the projector's position. The projector and sensor elements may be separated by a fixed distance.

[0077] For example, the detector may include at least two sensor elements, each having an optical sensor matrix. At least one first sensor element and at least one second sensor element may be positioned at different spatial locations. The relative distance between the first sensor element and the second element may be fixed. At least one first sensor element may be configured to determine at least one first reflection pattern, in particular at least one first reflection feature, and at least one second sensor element may be configured to determine at least one second reflection pattern, in particular at least one second reflection feature. The evaluation device may be configured to select at least one image determined by the first or second sensor element as a reflection image, and to select at least one image determined by the other of the first or second sensor element as a reference image.

[0078] The evaluation device may be configured to match each of the reflection features with each of the reference features in the displacement region by using at least one linear scaling algorithm. Beam profile analysis may allow for a reduction in the number of possibilities.

[0079] The evaluation device may be configured to determine at least one reference feature in at least one reference image corresponding to at least one reflected feature. The evaluation device may be configured to perform image analysis and identify features in the reflected image. The evaluation device may be configured to identify at least one reference feature in the reference image having essentially identical vertices to the selected reflected feature. The term “essentially identical” means identical by 10%, preferably 5%, and most preferably 1%. The reference feature corresponding to the reflected feature may be determined using epipolar geometry. For an explanation of epipolar geometry, refer to, for example, Chapter 2: “Dreidimensionales Computersehen” in X. Jiang and H. Bunke, Springer, Berlin Heidelberg, 1997. Epipolar geometry may assume that the reference image and the reflected image may be images of an object determined at different spatial locations and / or spatial orientations with a fixed distance between them. The evaluation device may be configured to determine the epipolar line in the reference image. The assumed relative positions of the reference image and the reflected image may be known. For example, the assumed relative position of the reference image may be determined in a prior calibration or hysteretic calibration, in which steps a) to e) are performed. For example, the assumed relative positions of the reference image and the reflected image may be manufacturer values. For example, the assumed relative positions of the reference image and the reflected image may be stored in at least one memory unit of the evaluation device. The evaluation device may be configured to determine a straight line extending from a selected reflective feature of the reflected image. The straight line may include possible reflective features corresponding to the selected reflective feature. The straight line and baseline extend to the epipolar plane. The corresponding possible reflective features may be imaged on a straight line in the reference image, so-called epipolar line, so that the reference image is determined at a different relative position from the reflected image. Thus, the reference feature of the reference image corresponding to the selected reflective feature of the reflected image is assumed to lie on the epipolar line.However, as outlined above, due to image distortion or changes in external parameters such as aging, temperature changes, and mechanical stress, epipolar lines may intersect with each other or become very close to each other, and / or the correspondence between reference features and reflected features may become unclear.

[0080] The evaluation device may be configured to determine the longitudinal region of each reflective feature. The longitudinal region may be given by the initial distance information and error interval ±ε of the reflective feature determined from the combined signal Q. The evaluation device may be configured to determine at least one displacement region in the reference image corresponding to the longitudinal region. As used herein, the term “displacement region” refers to a region in the reference image from which a reference feature corresponding to a selected reflective feature may be imaged. In particular, a displacement region may be a region in the reference image from which the reference feature corresponding to a selected reflective feature is expected to be located. Depending on the distance to the object, the image position of the reference feature corresponding to the reflective feature may be displaced in the reference image compared to the image position of the reflective feature in the reflective image. A displacement region may contain only one reference feature. A displacement region may also contain more than one reference feature.

[0081] The displacement region may include an epipolar line or a section of an epipolar line. The displacement region may include more than one epipolar line or more sections of more than one epipolar line. The displacement region may extend along an epipolar line, be perpendicular to an epipolar line, or both. An evaluation device may be configured to determine a reference feature along an epipolar line corresponding to initial distance information and to determine the extent of a displacement region along an epipolar line or a displacement region perpendicular to an epipolar line corresponding to an error interval ±ε. Measurement uncertainty in distance measurements using a coupled signal Q can result in a non-circular displacement region because the measurement uncertainty can differ in different directions. In particular, the measurement uncertainty along an epipolar line or epipolar line(s) can be greater than the measurement uncertainty in directions perpendicular to the epipolar line or epipolar line(s). The displacement region may include the extent of displacement in directions perpendicular to an epipolar line or epipolar line(s). The evaluation device may be configured to match a selected reflection feature to at least one reference feature in the displacement region. As used herein, the term “matching” means determining and / or evaluating the corresponding reference and reflection features. The evaluation device may be configured to match a selected feature of a reflection image to a reference feature in the displacement region by using at least one evaluation algorithm, taking into account the determined initial distance information. The evaluation algorithm may be a linear scaling algorithm. The evaluation device may be configured to determine the epipolar line closest to and / or within the displacement region. The evaluation device may be configured to determine the epipolar line closest to the image position of the reflection feature. The extent of the displacement region along the epipolar line may be greater than the extent of the displacement region perpendicular to the epipolar line. The evaluation device may be configured to determine the epipolar line before determining the corresponding reference feature. The evaluation device may determine the displacement region around the image position of each reflection feature. The evaluation device may be configured to assign epipolar lines to each displacement region of each image position of a reflective feature, for example, by assigning the epipolar line closest to the displacement region and / or within the displacement region and / or along a direction perpendicular to the epipolar line.The evaluation device may be configured to determine a reference feature corresponding to the image position of a reflection feature by determining a reference feature in the assigned displacement region that is closest to and / or within the assigned displacement region and / or along the assigned epipolar line.

[0082] In addition, or instead, the evaluation device performs the following steps: - Determine the displacement region of each reflection feature relative to the image position. - Assigning epipolar lines to the displacement region of each reflection feature by assigning the epipolar line closest to the displacement region and / or within the displacement region and / or along the direction perpendicular to the epipolar line. - Assigning at least one reference feature to each reflection feature and / or determining at least one reference feature to each reflection feature by assigning a reference feature in the assigned displacement region that is closest to and / or within the assigned displacement region and / or along the assigned epipolar line. It can be configured to perform [this action].

[0083] In addition, or instead, the evaluation device may be configured to distinguish between more than one epipolar line and / or reference feature that would be assigned to a reflective feature, such as by comparing the distances of epipolar lines in the reflective feature and / or reference image, and / or by comparing error weighting distances, such as the ε-weighted distances of epipolar lines in the reflective feature and / or reference image, and by assigning epipolar lines and / or reference features within shorter distances and / or ε-weighted distances to reference features and / or reflective features.

[0084] Detectors, particularly evaluation devices, may be configured to pre-classify selected reflection features using a coupled signal Q, thereby enabling ambiguous assignment to a single reference feature. In particular, the illumination features of an illumination pattern may be arranged such that the corresponding reference features of a reference image can have as long a relative distance from each other as possible along the epipolar line. The illumination features of an illumination pattern may be arranged such that only a small number of reference features are located along the epipolar line.

[0085] Using beamprofile analysis can enable the estimation of initial distance information, such as vertex coordinates within error intervals. By determining the displacement regions corresponding to this initial distance information, the corresponding error intervals can reduce the number of possible solutions along the epipolar line required to significantly match the reference and reflection features. The number of possible solutions can even be reduced to one. The determination of initial distance information can be performed during pre-evaluation before matching the reflection and reference features. This can reduce computational requirements, resulting in a significant cost reduction and enabling use in mobile or outdoor devices.

[0086] The calibration method includes determining the epipolar line of the matched reference feature in the reference image for each matched pair of reflective and reference features. In particular, the epipolar line used to match each matched reference and reflective feature may be used as the epipolar line of the aforementioned pair of matched reflective and reference features.

[0087] The calibration method further includes determining the epipolar line distance d of the matched reflection feature to the epipolar line described above. As used herein, the term “epipolar line distance” may refer to the distance of the reflection feature and the distance of the epipolar line used to match the matched reference feature, which is represented as the corresponding epipolar line. The distance may be determined by determining the image coordinates of the reflection image and the image coordinates of the corresponding epipolar line and comparing the image coordinates. The minimum distance to the corresponding epipolar line may be used as the epipolar line distance. In the case of good external calibration, the epipolar line distance is close to zero. This is precisely the epipolar condition. In the case where the detector is decalibrated, the reflection feature may be likely to match the reference feature. However, it may always be possible to determine an epipolar line distance to an incorrect or true corresponding reference feature. In the matching process, the reconstruction algorithm used may match the reflection feature to the reference feature if the epipolar line distance is within an acceptable range. Therefore, it is possible that the reflected feature may match an incorrect reference feature, resulting in a non-zero epipolar line distance. If an incorrect pair of matched reflected and reference features is determined, the evaluation device may be configured to perform steps b) to e). The calibration method may be based on evaluating the resulting epipolar line distance. The calibration method may consider the epipolar distance independently of the fact that the correspondence between the reflected and reference features is accurate or incorrect. The epipolar distance may be appropriate for the proposed calibration strategy even when the reflected feature matches an incorrect reference feature.

[0088] Step d) includes evaluating the epipolar line distance d as a function of the image position (x,y) in the reference image, thereby determining the geometric pattern. As used herein, the term “geometric pattern” may refer to the distribution of the epipolar line distance. The epipolar line distance may be defined as a function d(x,y) on the position (x,y) in the reference image. The epipolar line distance function d can be analyzed to calculate corrections for rotation and / or translational movement. In the case of a decalibrated system, the function d may generate a geometric pattern. The shape of this geometric pattern of the epipolar line distance function may uniquely indicate the degree of decalibration. Geometric patterns such as repetition, steepness, discontinuity, and curvature in the function d(x,y) can be used to restore calibration. If the rotation and / or translational movement of the projector and / or sensor elements changes, this result can be observed in the function d as a geometric pattern. The evaluation device may be configured to run an algorithm designed to analyze d(x,y) and calculate corrections for rotation and / or translational movement. The evaluation device may be configured to determine the correction of the reflected image by evaluating one or more of the geometric pattern's shape, repeatability, steepness, discontinuity, and curvature.

[0089] The evaluation device may be configured to correct the reflected image based on the determined correction. As outlined above, the reflected image refers to the evaluation of the optical sensor image with respect to at least one feature and / or the transformation of external parameters such as rotation and translation. The correction for rotation and / or translation of the reflected image may be at least one correction factor applied to the image position of the reflected image. The evaluation device may be configured to determine whether the geometric pattern matches within a predefined tolerance to the geometric pattern of the calibrated detector, or whether the geometric pattern deviates from the calibration geometric pattern by a larger than predefined tolerance. The determined correction may provide information about the degree of decalibration. If the detector is already optimally calibrated, the correction step may be very small, and the correction may have a very small effect. In the case of a match within tolerance, the evaluation device may maintain the initial external parameters and / or discard the correction of the reflected image. Otherwise, the correction may be applied to the reflected image. The determined correction may be used to correct rotation and / or translation. Rotation and translation describe the spatial context between the sensor element and the projector. Rotation and translation may include a relationship between "reflection features" and "reference features." From this relationship, triangulation distance information can be calculated using triangulation techniques. Therefore, in cases of determined changes in rotation and / or translation, the resulting triangulation information can be corrected.

[0090] The evaluation device may be configured to determine at least one triangulation distance information for a reflected feature by using triangulation, taking into account the determined corrections. As used herein, the term “triangulation distance” may refer to the vertex coordinate determined by using triangulation. The evaluation device may be configured to determine the displacements of the matched reference feature and the reflected feature. As used herein, the term “displacement” may refer to the difference between the position in the reference image and the position in the reflected image. The evaluation device may be configured to determine the triangulation distance of the matched reference feature using a predetermined relationship between the vertex coordinate and the displacement. The evaluation device is configured to perform a calibration method on the fly while determining the triangulation distance information.

[0091] As outlined above, the detector may be configured to determine at least one vertex coordinate of an object, including the option to determine the vertex coordinate of the entire object or the vertex coordinate of one or more parts thereof. However, other coordinates of the object, including one or more transverse coordinates and / or rotational coordinates, may be determined by the detector, in particular by the evaluation device. Thus, as an example, one or more transverse sensors may be used to determine at least one transverse coordinate of an object. Generally, various transverse sensors are known in the art, such as the transverse sensors disclosed in WO2014 / 097181A1 and / or other position-sensitive devices (PSDs) such as quadrant diodes, CCD chips, or CMOS chips. In addition or instead, as an example, the detector according to the present invention may include one or more PSDs disclosed in RAStreet (Ed.): Technology and Applications of Amorphous Silicon, Springer-Verlag Heidelberg, 2010, pp. 346-349. Other embodiments are feasible. Those devices may also generally be implemented in the detector according to the present invention. For example, a portion of a light beam can be separated within the detector by at least one beam splitting element. The separated portion can be guided toward a lateral sensor, such as a CCD chip, CMOS chip, or camera sensor, and the lateral position of the light spot generated by the separated portion on the lateral sensor can be determined, thereby determining at least one transverse coordinate of an object. As a result, the detector according to the present invention may be a one-dimensional detector, such as a simple distance measuring device, or it may be embodied as a two-dimensional detector or even a three-dimensional detector. Furthermore, a three-dimensional image can also be created by scanning scenery or the environment in one-dimensional form, as outlined above or outlined in more detail below. As a result, the detector according to the present invention may be, in particular, one of a one-dimensional detector, a two-dimensional detector, or a three-dimensional detector. The evaluation device may further be configured to determine at least one transverse coordinate x,y of an object. The evaluation device may be configured to combine vertical coordinate information and transverse coordinate information to determine the position of an object in space.

[0092] In further embodiments, the present invention discloses a method for calibrating at least one detector according to the present invention, including relating to one or more embodiments that reference detectors such as those disclosed above or which are disclosed in more detail below. The method comprises the following method steps, which may be performed in a given order or in a different order. Furthermore, there may be one or more additional method steps that are not described. Furthermore, one of the method steps, two or more of the method steps, or all of the method steps may be repeated.

[0093] The method involves the following steps: i) Initial distance information, - Illuminating an object with at least one illumination pattern generated by at least one projector of the detector, wherein the illumination pattern includes a plurality of illumination features. - In response to irradiation, generate at least one sensor signal for each reflected light beam that collides on the photosensitive area of ​​the optical sensor of a sensor element having an optical sensor matrix. - Determining at least one reflected image by using a sensor element that includes multiple reflection features, wherein each reflection feature includes a beam profile. - Evaluating sensor signals using at least one evaluation device, thereby determining the combined signal Q, and determining initial distance information of reflection features by analyzing the beam profiles of each reflection feature, wherein the beam profile analysis includes evaluating the combined signal Q from each sensor signal. The decision will be made by, ii) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features. iii) For each matched pair of reflective and reference features, determine the epipolar line of the matched reference feature in the reference image, iv) Determining the epipolar line distance d of the reflection feature that matches the epipolar line, v) Evaluate the epipolar line distance d as a function of the image position (x,y) in the reference image, and thereby determine the geometric pattern. vi) Determine at least one correction for rotation and / or translation of the reflected image according to the geometric pattern, Includes.

[0094] The method may include correcting the reflected image based on the determined correction. The method may further include determining at least one triangulation distance information for the reflected feature by using triangulation, taking the determined correction into consideration.

[0095] For details, options, and definitions, references to detectors as discussed above may be made. Thus, in particular, as outlined above, the method may involve using a detector according to the present invention, such as relating to one or more embodiments given above or given in more detail below.

[0096] In further aspects of the present invention, methods of using the detectors according to the present invention are proposed for the purpose of use, relating to one or more embodiments given above or in further detail below, selected from the group consisting of: location measurement in traffic technology; entertainment applications; security applications; surveillance applications; safety applications; human-machine interface applications; tracking applications; photography applications; imaging applications or camera applications; mapping applications for generating a map of at least one space; homing or tracking beacon detectors for vehicles; outdoor applications; mobile applications; communication applications; machine vision applications; robotics applications; quality control applications; and manufacturing applications. Further uses of the detectors and devices of the present invention are referred to WO2018 / 091649A1, WO2018 / 091638A1, and WO2018 / 091640A1, the contents of which are included by reference.

[0097] In a further embodiment, a computer program is proposed that, when executed on a computer or computer network, causes a computer or computer network to fully or partially execute a method for calibrating at least one detector according to the present invention, the computer program being configured to perform and / or execute at least steps i) to vi) of the method for calibrating at least one detector according to the present invention. Similarly, a computer-readable storage medium is disclosed that, when the program is executed by a computer or computer network, contains instructions causing a computer or computer network to execute a method according to the present invention, such as relating to one of the embodiments disclosed above and / or one of the embodiments disclosed in further detail below. As used herein, the term “computer-readable storage medium” may in particular mean non-temporary data storage means, such as a hardware storage medium storing computer-executable instructions. The computer-readable data carrier or storage medium may in particular be or include storage media such as random access memory (RAM) and / or read-only memory (ROM).

[0098] Therefore, in particular, one of the method steps, two or more of the method steps, or even all of the method steps may be performed by using a computer or computer network, preferably by using a computer program.

[0099] Further disclosed and proposed herein are computer program products having program code means for performing a method according to the present invention in one or more embodiments included herein when the program is executed on a computer or computer network. In particular, the program code means may be stored in a computer-readable data carrier and / or computer-readable storage medium.

[0100] Further disclosed and proposed herein are data carriers having data structures stored therein, which, after being loaded into the computer or computer network, can perform methods according to one or more embodiments disclosed herein.

[0101] Further disclosed and proposed herein is a non-temporary computer-readable medium that, when executed by one or more processors, includes instructions causing one or more processors to execute one or more methods relating to one or more embodiments disclosed herein.

[0102] Further disclosed and proposed herein are computer program products having program code means stored in a machine-readable carrier for performing one or more methods relating to the embodiments disclosed herein when the program is executed on a computer or computer network. As used herein, a computer program product refers to a program as a tradable product. Products can generally exist in any format, such as in paper format or on a computer-readable data carrier and / or computer-readable storage medium. In particular, computer program products can be distributed through data networks.

[0103] Furthermore, disclosed and proposed herein are modulated data signals that include instructions readable by a computer system or computer network for performing a method relating to one or more embodiments disclosed herein.

[0104] In particular, further disclosed herein is: - A computer or computer network comprising at least one processor, wherein the processor is adapted to perform a method relating to one of the embodiments described herein, - A computer-loadable data structure adapted to perform a method according to one of the embodiments described herein while the data structure is executed on a computer, - A computer program, wherein the computer program is adapted to perform a method according to one of the embodiments described herein while the program is running on a computer, - A computer program that includes programming means for performing a method according to one of the embodiments described herein, while the computer program is running on a computer or computer network, - A computer program including the program means according to the embodiment described above, wherein the program means is stored in a computer-readable storage medium, and the computer program is stored in a computer-readable storage medium. - A storage medium on which a data structure is stored, and which, after being loaded into the main storage and / or working storage of a computer or computer network, is adapted to perform a method according to one of the embodiments described herein. - A computer program product having program code means, wherein the program code means can be stored in a storage medium to perform a method according to one of the embodiments described herein when the program code means is executed on a computer or computer network, That is the case.

[0105] As used herein, the terms “have,” “comprise,” or “include,” or any grammatical variation thereof, are used non-exclusively. Thus, these terms can refer to both situations in which the entity described in this context has no further features in addition to the features introduced by these terms, and situations in which one or more further features exist. For example, the expressions “A has B,” “A comprises B,” and “A includes B” can refer to both situations in which A has no other elements in addition to B (i.e., A constitutes B exclusively), and situations in which entity A has one or more further elements in addition to B, such as element C, elements C and D, or further elements.

[0106] Furthermore, it should be noted that the terms “at least one,” “one or more,” or similar expressions indicating that a feature or element may exist one or more times, are typically used only once when introducing each feature or element. In most cases, when referring to each feature or element, the expression “at least one” or “one or more” is not repeated, regardless of the fact that each feature or element may exist one or more times.

[0107] Furthermore, as used below, the terms “preferably,” “more preferably,” “particularly,” “more particularly,” “specifically,” “more specifically,” or similar terms are used with any feature without limiting the possibility of alternatives. Thus, the features introduced by those terms are optional features and are not intended to limit the scope of the claims in any way. The present invention can be carried out by using alternative features, as will be recognized by those skilled in the art. Similarly, features or similar expressions introduced in “in an embodiment of the invention” are intended to be optional features without any limitation relating to alternative embodiments of the invention, without any limitation relating to the scope of the invention, and without any limitation relating to the possibility of combining the features introduced in such a manner with any other optional or non-optional features of the invention.

[0108] Overall, in the context of the present invention, the following embodiments are considered preferred:

[0109] Embodiment 1 A detector that determines the position of at least one object, - At least one projector for illuminating an object with at least one illumination pattern, wherein the illumination pattern includes a plurality of illumination features, - A sensor element having an optical sensor matrix, each optical sensor having a photosensitive area, each optical sensor designed to generate at least one sensor signal in response to illumination of its respective photosensitive area by a reflected light beam propagating from an object to a detector, the sensor element configured to determine at least one reflection image comprising a plurality of reflection features, each reflection feature comprising a beam profile, and the at least one sensor element, - At least one evaluation device configured to determine initial distance information of a reflection feature by analyzing the beam profile of each reflection feature, wherein the beam profile analysis includes evaluating the combined signal Q from each sensor signal, and the evaluation device a) Considering the initial distance information, match the reflection features to the reference features of the reference image, thereby determining the matched pair of reflection features and reference features. b) For each matched pair of reflection features and reference features, determine the epipolar line of the matched reference feature in the reference image, c) Determining the epipolar line distance d of the reflection feature that matches the epipolar line, d) Evaluate the epipolar line distance d as a function of the image position (x,y) in the reference image, and thereby determine the geometric pattern. e) Determine at least one correction for rotation and / or translation of the reflected image according to the geometric pattern. An evaluation device configured to perform a calibration method including, A detector equipped with the following features.

[0110] Embodiment 2 The evaluation device is the detector according to the above embodiment, configured to correct the reflected image based on the determined correction.

[0111] Embodiment 3 The detector according to any one of the embodiments described above, wherein the evaluation device is configured to determine at least one triangulation distance information of a reflection feature by using triangulation, taking into account the determined correction.

[0112] Embodiment 4 The detector according to the above embodiment, wherein the evaluation device is configured to perform a calibration method on the fly while determining triangulation distance information.

[0113] Embodiment 5 The detector according to the above embodiment, wherein the evaluation device is configured to determine at least one external parameter of the detector, the external parameter being at least one parameter selected from the group consisting of rotation angle between the coordinates of the projector and the sensor element, translation component between the coordinates of the projector and the sensor element, aperture angle, center of the sensor element, aperture, and focal length.

[0114] Embodiment 6 The detector according to the above embodiment is configured to perform steps b) to e) if an incorrect pair of matched reflection features and reference features is also determined.

[0115] Embodiment 7 The detector according to the above embodiment, wherein the evaluation device is configured to determine correction of the reflected image by evaluating one or more of the shape, repeatability, steepness, discontinuity, and curvature of the geometric pattern.

[0116] Embodiment 8 The detector according to the above embodiment, wherein the irradiation pattern includes at least one periodic normal pattern selected from a group consisting of at least one periodic normal point pattern, at least one hexagonal pattern, and at least one rectangular pattern.

[0117] Embodiment 9 The detector according to any one of the above embodiments, wherein the evaluation device is configured to derive a combined signal Q by one or more of the following: dividing a sensor signal, dividing a multiple of a sensor signal, or dividing a linear combination of sensor signals, and the evaluation device is configured to use at least one predetermined relationship between the combined signal Q and a y-coordinate for determining initial distance information.

[0118] Embodiment 10 The evaluation device is a detector according to any one of the embodiments described above, configured to perform image analysis of a reflected image and thereby identify the reflection features of the reflected image.

[0119] Embodiment 11 The detector according to any one of the embodiments described above, wherein the evaluation device is configured to determine the longitudinal region of each reflection feature, the longitudinal region being given by initial distance information and error interval ±ε of the reflection feature determined from a combined signal Q, and the evaluation device is configured to determine at least one displacement region in a reference image corresponding to the longitudinal region.

[0120] Embodiment 12 The evaluation device is the detector according to the above embodiment, configured to match each of the reflection features with each of the reference features in the displacement region by using at least one linear scaling algorithm.

[0121] Embodiment 13 A method for calibrating at least one detector as described in any one of the embodiments described above, the method comprising the following steps: vii) Initial distance information, - Illuminating an object with at least one illumination pattern generated by at least one projector of the detector, wherein the illumination pattern includes a plurality of illumination features. - In response to irradiation, generate at least one sensor signal for each reflected light beam that collides on the photosensitive area of ​​the optical sensor of a sensor element having an optical sensor matrix. - Determining at least one reflected image by using a sensor element that includes multiple reflection features, wherein each reflection feature includes a beam profile. - Evaluating sensor signals using at least one evaluation device, thereby determining the combined signal Q, and determining initial distance information of reflection features by analyzing the beam profiles of each reflection feature, wherein the beam profile analysis includes evaluating the combined signal Q from each sensor signal. The decision will be made by, viii) Considering the initial distance information, match the reflection features to the reference features of the reference image, thereby determining the matched pair of reflection features and reference features. ix) For each matched pair of reflection features and reference features, determine the epipolar line of the matched reference feature in the reference image, x) Determining the epipolar line distance d of the reflection feature that matches the epipolar line, xi) Evaluate the epipolar line distance d as a function of the image position (x,y) in the reference image, and thereby determine the geometric pattern. xii) Determine at least one correction for rotation and / or translation of the reflected image according to the geometric pattern, Methods that include...

[0122] Embodiment 14 The method of the above embodiment, comprising correcting the reflected image based on the determined correction and determining at least one triangulation distance information of the reflected features by using triangulation taking the determined correction into account.

[0123] Embodiment 15 A method of using a detector described in any one of the embodiments described above, with reference to a detector for an intended use, selected from the group consisting of location measurement in traffic technology, entertainment applications, security applications, surveillance applications, safety applications, human-machine interface applications, logistics applications, tracking applications, outdoor applications, mobile applications, communication applications, photography applications, machine vision applications, robotics applications, quality control applications, and manufacturing applications.

[0124] Further optional details and features of the present invention are evident from the description of preferred exemplary embodiments below, together with the dependent claims. In this context, certain features may be implemented in separate forms or in combination with other features. The present invention is not limited to exemplary embodiments. Exemplary embodiments are schematically shown in the drawings. The same reference numerals in the individual drawings refer to the same element or element having the same function, or elements corresponding to each other with respect to their functions. [Brief explanation of the drawing]

[0125] [Figure 1] An embodiment of the detector according to the present invention is shown. [Figure 2] This shows the reflection pattern of a calibrated detector that matches the reference pattern. [Figure 3A] An embodiment of the reflection pattern of a decalibrated (rotated) detector that matches a reference pattern is shown. [Figure 3B] An embodiment of the reflection pattern of a decalibrated (rotated) detector that matches a reference pattern is shown. [Figure 4] Further embodiments of the reflection pattern, the matched reference pattern, and the evaluated epipolar distance function d(x,y) are shown. [Figure 5] An exemplary flowchart of an embodiment of a method for calibrating at least one detector according to the present invention is shown. [Modes for carrying out the invention]

[0126] Figure 1 shows in a highly schematic form an embodiment of a detector 110 for determining the position of at least one object 112 according to the present invention. The detector 110 includes at least one sensor element 114 having a matrix 116 of optical sensors 118. Each optical sensor 118 has a photosensitive area 120.

[0127] The sensor element 114 can be formed unitarily, as a single device, or as a combination of several devices. The matrix 116 may be, or may include, a rectangular matrix having one or more rows and one or more columns. The rows and columns may be arranged in a rectangular form in particular. However, other arrangements, such as non-rectangular arrangements, are also possible. For example, a circular arrangement is also possible, in which the elements are arranged in concentric circles or ellipses around a central point. For example, the matrix 116 may be a single row of pixels. Other arrangements are possible.

[0128] The optical sensors 118 of the matrix 116 may be identical in size, sensitivity, and one or more of the other optical, electrical, and mechanical properties. The photosensitive areas 120 of all the optical sensors 118 of the matrix 116 may be located in a common plane, which preferably faces the object 112, so that the light beam propagating from the object to the detector 110 can generate a light spot on the common plane. The photosensitive areas 120 may be located on the surface of each optical sensor 118. However, other embodiments are possible.

[0129] The optical sensor 118 may include, for example, at least one CCD device and / or CMOS device. For example, the optical sensor 118 may be part of or constitute a pixelated optical device. For example, the optical sensor may be part of or constitute at least one CCD device and / or CMOS device having a matrix of pixels, each pixel forming a photosensitive area 120. Preferably, the detector is configured so that the optical sensor 118 is simultaneously exposed within a period of time in which it is represented as a frame or imaging frame. For example, the optical sensor 118 may be part of or constitute at least one global shutter CMOS.

[0130] The optical sensor 118 may be, or may include, a photodetector, preferably an inorganic photodetector, more preferably an inorganic semiconductor photodetector, and most preferably a silicon photodetector. In particular, the optical sensor 118 may be highly sensitive in the infrared spectral range. All or at least a group of optical sensors 118 in matrix 116 may be identical. A group of identical optical sensors 118 in matrix 116 may be provided for different spectral ranges, or all optical sensors may be identical with respect to spectral sensitivity. Furthermore, the optical sensors 118 may be identical in size and / or with respect to their electronic or optical-electronic properties. Matrix 116 may consist of independent optical sensors 118. Thus, matrix 116 may consist of inorganic photodiodes. Alternatively, however, commercially available matrices may be used, such as one or more CCD detectors, such as a CCD detector chip, and / or CMOS detectors, such as a CMOS detector chip.

[0131] The optical sensor 118 may form or be part of a sensor array, such as the matrix mentioned above. For example, the detector 110 may include an array of optical sensors 118, such as a rectangular array having m rows and n columns, where m and n are independently positive integers. Preferably, there are more than one column and more than one row, i.e., n>1, m>1. For example, n may be 2 to 16 or more, or m may be 2 to 16 or more. Preferably, the ratio of the number of rows to the number of columns is close to 1. For example, n and m may be selected such that 0.3 ≤ m / n ≤ 3, such as by choosing m / n = 1:1, 4:3, 16:9, or similar. For example, the array may be a square array having an equal number of rows and columns, such as by choosing m=2, n=2 or m=3, n=3, or similar.

[0132] The matrix 116 may be a rectangular matrix having at least one row, preferably more rows, and more columns. For example, the rows and columns may be oriented essentially vertically. To provide a wide view, the matrix 116 may have at least 10 rows, preferably at least 50 rows, and more preferably at least 100 rows. Similarly, the matrix may have at least 10 columns, preferably at least 50 columns, and more preferably at least 100 columns. The matrix 116 may include at least 50 optical sensors 118, preferably at least 100 optical sensors 118, and more preferably at least 500 optical sensors 118. The matrix 116 may include several pixels within a multi-megapixel range. However, other embodiments are feasible.

[0133] The detector 110 may further include a projector 122 for illuminating an object 112 with at least one illumination pattern 124. The projector 122 may, in particular, include at least one laser source 126 for generating at least one light beam. The projector 122 may, in particular, include at least one diffractive optical element 128 for generating and / or forming the illumination pattern 124 from the light beam of the laser source 126. The projector 122 may be configured so that the illumination pattern 124 propagates from the projector 122, in particular from at least one aperture 130 of the housing of the projector 122 toward the object 112. The projector 122 may be configured to generate and / or project a point cloud, for example, the projector 122 may include at least one digital light processing (DLP) projector, at least one LCoS projector, at least one laser source, at least one array of laser sources; at least one light-emitting diode; at least one array of light-emitting diodes. The laser source 126 may include a focusing optical system 134. The projector 122 may include multiple laser sources 126. In addition, additional irradiation patterns may be generated by at least one ambient light source.

[0134] The projector 122 may include at least one control unit 136. The control unit 136 may be configured to control the laser source 126. The control unit 136 may include at least one processing device, in particular at least one processor and / or at least one application-specific integrated circuit (ASIC). The control unit 136 may include one or more programmable devices, such as one or more computers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), or field-programmable gate arrays (FPGAs), configured to perform control of the laser source 126. The control unit 136 may include at least one processing device having software code stored therein, including some computer commands. The control unit 136 may provide one or more hardware elements for performing control of the laser source 126 and / or provide one or more processors having software running therein for performing control of the laser source. The control unit 136 may be configured to emit and / or generate at least one electronic signal for controlling the laser source. The control unit 136 may have one or more wireless and / or wired interfaces, and / or other types of control connections, for controlling the laser source 126. The control unit 136 and the laser source may be interconnected by one or more connectors and / or one or more interfaces.

[0135] The irradiation pattern 124 includes a plurality of irradiation features 125. The irradiation pattern 124 may include at least one periodic normal pattern selected from a group consisting of at least one periodic normal point pattern; at least one hexagon pattern; and at least one rectangular pattern.

[0136] For example, the projector 122 in Figure 1 may include a single light source, in particular a single laser source 126, configured to generate at least one optical beam, also referred to as a laser beam. The projector 122 may include at least one transfer device, in particular a DOE 128, for diffracting and replicating the laser beam generated by the single laser source for generating an illumination pattern 124 including patterned illumination features. The diffractive optical element 128 may be configured for beamforming and / or beam splitting.

[0137] For example, the projector 122 may include at least one array of densely packed light sources, in particular laser sources 126, according to a configured pattern for generating clusters of light beams. The density of the laser sources 126 may depend on the elongation of the housings of the individual light sources and the identifiability of the light beams. The projector 122 may include at least one transfer device, in particular DOE 128, for diffracting and replicating clusters of light beams to generate an illumination pattern 124 containing patterned illumination features.

[0138] Each optical sensor 118 is designed to generate at least one sensor signal in response to the illumination of its respective photosensitive area 120 by a reflected light beam propagating from the object 112 to the detector 110. Furthermore, the sensor element 114 is configured to determine at least one reflected image 142, which includes at least one reflected pattern 138. The reflected image 142 may include points as reflected features. These points result from the reflected light beam emanating from the object 112. The sensor element 114 may also be configured to determine a reflected pattern 138. The reflected pattern 138 may include at least one feature corresponding to at least one illumination feature 125 of the illumination pattern 124. The reflected pattern 138 may include at least one distortion pattern compared to the illumination pattern 124, the distortion depending on the distance to the object 112, such as the surface properties of the object 112.

[0139] The detector 110 may include at least one transfer device 140, which may include at least one lens, for example, selected from a group consisting of at least one focusable lens, at least one aspherical lens, at least one spherical lens, and at least one Fresnel lens; at least one diffractive optical element; at least one concave mirror; at least one beam deflection element, preferably at least one mirror; at least one beam splitting element, preferably at least one beam splitting cube or beam splitting mirror; and at least one multi-lens system. In particular, the transfer device 140 may include at least one collimating lens configured to focus at least one object point in the image plane.

[0140] The detector 110 includes at least one evaluation device 144. The evaluation device 144 may be configured to select at least one reflection feature of the reflection image 142. The evaluation device 144 may be configured to determine initial distance information, i.e., the y-coordinate of the selected feature of the reflection pattern, by selecting at least one feature of the reflection pattern 138 and evaluating the combined signal Q from the sensor signal as described above. Thus, the detector 110 may be configured to pre-classify at least one reflection feature of the reflection image 142.

[0141] The evaluation device 144 may be configured to perform at least one image analysis and / or image processing to identify reflection features. The image analysis and / or image processing may use at least one feature detection algorithm. The image analysis and / or image processing may include one or more of the following: filtering; selection of at least one region of interest; formation of a difference image between the image created by the sensor signal and at least one offset; inversion of the sensor signal by inverting the image created by the sensor signal; formation of a difference image between images created by the sensor signal at different times; background correction; decomposition into color channels; decomposition into tones; saturation; and brightness channels; frequency decomposition; singular value decomposition; application of a Canny edge detector; application of a Laplacian to a Gaussian filter; application of a Difference to a Gaussian filter; application of a Sobel operator; application of a Laplace operator; application of a Scharr operator; application of a Prewitt operator; application of a Roberts operator; application of a Kirsch operator; application of a high-pass filter; application of a low-pass filter; application of a Fourier transform; application of a Radon transform; application of a Hough transform; application of a wavelet transform; thresholding; and creation of a binary image. The area of ​​interest can be determined manually by the user, or it can be determined automatically by recognizing objects in the image generated by the optical sensor 118, for example.

[0142] The evaluation device 144 is configured to determine initial distance information, i.e., the vertex coordinate z of at least one selected reflection feature of the reflection image 142, by evaluating the combined signal Q from the sensor signals. The evaluation device 144 may be configured to derive the combined signal Q by one or more of the following: dividing the sensor signals, dividing the sensor signals into multiples, or dividing the linear combination of the sensor signals. The evaluation device 144 may be configured to use at least one predetermined relationship between the combined signal Q and the longitudinal region for determining the longitudinal region. For example, the evaluation device 144 may,

[0143]

number

[0144] Each sensor signal may contain information from at least one area of ​​the beam profile of the light beam. The photosensitive area 120 may be arranged such that the first sensor signal contains information from the first area of ​​the beam profile and the second sensor signal contains information from the second area of ​​the beam profile. The first area and the second area of ​​the beam profile may be adjacent or overlapping areas, or both. The first area and the second area of ​​the beam profile may not coincide in terms of area.

[0145] The evaluation device 144 may be configured to determine and / or select a first area of ​​the beam profile and a second area of ​​the beam profile. The first area of ​​the beam profile may essentially include edge information of the beam profile, and the second area of ​​the beam profile may essentially include center information of the beam profile. The beam profile may have a center, i.e., the geometric center of the maximum value and / or plateau of the beam profile and / or the optical spot, and a falling edge extending from the center. The second area may include an internal area of ​​the cross-section, and the first area may include an external area of ​​the cross-section. Preferably, the center information has a ratio of less than 10% edge information, more preferably less than 5% edge information, and most preferably, the center information does not include edge content. The edge information may include information about the entire beam profile, particularly from the center and edge areas. The edge information has a ratio of less than 10% center information, more preferably less than 5% center information, and more preferably, the edge information does not include center content. At least one area of ​​the beam profile may be determined and / or selected as a second area of ​​the beam profile if it is near or around the center and essentially contains center information. At least one area of ​​the beam profile may be determined and / or selected as a first area of ​​the beam profile if it includes at least a portion of the falling edge of the cross section. For example, the entire area of ​​the cross section may be determined as the first area. The first area of ​​the beam profile may be area A2, and the second area of ​​the beam profile may be area A1. Similarly, the center signal and edge signal may also be determined by using sections of the beam profile, such as circular sections of the beam profile. For example, the beam profile may be divided into two sections by a secant or code that does not pass through the center of the beam profile. Thus, one section essentially contains edge information, and the other section essentially contains center information. For example, to further reduce the amount of edge information in the center signal, the edge signal may be further subtracted from the center signal.

[0146] Edge information may include information relating to several photons in a first area of ​​the beam profile, and central information may include information relating to several photons in a second area of ​​the beam profile. The evaluation device 144 may be configured to determine the area integral of the beam profile. The evaluation device 144 may be configured to determine the edge information by integrating and / or summing the first area. The evaluation device 144 may be configured to determine the central information by integrating and / or summing the second area. For example, the beam profile may be a trapezoidal beam profile, and the evaluation device may be configured to determine the trapezoidal integral. Furthermore, when a trapezoidal beam profile is assumed, the determination of edge and central signals may be replaced by an equivalent evaluation that derives edge and central signals by geometric considerations, utilizing the characteristics of the trapezoidal beam profile, such as determining the slope and position of the edges and the height of the central plateau.

[0147] The evaluation device 144 may be configured to use at least one predetermined relationship between a combined signal and a y-coordinate. The predetermined relationship may be one or more of empirical relationships, semi-empirical relationships, and analytically derived relationships. The evaluation device 144 may include at least one data storage device for storing the predetermined relationships, such as a lookup list or a lookup table.

[0148] Depth measurement using beam profile analysis with a combined signal Q can enable reliable distance determination even in environments that produce multiple reflections by biasing the light source, or in the case of reflectance measurement objects by reducing computational requirements, particularly by reducing processing power. Beam profile analysis can enable estimation of a depth map from images of the sensor element 114. In particular, the distance determined by beam profile analysis can provide distance estimates for each illumination feature 125, which can be refined by triangulation methods with respect to known positions of the sensor element 114 and projector 122, especially fixed positions. To calculate the refined vertex coordinates using triangulation, the so-called correspondence problem needs to be solved. Generally, three-dimensional reconstruction methods using triangulation require an externally calibrated system. In the case of an externally calibrated system, each reflection feature can be matched to a reference grid point, i.e., a reference feature, along with the initial distance information estimated from beam profile analysis. Therefore, if the epipolar conditions are met and beam profile analysis yields a reliable depth estimate, each detected reflection feature can be matched to a corresponding reference grid point. However, if the correspondence is determined to be incorrect, distance measurements based on triangulation become very poor. Depending on the hardware, a pre-calibrated detector may degrade due to physical stress or temperature shifts, resulting in changes in relative position and rotation over time. Changes in the relative positions of the sensor element 114 and the projector 122 result in changes in the reflection image, as well as incorrectly determined correspondences between reference features and reflection features, thus leading to erroneous distance measurement results. The present invention proposes a calibration method, in particular, to perform recalibration, for determining the external parameters of the detector 110. The calibration method according to the present invention may allow for on-the-fly correction of erroneous distance measurements without initiating a new static calibration process. In particular, the calibration method may be performed automatically, such as without any user intervention.

[0149] Calibration may include at least one process for determining at least one external parameter of the detector 110 and / or determining a correction to the measured values ​​of the detector 110, in particular, the measured values ​​of the positions of reflective features in the reflected image. The evaluation device 144 may be configured to determine at least one external parameter of the detector 110. The external parameter may include at least one parameter selected from a group consisting of the rotation angle between the coordinates of the projector 122 and the sensor element 114, the translational component between the coordinates of the projector 122 and the sensor element 114, the aperture angle, the center of the sensor element 114, the aperture, and the focal length.

[0150] The calibration method includes matching the reflected features to the reference features of a reference image, taking into account the initial distance information, thereby determining the matched pairs of reflected and reference features. The evaluation device 144 may be configured to match each of the reflected features to each of the reference features in the displacement region by using at least one linear scaling algorithm. Beam profile analysis may allow for a reduction in the number of possibilities.

[0151] The evaluation device 144 may be configured to determine at least one reference feature in at least one reference image corresponding to at least one reflected feature. The evaluation device 144 may be configured to perform image analysis and identify features in the reflected image. The evaluation device 144 may be configured to identify at least one reference feature in the reference image having essentially the same y-coordinate as the selected reflected feature. The reference feature corresponding to the reflected feature may be determined using epipolar geometry. For an explanation of epipolar geometry, refer to, for example, Chapter 2: "Dreidimensionales Computersehen" by X. Jiang and H. Bunke, Springer, Berlin Heidelberg, 1997. Epipolar geometry can be assumed to mean that the reference image and the reflected image may be images of an object determined at different spatial positions and / or spatial orientations with a fixed distance between them. The reference image and the reflected image may be images of an object determined at different spatial positions with a fixed distance between them. The evaluation device 144 may be configured to determine epipolar lines in the reference image. The assumed relative positions of the reference image and the reflected image may be known. For example, the assumed relative position of the reference image may be determined in a prior calibration or hysteretic calibration, in which steps a) to e) are performed. For example, the assumed relative positions of the reference image and the reflected image may be manufacturer values. For example, the assumed relative positions of the reference image and the reflected image may be stored in at least one memory unit of the evaluation device. The evaluation device 144 may be configured to determine a straight line extending from a selected reflective feature of the reflected image. The straight line may include possible reflective features corresponding to the selected reflective feature. The straight line and baseline extend to the epipolar plane. The corresponding possible reflective features may be imaged on a straight line in the reference image, a so-called epipolar line, so that the reference image is determined at a different relative position from the reflected image. Thus, the reference feature of the reference image corresponding to the selected reflective feature of the reflected image is assumed to lie on the epipolar line.However, as outlined above, due to image distortion or changes in external parameters such as aging, temperature changes, and mechanical stress, epipolar lines may intersect with each other or become very close to each other, and / or the correspondence between reference features and reflected features may become unclear.

[0152] The evaluation device 144 may be configured to determine the longitudinal region of each reflection feature. The longitudinal region may be given by the initial distance information and error interval ±ε of the reflection feature determined from the combined signal Q. The evaluation device 144 may be configured to determine at least one displacement region in the reference image corresponding to the longitudinal region. The displacement region may be a region in the reference image where the reference feature corresponding to the selected reflection feature may be imaged. In particular, the displacement region may be a region in the reference image where the reference feature corresponding to the selected reflection feature is expected to be located. Depending on the distance to the object, the image position of the reference feature corresponding to the reflection feature may be displaced in the reference image compared to the image position of the reflection feature in the reflection image. The displacement region may contain only one reference feature. The displacement region may also contain more than one reference feature.

[0153] The displacement region may include an epipolar line or a section of an epipolar line. The displacement region may include more than one epipolar line or more sections of more than one epipolar line. The displacement region may extend along an epipolar line, be perpendicular to an epipolar line, or both. The evaluation device 144 may be configured to determine a reference feature along an epipolar line corresponding to initial distance information and to determine the extent of a displacement region along an epipolar line or a displacement region perpendicular to an epipolar line corresponding to an error interval ±ε. The measurement uncertainty of distance measurements using the coupled signal Q may result in a non-circular displacement region because the measurement uncertainty may differ in different directions. In particular, the measurement uncertainty along an epipolar line or epipolar line(s) may be greater than the measurement uncertainty in directions perpendicular to the epipolar line or epipolar line(s). The displacement region may include the extent in directions perpendicular to the epipolar line or epipolar line(s). The evaluation device may be configured to match a selected reflection feature to at least one reference feature within the displacement region. The evaluation device 144 may be configured to match a selected feature of the reflection image to a reference feature within the displacement region by using at least one evaluation algorithm, taking into account the determined initial distance information. The evaluation algorithm may be a linear scaling algorithm. The evaluation device 144 may be configured to determine the epipolar line closest to and / or within the displacement region. The evaluation device may be configured to determine the epipolar line closest to the image position of the reflection feature. The extent of the displacement region along the epipolar line may be greater than the extent of the displacement region orthogonal to the epipolar line. The evaluation device 144 may be configured to determine the epipolar line before determining the corresponding reference feature. The evaluation device 144 may determine the displacement region around the image position of each reflection feature. The evaluation device 144 may be configured to assign epipolar lines to each displacement region of each image position of a reflection feature, for example, by assigning the epipolar line closest to the displacement region and / or within the displacement region and / or along a direction perpendicular to the epipolar line.The evaluation device 144 may be configured to determine a reference feature corresponding to the image position of a reflection feature by determining a reference feature in the assigned displacement region that is closest to and / or within the assigned displacement region and / or along the assigned epipolar line.

[0154] In addition, or instead, the evaluation device 144 may be configured to perform the following steps: - Determine the displacement region of each reflection feature relative to the image position. - Assigning epipolar lines to the displacement region of each reflection feature by assigning the epipolar line closest to the displacement region and / or within the displacement region and / or along the direction perpendicular to the epipolar line. - Assigning at least one reference feature to each reflection feature and / or determining at least one reference feature to each reflection feature by assigning a reference feature in the assigned displacement region that is closest to and / or within the assigned displacement region and / or along the assigned epipolar line.

[0155] In addition, or instead, the evaluation device 144 may be configured to compare the distances of epipolar lines in the reflective features and / or reference image, and / or compare error weighting distances, such as the ε-weighted distances of the epipolar lines in the reflective features and / or reference image, and to determine between more than one epipolar line and / or reference feature that would be assigned to the reflective features, by assigning the epipolar line and / or reference feature within a shorter distance and / or ε-weighted distance to the reference feature and / or reflective feature.

[0156] As outlined above, the detector 110, in particular the evaluation device 144, may be configured to pre-classify selected reflection features using a coupled signal Q, thereby enabling ambiguous assignment to a single reference feature. In particular, the illumination features of the illumination pattern may be arranged such that the corresponding reference features of the reference image can have as long a relative distance from each other as possible on the epipolar line. The illumination features of the illumination pattern may be arranged such that only a small number of reference features are located on the epipolar line.

[0157] Using beamprofile analysis can enable the estimation of initial distance information, such as vertex coordinates within error intervals. By determining the displacement regions corresponding to this initial distance information, the corresponding error intervals can reduce the number of possible solutions along the epipolar line required to significantly match the reference and reflection features. The number of possible solutions can even be reduced to one. The determination of initial distance information can be performed during pre-evaluation before matching the reflection and reference features. This can reduce computational requirements, resulting in a significant cost reduction and enabling use in mobile or outdoor devices.

[0158] The calibration method includes determining the epipolar line of the matched reference feature in the reference image for each matched pair of reflective and reference features. In particular, the epipolar line used to match each matched reference and reflective feature may be used as the epipolar line of the aforementioned pair of matched reflective and reference features.

[0159] The calibration method further includes determining the epipolar line distance d of the matched reflection feature to the epipolar line described above. The epipolar line distance may be the distance to the reflection feature and the distance of the epipolar line used to match the matched reference feature, which is represented as the corresponding epipolar line. The distance may be determined by determining the image coordinates of the reflection image and the image coordinates of the corresponding epipolar line and comparing the image coordinates. The minimum distance to the corresponding epipolar line may be used as the epipolar line distance.

[0160] In the case of good external calibration, the epipolar line distance is close to zero. A reflection pattern 138 (circle) that matches the reference pattern 146 (square) of the calibrated detector 110 is shown in Figure 2.

[0161] In cases where the detector is decalibrated, the reflection features may be likely to match the reference features. Figures 3A and 3B show two embodiments of the reflection pattern 138 (circular) of a decalibrated detector 110 in the case of a rotated sensor element 114 and / or projector 122 matching the reference pattern 146 (square). However, it may always be possible to determine an epipolar line distance to an incorrect or true corresponding reference feature. In the matching process, the reconstruction algorithm used may match the reflection features to the reference features if the epipolar line distance is within an acceptable range. Thus, it is possible that the reflection features may match an incorrect reference feature, resulting in a non-zero epipolar line distance. If an incorrect pair of matched reflection and reference features is determined, the evaluation device 144 may be configured to perform steps b) to e). The calibration method may be based on evaluating the resulting epipolar line distance. The calibration method may consider the epipolar distance independently of whether the correspondence between the reflected feature and the reference feature is accurate or inaccurate. The epipolar distance may be appropriate for the proposed calibration strategy even when the reflected feature matches an incorrect reference feature.

[0162] Step d) includes evaluating the epipolar line distance d as a function of the image position (x,y) in the reference image, thereby determining the geometric pattern. The geometric pattern may be a distribution of the epipolar line distance. The epipolar line distance can be defined as a function d(x,y) on the position (x,y) in the reference image. The epipolar line distance function d can be analyzed to calculate corrections for rotation and / or translational movement. In the case of a decalibrated system, function d may generate a geometric pattern. The shape of this geometric pattern of the epipolar line distance function may uniquely indicate the degree of decalibration. Geometric patterns such as repetition, steepness, discontinuity, and curvature in function d(x,y) can be used to restore calibration. If the rotation and / or translational movement of the projector and / or sensor elements changes, this result can be observed in function d as a geometric pattern. The evaluation device 144 may be configured to run an algorithm designed to analyze d(x,y) and calculate corrections for rotation and / or translational movement. The evaluation device 144 may be configured to determine the correction of the reflected image by evaluating one or more of the shape, repeatability, steepness, discontinuity, and curvature of the geometric pattern. Figure 4 shows further embodiments of the reflected pattern and the matched reference pattern on the left, and further embodiments of the evaluated epipolar linear distance function d(x,y) on the right.

[0163] The evaluation device 144 may be configured to correct the reflected image based on the determined correction. As outlined above, the reflected image refers to the evaluation of the optical sensor image with respect to at least one feature and / or the transformation of external parameters such as rotation and translation. The correction for rotation and / or translation of the reflected image may be at least one correction factor applied to the image position of the reflected image. The evaluation device 144 may be configured to determine whether the geometric pattern matches within a predefined tolerance to the geometric pattern of the calibrated detector, or whether the geometric pattern deviates from the calibration geometric pattern by a larger than predefined tolerance. The determined correction may provide information about the degree of decalibration. If the detector is already optimally calibrated, the correction step may be very small, and the correction may have a very small effect. In the case of a match within tolerance, the evaluation device 144 may maintain the initial external parameters and / or discard the correction of the reflected image. Otherwise, the correction may be applied to the reflected image. The determined correction may be used to correct rotation and / or translation. Rotation and translation describe the spatial context between the sensor element and the projector. Rotation and translation may include a relationship between "reflection features" and "reference features." From this relationship, triangulation distance information can be calculated using triangulation techniques. Therefore, in cases of determined changes in rotation and / or translation, the resulting triangulation information can be corrected.

[0164] The evaluation device 144 may be configured to determine at least one triangulation distance information for a reflected feature by using triangulation, taking into account the determined corrections. The evaluation device 144 may be configured to determine the displacements of the matched reference feature and the reflected feature. The displacement may be the difference between the position in the reference image and the position in the reflected image. The evaluation device may be configured to determine the triangulation distance of the matched reference feature using a predetermined relationship between the vertex coordinate and the displacement. The evaluation device 144 is configured to perform a calibration method on the fly while determining the triangulation distance information.

[0165] Figure 5 shows an exemplary flowchart of an embodiment of a method for calibrating at least one detector according to the present invention. The method consists of the following steps: i) Initial distance information (represented by reference numeral 148) - An object is illuminated by at least one illumination pattern generated by at least one projector of the detector, the illumination pattern includes multiple illumination features, - In response to irradiation, at least one sensor signal is generated for each reflected light beam that collides on the photosensitive area of ​​the optical sensor of a sensor element having an optical sensor matrix, - By using a sensor element that includes multiple reflection features, at least one reflection image is determined, and each of the reflection features includes a beam profile. - By using at least one evaluation device, the sensor signal is evaluated, thereby determining the combined signal Q; initial distance information of the reflection features is determined by analysis of the beam profiles of each reflection feature, the beam profile analysis of which includes evaluating the combined signal Q from each sensor signal. To be determined by, ii) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information (represented by reference numeral 150), thereby determining the matched pair of reflection features and reference features, iii) For each matched reflective feature and reference feature pair (represented by reference numeral 152), determine the epipolar line of the matched reference feature in the reference image, iv) Determining the epipolar line distance d of the reflection feature that matches the epipolar line (represented by reference numeral 154), v) Evaluate the epipolar line distance d as a function of the image position (x,y) in the reference image (represented by reference numeral 156), and thereby determine the geometric pattern. vi) Determine at least one correction for rotation and / or translation of the reflected image according to the geometric pattern (represented by reference numeral 158), Includes. [Explanation of Symbols]

[0166] 110 detectors 112 Object 114 Sensor elements 116 Matrix 118 Optical Sensors 120 Photosensitive Area 122 Projectors 124 irradiation patterns 125 Irradiation Characteristics 126 Laser Sources 128 DOE 130 opening 132 cabinets 134 Optical system 136 Control Unit 138 Reflection Patterns 140 Transfer Devices 142 Reflected image 144 evaluation devices 146 Reference Patterns 148 Determining initial distance information 150 matches 152 Determination of the epipolar line 154 Determination of Epipolar Line Distance 156 Evaluation of Epipolar Line Distance 158. Determine at least one correction.

Claims

1. A detector (110) for determining the position of at least one object (112), - At least one projector (122) for illuminating the object (112) with at least one irradiation pattern (124), wherein the irradiation pattern (124) includes a plurality of irradiation features (125), - At least one sensor element (114) having a matrix (116) of optical sensors (118), each of the optical sensors having a photosensitive area (120), each optical sensor (118) is designed to generate at least one sensor signal in response to illumination of each photosensitive area (120) by a reflected light beam propagating from the object (112) to the detector (110), the sensor element is configured to determine at least one reflection image (142) including a plurality of reflection features, each of the reflection features including a beam profile, at least one sensor element (114) and - At least one evaluation device (144) configured to determine the initial distance information of the reflection features by analyzing the beam profiles of each of the reflection features, wherein the analysis of the beam profiles includes evaluating the coupled signal Q from the sensor signal, and the evaluation device (144) a) (150) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features, b) (152) For each of the matching reflective feature and reference feature pairs, determine the epipolar line of the matching reference feature in the reference image, c) (154) Determining the epipolar line distance d of the matched reflection feature to the epipolar line, d) (156) Evaluate the epipolar line distance d as a function of the image position (x, y) in the reference image, and thereby determine the geometric pattern, e) (158) Determine at least one correction for the rotation and / or translation of the reflected image (142) according to the geometric pattern. An evaluation device (144) configured to perform a calibration method including, A detector (110) is provided.

2. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to correct the reflected image (142) based on the determined correction.

3. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine at least one triangulation distance information of the reflection feature by using triangulation with respect to the determined correction.

4. The detector (110) according to claim 3, wherein the evaluation device (144) is configured to perform the calibration method on the fly while determining the triangulation distance information.

5. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine at least one external parameter of the detector, the external parameter being at least one parameter selected from the group consisting of a rotation angle between the coordinates of the projector (122) and the sensor element (114), a translation component between the coordinates of the projector (122) and the sensor element (114), an aperture angle, the center of the sensor element, an aperture, and a focal length.

6. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to perform steps b) to e) even for pairs of reflection features and reference features that were incorrectly determined as matching pairs of reflection features and reference features in step a).

7. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine the correction of the reflected image by evaluating one or more of the shape, repeatability, steepness, discontinuity, and curvature of the geometric pattern.

8. The detector (110) according to claim 1, wherein the irradiation pattern includes at least one periodic and regular pattern selected from a group consisting of at least one periodic and regular point pattern, at least one hexagonal pattern, and at least one rectangular pattern.

9. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to derive the combined signal Q by one or more of the following: dividing the sensor signal, dividing by a multiple of the sensor signal, and dividing by a linear combination of the sensor signals, and the evaluation device (144) is configured to use at least one predetermined relationship between the combined signal Q and a y-coordinate to determine the initial distance information.

10. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to perform image analysis of the reflected image (142) and thereby identify the reflection characteristics of the reflected image (142).

11. The detector (110) according to claim 1, wherein the evaluation device (144) is configured to determine a longitudinal region of each reflection feature, the longitudinal region being given by the initial distance information and error interval ±ε of the reflection feature determined from the combined signal Q, and the evaluation device (144) is configured to determine at least one displacement region in the reference image corresponding to the longitudinal region.

12. The detector (110) according to claim 11, wherein the evaluation device (144) is configured to match each of the reflection features with each of the reference features in the displacement region by using at least one linear scaling algorithm.

13. A method for calibrating at least one detector (110) according to any one of claims 1 to 12, the method comprising the following steps: i) (148) Initial distance information, - Irradiating an object (112) with at least one irradiation pattern (124) generated by at least one projector (122) of the detector (110), wherein the irradiation pattern (124) includes a plurality of irradiation features (125), - To generate at least one sensor signal for each reflected light beam that collides with the photosensitive area (120) of the optical sensor (118) of the sensor element (114) having a matrix (116) of the optical sensor (118) in response to irradiation. - To determine at least one reflection image (142) including a plurality of reflection features by using the sensor element (114), wherein each of the reflection features includes a beam profile. - Evaluating the sensor signal using at least one evaluation device (144) to determine the combined signal Q, and determining the initial distance information of the reflection features by analyzing the beam profiles of each of the reflection features, wherein the analysis of the beam profiles includes evaluating the combined signal Q from the sensor signal. The decision will be made by, ii) (150) Matching the reflection features to the reference features of the reference image, taking into account the initial distance information, thereby determining the matched pair of reflection features and reference features, iii) (152) For each of the matching pairs of reflection features and reference features, determine the epipolar line of the matching reference feature in the reference image, iv) (154) Determining the epipolar line distance d of the matched reflection feature to the epipolar line, v) (156) Evaluate the epipolar line distance d as a function of the image position (x, y) in the reference image, and thereby determine the geometric pattern, vi) (158) Determine at least one correction for the rotation and / or translation of the reflected image (142) according to the geometric pattern, Methods that include...

14. The method according to claim 13, comprising correcting the reflected image (142) based on the determined correction, and determining at least one triangulation distance information of the reflected feature by using triangulation taking the determined correction into consideration.

15. A method of using a detector (110) according to any one of claims 1 to 12 for an intended use selected from the group consisting of location measurement in traffic technology, entertainment applications, security applications, surveillance applications, safety applications, human-machine interface applications, logistics applications, tracking applications, outdoor applications, mobile applications, communication applications, photography applications, machine vision applications, robotics applications, quality control applications, and manufacturing applications.

Images