Optoelectronic sensor for detecting objects in a monitored area

Abstract

The invention relates to an optoelectronic sensor, in particular a time-of-flight camera or a LiDAR sensor, for detecting at least one object in a monitoring area, wherein the optoelectronic sensor comprises a light transmitter, a light receiver, and an evaluation unit. The light transmitter is configured to emit transmitted light into the monitoring area. The light receiver is configured to receive received light reflected from the monitoring area. The evaluation unit is configured to obtain distance data about the monitoring area based on the received light, wherein the distance data comprises intensity values ​​and associated distance values. For all those data components in the distance data that do not originate from the object, and in particular for all those data components whose intensity value is less than a first intensity threshold, the distance to the object is calculated based on the distance data.in particular to determine the distance of the data component to the object, and, based on the determined distance of the data component in question to the object, a model for the scattered light behavior of the optoelectronic sensor, and the intensity value of the data component in question, to determine a probability value that the data component in question represents a disturbance, and to identify all those data components as disturbances whose determined probability value is equal to or greater than a probability threshold.

Description

[0001] The invention relates to an optoelectronic sensor, in particular a time-of-flight camera or a LiDAR sensor, for detecting at least one object in a monitoring area, wherein the optoelectronic sensor comprises means for recognizing data components in acquired distance data over the monitoring area as disturbances.

[0002] Optoelectronic sensors can be used for industrial safety applications and enable safe environmental perception of a monitored area, particularly safe three-dimensional environmental perception, thereby increasing the safety and efficiency of industrial processes in industrial plants. Examples of such optoelectronic sensors include Time-of-Flight (ToF) cameras and LiDAR (Light Detection and Ranging) sensors. Optoelectronic sensors can be, for example, stationary within the industrial plant or mounted on robots that can move autonomously within the industrial plant. An industrial plant could be, for example, a production hall, a warehouse, a power plant, a chemical plant, a food processing plant, or an animal husbandry facility.The industrial plant may contain reflectors, especially retroreflectors, which can serve as obstacle marking / highlighting and / or navigation features and can be used by optoelectronic sensors on autonomous mobile robots (e.g. AGVs - Autonomous Guided Vehicles) for controlling, localizing and / or navigating the robots.

[0003] If highly remitting (especially reflective) objects are present in the monitoring area, such as reflectors and especially retroreflectors, safety vests, metallic or mirrored objects, the received light in typical optoelectronic sensor receiving lenses can be partially scattered by lens edges and / or other optical elements. This can occur particularly with optoelectronic sensors that scan an environment simultaneously and therefore not in individual measurements. The scattered light, especially against a weakly remitting or distant background, can appear as image distortions around the object and lead to inaccurate distance measurements. These image distortions can unintentionally trigger a warning or protection field configured in the optoelectronic sensor, thus unnecessarily causing a safety stop of an autonomous robot.This reduces the availability of the robots for their intended use and / or can even render the optoelectronic sensor completely unusable for use in the industrial plant, as it repeatedly triggers a safety stop of the robots at the same positions in the industrial plant (i.e. near the retroreflectors).

[0004] Known optoelectronic sensors attempt to manage or avoid image distortion by using receiving lenses with fewer internal reflections. However, such receiving lenses are complex, expensive, difficult to manufacture, and / or may still only partially prevent image distortion. Other known optoelectronic sensors reduce the intensity of the transmitted light. However, reducing the transmitted light usually results in a reduction in range, a reduction in the field of view, detection losses, and / or loss of accuracy.

[0005] DE 10 2020 215 312 A1 describes a method for the supplementary detection of objects by a LiDAR system.

[0006] DE 10 2018 132 473 B4 describes an optoelectronic sensor for detecting an object in a monitored area.

[0007] The invention is based on the objective of providing an improved optoelectronic sensor, particularly with regard to the detection of image distortions.

[0008] To solve the problem, an optoelectronic sensor with the features of claim 1 is provided.

[0009] The optoelectronic sensor according to the invention, in particular a time-of-flight camera or a LiDAR sensor, for detecting at least one object in a monitoring area, especially for vehicle navigation, comprises a light transmitter, a light receiver, and an evaluation unit. The light transmitter is configured to emit transmitted light into the monitoring area. The light receiver is configured to receive received light reflected from the monitoring area. The evaluation unit is configured to obtain distance data over the monitoring area based on the received light, wherein the distance data includes intensity values ​​and associated distance values. The evaluation unit is further configured to perform the following functions for all those data components (e.g.,The evaluation unit is designed to determine, based on the distance data, a distance to the object, particularly to the object's edge, for each of the following data components: image points or pixels in the distance data that do not originate from the object, and, in particular, for all those data components in the distance data whose (obtained) intensity value is less than a (predetermined) first intensity threshold. Based on the determined distance of the respective data component to the object, a probability value is then determined that the data component in question represents a disturbance. Furthermore, the evaluation unit is designed to recognize as disturbances all those data components whose determined probability value is equal to or greater than a (predetermined) probability threshold.

[0010] In other words, the invention is based on the understanding that for all data components in the distance data that do not originate from a (strongly) reflecting object (such as a reflector, and in particular a retroreflector), a probability value can be determined, based on their distance to the object, especially to the edge of the object, their intensity value, and a calibrated or simulated scattered light model, indicating that the respective data component represents a disturbance. Based on these determined probability values, which, in the overall context for all data components, can also be referred to as a probability mask or map, those data components in the distance data that represent a disturbance with a certain probability can be identified and filtered out.

[0011] The data components in the distance data that do not originate from the object can be detected using common image segmentation algorithms, and especially machine learning algorithms. In particular, it can be exploited that the disturbances caused by scattered light usually have a lower intensity than the emitting object itself. Additionally or alternatively, the data components in the distance data that do not originate from the object can therefore be detected (and segmented) by filtering using a (first) intensity threshold. For all data components in the distance data whose intensity value falls below this (first) intensity threshold, and thus do not originate from the object, a probability value can be determined (individually) based on the scattered light model, indicating that the respective data component represents a disturbance. Based on these determined probability values, the data components that represent a disturbance can be filtered out.

[0012] When filtering distance data, a challenge arises: it is often difficult to determine whether the measurements are valid (and, for example, originate from a dark scene) or whether they actually represent an interference. This is particularly relevant for safety-related applications, as functional safety requires retaining as much measurement data as possible that cannot be reliably identified as a measurement error or interference. Conventional filters often become less efficient the closer a pixel in the distance data is to a highly reflecting object. Furthermore, the interference can be more pronounced the larger the highly reflecting object is within the monitored area.The optoelectronic sensor according to the invention with the filtering based on the probability limit can be more efficient and / or reliable with regard to the detection of image disturbances, especially in the presence of a highly remitting object in the monitoring area.

[0013] The model for the stray light behavior of the optoelectronic sensor can be calibrated, created, or derived using real measurement data. The stray light behavior of the optoelectronic sensor may be known in conjunction with the object in the monitoring area (e.g., a retroreflector). In other words, it may be specified that only certain highly remitting objects, particularly retroreflectors, are present in the monitoring area. By recording distance data across the monitoring area with and without the object, a model for the stray light behavior can then be calibrated, created, or derived. Additionally or alternatively, the model can be calibrated, created, or derived using a simulation. It is also conceivable that the model could be calibrated, created, or derived using a machine learning algorithm.Using the calibrated, created, or derived model for the scattered light behavior, an expected intensity deviation per data segment, e.g., per pixel, can be calculated, depending on the object's position and size in the distance data. Once the model for the scattered light behavior of the optoelectronic sensor has been calibrated, created, or derived for one specific optoelectronic sensor, it can also be applied to other optoelectronic sensors with identical or similar receiving lenses. The calibration process can be automated using equipment that repositions the object within the monitored area.

[0014] The distance value of a data component within the distance data can correspond to a distance measured in meters between the monitored area (or an object within the monitored area) and the optoelectronic sensor. The distance of the data component to the object (or the object's edge) can correspond to a distance measured in pixels or image points (on the sensor) or be converted to a distance in meters.

[0015] The first intensity threshold can, for example, be an arbitrary unit, such as 20,000 AU or 20,000 digits. The data components in the object's distance data can then have intensity values ​​equal to or greater than this first intensity threshold. The first intensity threshold can also correspond to the maximum intensity measurable by the optoelectronic sensor. In this case, the intensity values ​​in the object's distance data can reach the value of the first intensity threshold and be described as "overdriven."

[0016] The first intensity limit can correspond to a percentage of the maximum intensity measurable with the optoelectronic sensor, and may, for example, correspond to 50%, preferably 60%, preferably 70%, preferably 80%, preferably 90%, preferably 95%, preferably 99% and even more preferably 100% (i.e. the saturation value) of the maximum measurable intensity.

[0017] The probability threshold can be, for example, 50% (or 0.5), preferably 60%, preferably 70%, preferably 80%, preferably 90%, and even more preferably 95%. The probability threshold can preferably be equal to or greater than 0.5 and less than or equal to 0.7.

[0018] It is understood that the evaluation unit can be part of the optoelectronic sensor or an external computing unit, e.g. a server, with which the optoelectronic sensor is in (wireless or wired) signal communication.

[0019] The optoelectronic sensor is preferably a safety sensor, specifically a safety ToF camera or a safety LiDAR sensor. The terms "safe" and "safety" can be understood in the sense of safety-specific standards such as DIN EN ISO 13849 or DIN EN 61508. The optoelectronic sensor can therefore enable the control of errors up to a certain safety level.

[0020] The transmitted light is re-emitted by the object (and also by other objects) in the monitored area as received light.

[0021] According to one embodiment, the object comprises a reflector (e.g., a retroreflector). In particular, the object can be a reflector, and preferably a retroreflector. The retroreflector can preferably reflect the transmitted light back towards the light source regardless of the angle of incidence. The retroreflector can comprise a plurality of small angled or hemispherical mirrors that can reflect the light back towards the light source with a small dispersion of preferably a few degrees. The reflector can have a reflectance of 80% or greater, preferably 85% or greater, preferably 90% or greater, preferably 95% or greater, and preferably 99% or greater. The intensity values ​​of the reflector in the distance data can (regardless of the distance of the reflector to the optoelectronic sensor) exhibit the maximum measurable intensity value, i.e., they are (always) overdriven.The reflector, in particular a retroreflector, can be installed in the industrial plant for the purpose of controlling, localizing, and / or navigating autonomous robots. It can be specified that only certain reflectors, in particular retroreflectors, are located within the monitored area.

[0022] The evaluation unit can be configured to determine remission values ​​based on distance data. These remission values ​​can be estimated or calculated from the acquired intensity and distance values. For example, the remission value can be proportional to the acquired intensity value multiplied by the acquired distance value (in meters) squared (remission value ∼ intensity value * distance value * distance value). In general, remission can estimate the "reflectivity of the considered surface" based on the measured intensity of an object and the measured distance. In other words, it can provide an estimate of the material properties of the considered surfaces (or objects) as detected by the optoelectronic sensor. Additionally, the remission values ​​can be normalized by the intensity of the transmitted light.

[0023] According to one embodiment, the distance data comprises a plurality of pixels, each with an intensity value and an associated distance value.

[0024] According to the invention, the evaluation unit is configured to determine the distance, size, remission, and / or intensity of the remitting object based on the distance data. The distance of the object can refer here to the distance between the object and the optoelectronic sensor.

[0025] According to one embodiment, the probability value P BP , that the relevant data segment represents a disturbance, is determined using the following equation 1, PBP~F(xBP)IBP, where I BP the measured intensity value of the relevant data component, x BP the distance or distance (for example, measured in pixels) of the relevant data portion to the object, in particular to the edge of the object, and F(x BP) based on the model F for the scattered light behavior as a function of x BP The determined scattered light value is... In other words, I... BP for example, the sum of a basic intensity value I originating from the monitoring area without the object NoObj and an intensity value I originating from the object Obj correspond to where F(x BP ) the intensity value I Obj of the object. The probability value P BP It can then also depend on or the value 1 minus the ratio of I NoObj to I BP (also 1 - I NoObj / I BP ) correspond. Even if I NoObj Although it may not be necessarily known, it can nevertheless be known that the ratio of I NoObj to I BP changes with an object. For example, is I NoObj very similar to I BP or identical to it (i.e., there is no object in the monitored area), PBP against the value zero. Should the base intensity value I NoObj As P approaches zero, it goes BP against the value 1. Therefore, the closer the data component is to the object, the more likely it is to represent a disturbance. A higher base intensity value I NoObj This can lead to a lower probability that the relevant data segment represents a disturbance and / or that the object produces disturbances.

[0026] According to the invention, the evaluation unit is designed to determine the probability value that the relevant data component represents a disturbance, depending on the determined distance of the relevant data component to the object (in particular to the edge of the object), the intensity value of the relevant data component, the maximum measurable intensity (e.g., when the distance of the data component to the object is zero and / or when the distance of the object to the optoelectronic sensor is zero) and the size of the object.

[0027] According to one embodiment, the evaluation unit is designed to determine the size of the remitting object based on those data components in the distance data whose (obtained) intensity value is equal to or greater than the first intensity limit value.

[0028] According to one embodiment, the evaluation unit is designed to perform the detection of those data components in the distance data that represent a disturbance only if the determined size of the object is equal to or greater than a predetermined size limit, if the determined remission of the object is equal to or greater than a predetermined remission limit, and / or if the determined intensity of the object is equal to or greater than a second intensity limit.

[0029] The remission limit described herein can be measured and / or determined using test specimens that are based on a safety-specific standard. For example, the safety-specific standard may require that an object with 4% remission can still be reliably detected. The remission limit can therefore be set at 4%, as objects with lower remission can be ignored.

[0030] The second intensity threshold can be the same as the first, or higher or lower. The size of the object can be determined, for example, by the number of pixels or, converted to an (actual) spatial extent. The size of the object (e.g., a retroreflector) may be known. Furthermore, the object may be known to be very bright, i.e., highly reflecting. Based on this, it can then be determined whether such an object (e.g., a retroreflector) with these specific properties is located within the monitored area. If so, interference detection, or the detection (and filtering) of those data components that constitute interference, is performed. If not, the evaluation unit can terminate the evaluation of the distance data at this point, skip interference detection, and / or initiate the output of a signal.This selective detection (and filtering) approach can be particularly important for security applications, as it is crucial to avoid accidentally filtering out valid measurement data (i.e., data components from real, existing objects). Furthermore, this method can increase the efficiency of the optoelectronic sensor. If several sufficiently large and sufficiently reflective (bright) objects (e.g., retroreflectors) are detected, the interference detection process described here can be performed iteratively for each of these objects.

[0031] According to one embodiment, the evaluation unit is configured to determine the size of the object based on those pixels in the distance data whose intensity value is equal to or greater than the first intensity threshold, wherein the size of the object is preferably determined as the number of pixels. The evaluation unit is configured to determine, for each pixel whose intensity value is less than the first intensity threshold, a (minimum) distance to the nearest pixel of the object, wherein the distance to the object is preferably determined as the number of pixels.The evaluation unit is designed to determine the probability value that the pixel in question represents a disturbance, depending on its intensity value, its determined distance to the nearest pixel of the object, the maximum measurable intensity and / or the intensity of the object, and the size of the object, and to recognize all those pixels as disturbances whose probability value is equal to or greater than the probability limit.

[0032] According to one embodiment, the evaluation unit is configured to determine the distance of the remitting object (to the optoelectronic sensor) based on the distance data and to identify as interference all those data components whose probability value is equal to or greater than the probability threshold and whose corresponding distance value lies within a tolerance range around the determined distance of the object. This approach is based on the understanding that a data component representing interference usually appears at approximately the same distance to the optoelectronic sensor as the (actual) remitting object. The additional filtering according to the tolerance range around the determined distance of the object (or in other words, according to a distance corridor) can be used to validate the result of the filtering according to the probability threshold.If there are multiple remitting objects in the monitoring area, the process can be executed and / or iterated separately for each detected object.

[0033] The distance of the object can refer to a (mean) relative distance to the optoelectronic sensor. The tolerance range can, for example, encompass a range of ± 50 cm, preferably ± 20 cm, preferably ± 12 cm, preferably ± 10 cm, and preferably ± 5 cm around the determined distance of the object.

[0034] According to one embodiment, the tolerance range comprises a range of ± 20%, preferably ± 15%, preferably ± 10%, preferably ± 6%, preferably ± 5%, and preferably ± 1% of the value of the determined distance of the object (from the optoelectronic sensor) around the determined distance of the object.

[0035] According to one embodiment, the evaluation unit is configured to enter each pixel in the distance data whose (obtained) intensity value is equal to or greater than the first intensity limit value into a distance histogram, to detect a peak in the distance histogram, wherein the peak is preferably the largest peak (i.e., vertex) in the distance histogram, to determine the size of the object based on the number of pixels under the peak, and to determine the distance of the object based on the position of the peak in the distance histogram.

[0036] In other words, the distance data is searched for pixels with an intensity value equal to or greater than the first intensity threshold. Such pixels can then be described as overdriven. Pixels affected by (retro)reflectors are often, or even usually, overdriven. Based on the detected overdriven pixels, particularly strongly reflecting objects (e.g., retroreflectors) can then be detected and, in particular, identified.

[0037] According to one embodiment, the data components in the distance data that are identified as interference, particularly for controlling the movement of a robot, are removed from the distance data, marked as invalid, and / or ignored during further evaluation of the distance data.

[0038] A further object of the invention is a system comprising at least one optoelectronic sensor described herein and at least one autonomous robot, wherein the optoelectronic sensor is preferably attached to the autonomous robot and can be moved by it.

[0039] According to one embodiment, the system comprises a reflector, preferably a retroreflector. The reflector (especially a retroreflector) can be positioned within the monitoring area, and the robot can move within this area. The reflector (especially a retroreflector) can serve as an obstacle marker / highlight and / or navigation feature and can be used by the optoelectronic sensor for controlling, localizing, and / or navigating the robot.

[0040] A further object of the invention is the use of an optoelectronic sensor described herein for detecting at least one object in a monitoring area.

[0041] A further object of the invention is a method for detecting at least one object in a monitoring area, wherein transmitted light is emitted into the monitoring area, wherein received light remitted from the monitoring area is received, wherein distance data over the monitoring area are obtained based on the received light, and in particular are measured using a light time-of-flight method, wherein the distance data comprise intensity values ​​and associated distance values.Furthermore, for all those data components in the distance data that do not originate from the object, and in particular for all those data components whose (obtained) intensity value is less than a first intensity limit, a distance to the object, especially to the edge of the object, is determined based on the distance data, and based on the determined distance of the data component in question to the object, a model for the scattered light behavior of the optoelectronic sensor, and the intensity value of the data component in question, a probability value is determined that the data component in question represents a disturbance.All those data components in the distance data are then recognized as disturbances whose determined probability value is equal to or greater than a predetermined probability threshold, whereby a distance, a size, an intensity and / or a remission of the remitting object is determined based on the distance data, whereby the probability value that the data component in question represents a disturbance is determined depending on the determined distance of the data component in question to the object, the intensity value of the data component in question, the maximum measurable intensity, and the size of the object.

[0042] According to one embodiment, the object comprises a reflector, preferably a retroreflector.

[0043] According to one embodiment, a distance, remission, intensity and / or size of the remitting object is determined based on the distance data.

[0044] According to one embodiment, the probability value that the relevant data component represents a disturbance is determined depending on the determined distance of the relevant data component to the object, the intensity value of the relevant data component, the maximum measurable intensity (e.g., when the distance of the data component to the object is zero and / or when the distance of the object to the optoelectronic sensor is zero) and the size of the object.

[0045] According to one embodiment, all those data components are recognized as disturbances whose probability value is equal to or greater than the probability limit and whose associated distance value lies within a tolerance range around the determined distance of the object.

[0046] It is understood that what is described regarding the optoelectronic sensor according to the invention also applies to the use of the optoelectronic sensor, the system, and the method. This applies in particular to embodiments and advantages. Furthermore, it is understood that all features and embodiments disclosed herein can be combined unless expressly stated otherwise.

[0047] The invention is described below by way of example with reference to possible embodiments and the accompanying drawing. The drawing shows: Fig. 1 a schematic representation of an optoelectronic sensor according to an embodiment of the invention; Fig. 2A a bird's-eye view representation of distance data over a monitoring area obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 2B a bird's-eye view representation of distance data over the monitoring area with a highly reflecting object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 3A a central perspective representation of distance data over a first monitoring area with a first test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 3B a central perspective representation of distance data over a second monitoring area with a second test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 3C a central perspective representation of distance data over a third monitoring area with a third test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 4A a central perspective representation of distance data over the first monitoring area without the first test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 4B a central perspective representation of distance data over the second monitoring area without the second test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 4C a central perspective representation of distance data over the third monitoring area without the third test object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 5A a central perspective representation of the distance data of the first test object, adjusted for the basic intensity; Fig. 5B a central perspective representation of the distance data of the second test object adjusted for the basic intensity; Fig. 5C a central perspective representation of the distance data of the third test object, adjusted for the basic intensity; Fig. 6A a representation of a section of the distance data of the first test object, adjusted for the basic intensity; Fig. 6B a representation of a section of the distance data of the second test object, adjusted for the basic intensity; Fig. 6C shows a section of the distance data of the third test object, adjusted for the base intensity; Fig. 7A a representation of the y-axis averaged portion of the distance data of the first test object; Fig. 7B a representation of the y-axis averaged portion of the distance data of the second test object; Fig. 7C shows a representation of the y-axis averaged portion of the distance data of the third test object; Fig. 8A a graphical representation of a model calibrated with respect to the first test object for the scattered light behavior of an optoelectronic sensor according to an embodiment of the invention; Fig. 8B a graphical representation of a model calibrated with respect to the second test object for the scattered light behavior of an optoelectronic sensor according to an embodiment of the invention; Fig. 8C a graphical representation of a model calibrated with respect to the third test object for the scattered light behavior of an optoelectronic sensor according to an embodiment of the invention; Fig. 9A a central perspective representation of distance values ​​from distance data over a monitoring area with an object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 9B a central perspective representation of intensity values ​​obtained from the distance data of the object using an optoelectronic sensor according to an embodiment of the invention; Fig. 9C a representation of a segmentation of the distance data; Fig. 9D is a central perspective representation of the distances of the segmented image points to the object; Fig. 10A a representation of the modeled scattered light of the object; Fig. 10B a central perspective representation of a determined probability map; Fig. 10C a representation of the detected image disturbances; Fig. 10D a central perspective representation of the distance values ​​of the object's distance data, corrected for image distortions; Fig. 11A a bird's-eye view representation of the distance data over the monitoring area without the object obtained by means of an optoelectronic sensor according to an embodiment of the invention; Fig. 11B a central perspective representation of the distance data over the monitoring area with the object obtained by means of an optoelectronic sensor according to an embodiment of the invention; and Fig. 11C a bird's-eye view representation of the distance data over the monitoring area with the object obtained by means of an optoelectronic sensor according to an embodiment of the invention.

[0048] Fig. Figure 1 shows an optoelectronic sensor 100, in particular a time-of-flight camera or a LiDAR sensor, according to an embodiment for detecting at least one object 40 in a monitoring area. The optoelectronic sensor 100 comprises a light transmitter 10, a light receiver 20, and an evaluation unit 30. The light transmitter 10 is configured to emit transmitted light 11 into the monitoring area. The light receiver 20 is configured to receive reflected light 12 from the monitoring area, and in particular from the object 40 in the monitoring area. The evaluation unit 30 is configured to obtain distance data about the monitoring area based on the reflected light 12, wherein the distance data includes intensity values ​​and associated distance values. The evaluation unit 30 is further configured to perform a function for all those data components (e.g.,to determine a distance to object 40 (e.g., in image points or pixels) in the distance data that does not originate from object 40, and in particular for all those data components in the distance data whose intensity value is less than a first intensity limit, to determine a distance to object 40 (e.g., in image points or pixels), especially to the edge of object 40, and in each case, based on the determined distance of the data component in question, based on a model for the scattered light behavior of the optoelectronic sensor 100, and based on the intensity value of the data component in question, to determine a probability value that the data component in question represents a disturbance, and to identify all those data components as disturbances whose determined probability value is equal to or greater than a probability limit.

[0049] Evaluation unit 30 of the in Fig. The optoelectronic sensor shown in Figure 1 can be configured to determine, based on the distance data, a distance (to the optoelectronic sensor, e.g., in meters), a size, an intensity, and / or a remission of the emitting object 40. The evaluation unit 30 can, for example, determine the size of the emitting object 40 based on those data components in the distance data whose intensity value is equal to or greater than the first intensity threshold.

[0050] Evaluation unit 30 of the in Fig. The optoelectronic sensor 100 shown in Figure 1 can be configured to determine the distance of object 40 (only) and / or to perform the detection of those data components in the distance data that represent a disturbance (only) if the size of object 40 is equal to or greater than a predetermined size limit, if the intensity of object 40 is equal to or greater than a predetermined second intensity limit, and / or if the remission of the object is greater than or equal to a remission limit. This prevents valid data components (i.e., data components of real, existing objects) from being inadvertently filtered out, which is particularly important for safety applications. If several sufficiently large and sufficiently remitting objects are detected, the disturbance detection process can be performed iteratively for each of the detected objects.

[0051] The evaluation unit 30 of an optoelectronic sensor 100, as used, for example, in Fig. As shown in Figure 1, it can be designed to recognize all those data components (e.g., pixels) as disturbances whose probability value is equal to or greater than the probability limit and whose associated distance value lies within a tolerance range around the determined distance of the object 40 (to the optoelectronic sensor).

[0052] Fig. 2A and Fig. Figure 2B shows a bird's-eye view of distance data over a monitoring area without and with a strongly reflecting object 40, obtained using an optoelectronic sensor 100, such as that used, for example, in Fig. 1 is shown. As in Fig. 2A and Fig. As shown in Figure 2B, the distance data obtained can comprise a large number of pixels, each with an intensity value and an associated distance value.

[0053] Particularly when the monitored area is scanned simultaneously (as opposed to scanning in individual measurements), stray light in the optics of the light receiver 20 of the optoelectronic sensor 100 can complicate or prevent accurate distance measurement. Strong reflection of the transmitted light by a reflecting object 40, for example, a retroreflector, can lead to distorted distance measurements around the image area of ​​the object 40. These image disturbances 21 typically occur when the stray light has a higher energy than the base level or intensity reflected back by the monitored area, which can be the case, in particular, with a dark or distant background.

[0054] Fig. 2A and Fig. 2B illustrates such a situation with a scene featuring a chair in front of several windows, where the windows serve as a substitute for a dark background. As in Fig. As can be seen in Figure 2A, noise in the distance values ​​can be observed in the area of ​​the windows; however, this noise is still easily recognizable and therefore manageable. As soon as a strongly reflecting object 40, here a retroreflector, as in Fig. When the object 40 is placed on the chair (as shown in Figure 2B), all noise values ​​or image distortions 21 assume the distance of the object 40. The scattered light overwrites some to all pixels in the vicinity of the object 40 that do not themselves reflect strongly enough. This results in a spherical image distortion or artifact formed from a cloud of pixels 21, where the radius of the spherical artifact corresponds to the distance of the object 40 and the center point is the position of the optoelectronic sensor 100 (not shown in Figure 2B). Fig. 2A and Fig. 2B shown) can correspond.

[0055] In short, the in Fig. The pixels 21 shown in Figure 2B arise due to multiple reflections and scattering of the received light at optical elements in the light receiver 20 of the optoelectronic sensor 100. These pixels 21 represent disturbances and can trigger a warning or protection field configured in the optoelectronic sensor 100. Consequently, an unnecessary safety stop of an autonomous robot may be caused, which may be moving within an industrial plant and, in particular, within the monitored area of ​​the industrial plant. It is understood that the optoelectronic sensor 100, as shown in Figure 2B, is designed to prevent such disturbances. Fig. 1 is shown, to which an autonomous robot can be attached and, in particular, can be moved by it.

[0056] The procedures described herein allow the detection (and filtering) of those pixels that have been overwritten by scattered light with a certain probability. A model for the scattered light behavior of the optoelectronic sensor 100 is used, which can also be seen as an approximation of the influence area of ​​the reflecting object 40 in conjunction with the optics of the light receiver 20.

[0057] Fig. 3A to Fig. Section 8C illustrates how a model for the stray light behavior of the optoelectronic sensor with respect to specific objects in the monitored area can be created and / or calibrated. The procedure described here is based on the fact that the stray light behavior of the optics of the light receiver 20 of the optoelectronic sensor 100, as it is e.g. in Fig. As shown in Figure 1, the model for the scattered light behavior can be known in conjunction with certain (test) objects 41, 42, 43. Based on distance measurements of an otherwise identical monitoring area with and without the test object, the model can be calibrated with respect to a specific test object 41, 42, 43. The test objects described herein can be retroreflectors. The calibration can be performed by recording actual measurement data, as shown in Figure 1. Fig. 3A to Fig. 8C illustrates this. Additionally or alternatively, a model for the scattered light behavior can also be simulated.

[0058] Fig. 3A, Fig. 3B and Fig. Figures 3C each show a central perspective representation of distance data, more precisely the intensity values ​​from the distance data, over a first, second and third monitoring area with a strongly reflecting first test object 41, second test object 42 and third test object 43 respectively, acquired using an optoelectronic sensor 100, such as that found, for example, in Fig. Figure 1 shows the size of the first test object 41, determined by the evaluation unit 30 of the optoelectronic sensor 100, which is 7971 pixels, the size of the second test object 42 is 1055 pixels and the size of the third test object 43 is 2964 pixels.

[0059] Fig. 4A, Fig. 4B and Fig. Figures 4C each show a central perspective view of distance data, more precisely the intensity values ​​from the distance data, over the first, second and third monitoring areas respectively, without the test objects 41, 42, 43, obtained using an optoelectronic sensor 100, such as that used, for example, in Fig. 1 is shown. The in Fig. 4A, Fig. 4B and Fig. The distance data shown in 4C represent a recording of the base level or base intensity of the respective monitoring area.

[0060] Fig. 5A, Fig. 5B and Fig. Figures 5C each show a central perspective representation of the distance data, more precisely the intensity values, of the first test object 41, the second test object 42, and the third test object 43, respectively, adjusted for the respective basic intensity (e.g., by subtraction). Fig. 5A, Fig. 5B and Fig. 5C are a selected section 31, a selected section 32 and a selected section 33 respectively, each marked with a rectangle, wherein these sections are in Fig. 6A, Fig. 6B respectively Fig. 6C are shown in more detail. Sections 31, 32, 33 show the intensity profile from the edge of the respective test object 41, 42, 43 outwards and especially towards the (left) side of the respective monitoring area.

[0061] As in Fig. 7A, Fig. 7B and Fig. Figure 7C shows the selected sections 31, 32, 33 averaged along the y-axis. The resulting data points 51, 52, 53 are in Fig. 8A, Fig. 8B and Fig. 8B are graphically marked as crosses. To approximate the intensity deviation due to scattered light, a fit function 61, 62, 63 can be approximated to each data point according to the following general equation 2. F(x)=e(a⋅xb+c)=e(a⋅xb)⋅ec, where x in equation 2 is the distance (e.g. measured in image points or pixels) of the respective data point to the test object 41, 42, 43, in particular to the edge of the test object 41, 42, 43, where the parameter a can take a value between -2.3 and -2.6, the parameter b can take a value between 0.2 and 0.3, and the parameter c is determined based on the maximum intensity measurable with the optoelectronic sensor 100 / max (here 20000 AU) is calculated according to the following equation 3. c=log(Imax)=log(20000)=9.9035.

[0062] Equation 2 can also be seen as a generalized model for the scattered light behavior of the optoelectronic sensor 100.

[0063] The first fit function 61 for the first test object 41, the second fit function 62 for the second test object 42, and the third fit function 63 for the third test object 43 are in Fig. 8A, Fig. 8B or Fig. 8C is marked as a continuous line. In the Fig. In the graphically represented first fit function 61 for the first test object 41, parameter a has a value of -2.3432, parameter b a value of 0.21166, and parameter c a value of 9.9035. In the Fig. In the graphically represented second fit function 62 for the second test object 42, parameter a has a value of -2.4046, parameter b a value of 0.27182, and parameter c a value of 9.9035. In the Fig. In the graphically represented third fit function 63 for the third test object 43, parameter a has a value of -2.5399, parameter b a value of 0.22513, and parameter c a value of 9.9035. In this way, a model for the scattered light behavior of the optoelectronic sensor 100 can be calibrated for each known (test) object, e.g., for each known reflector type.

[0064] The measurements show that the values ​​of parameters a and b in the model for the scattered light behavior according to Equation 2 can depend on the size (i.e., the number of pixels) of the test object 41, 42, 43. In other words, parameter a can be chosen based on the size of the test object (or the number of pixels determined in the test object) and take a value between -2.3 and -2.6. Additionally or alternatively, parameter b can be chosen based on the size of the test object (or the number of pixels determined in the test object) and take a value between 0.2 and 0.3. The smaller the test object 41, 42, 43, the larger the value of parameter b can be.This can mean that for other objects for which no calibration has yet been performed, the values ​​for the parameters a and b can be chosen depending on the determined size of object 40, for example by applying a pre-calibrated function or interpolating values ​​taken from a pre-created lookup table.

[0065] Fig. 9A to Fig. 11C illustrates how disturbances in distance data can be detected and filtered using an optoelectronic sensor 100.

[0066] Fig. 9A and Fig. Figure 9B shows a central perspective representation of distance values ​​or intensity values ​​from distance data over a monitoring area with an object 40 obtained by means of an optoelectronic sensor 100, such as that found, for example, in Fig. 1 is shown.

[0067] The pixels that are not from the one in Fig. 9A or Fig. The objects shown in Figure 9B, specifically object 40, can be identified and segmented (and / or binarized) based on an intensity threshold. For example, all pixels with an intensity value of less than 90%, preferably 95%, preferably 99%, and even more preferably 100% of the maximum measurable intensity (here 20000 AU) with the optoelectronic sensor can be identified and, as shown in Figure 9B, segmented. Fig. 9C, shown in black, is set to zero in the segmentation. The remaining pixels that have an intensity value equal to or greater than the intensity threshold, i.e., the pixels of object 40, are assigned the value 1 in the segmentation. Fig. 9C shows the pixels of object 40 in white, with an intensity limit of 19000 AU applied.

[0068] For all segmented pixels (i.e., those not originating from object 40), a (minimal) distance to the nearest pixel of object 40 is determined, as shown in Fig. 9D representation. The object 40 can, as is known, correspond to the first test object 41, so that with respect to the object 40 a model for the scattered light behavior, i.e., fit function 61, already exists. Fig. 8A, is calibrated. Fig. 10A demonstrates this using the known model for stray light behavior, i.e., Fit function 61. Fig. 8A, approximate stray light with respect to object 40. For each segmented pixel, a probability value can be determined that the pixel in question represents an image disturbance, based on its determined distance to object 40, its intensity value, and the stray light approximated by the model for the pixel's distance. More precisely, the stray light approximated for the pixel's distance is divided by the pixel's intensity value (according to Equation 1). This procedure is based on the understanding that the higher the original intensity value of a pixel is relative to the stray light from object 40, the less likely that pixel has been overwritten by stray light. Conversely, a pixel with a low intensity value near object 40 is more likely to have been overwritten by stray light.The probability values ​​determined in this way for the segmented pixels are shown in the probability map in . Fig. 10B is shown.

[0069] Those pixels 21 of the segmented pixels that represent a disturbance with a certain probability can be identified and segmented (and / or binarized) based on a probability threshold. For example, all pixels 21 that have a probability value equal to or greater than a probability threshold of 0.5 can be identified and, as in Fig. 10C, shown in white, is set to the value 1 in the segmentation. The remaining pixels of the segmented pixels, which have a probability value of less than 0.5 and therefore are unlikely to represent a disturbance, are assigned the value zero. Fig. In 10C, these (with a certain probability) valid pixels are shown in black. The representation in Fig. 10C can also be called an image distortion mask.

[0070] Optionally, the image distortion mask can be set to Fig. 10C with a plausibility check via the distance values ​​in Fig. 9A can be improved. Since the image distortions usually have the same (radial) distance value as the distance of object 40 itself, they can be adjusted in the image distortion mask in Fig. 10C retains all those pixels whose associated distance value is within a tolerance range (or distance corridor) around the one determined based on the distance data in Fig. The distance of object 40 is determined in 9A.

[0071] The distance values ​​of the distance data in Fig. 9A can then be filtered using the image distortion mask, as shown in Fig. 10D representation. The distance values ​​of the detected pixels 21, which represent a disturbance, can then be invalidated and, for example, as in Fig. 10D is shown, and the value is set to zero.

[0072] Fig. Figure 11A shows a (three-dimensional) bird's-eye view representation of the distance data over the monitoring area. Fig. 9A to Fig. 10D without object 40 obtained by means of an optoelectronic sensor according to an embodiment of the invention. Fig. 11B and Fig. Figures 11C each show a (three-dimensional) central or bird's-eye view representation of the filtered distance data from Fig. 10D, with the pixels identified as disturbances (as described herein) highlighted.

[0073] Using the methods described herein, image distortions caused by highly remitting objects, such as retroreflectors, can be detected. Optoelectronic (industrial) sensors can be configured to suppress or ignore (reflector) interference. Furthermore, knowledge of the position, size, and shape of objects, such as retroreflectors, within the optoelectronic sensor's field of view can be used to trigger an appropriate system response, such as automatically adapting the protective field geometry while simultaneously reducing the travel speed of an autonomous robot. In all cases, the availability of the optoelectronic sensor can be improved, which is generally considered an important quality criterion.

Claims

[1] Optoelectronic sensor (100), in particular a time-of-flight camera or a LiDAR sensor, for detecting at least one object (40) in a monitored area, wherein the optoelectronic sensor (100) comprises a light transmitter (10), a light receiver (20) and an evaluation unit (30), wherein the light transmitter (10) is designed to emit transmitting light (11) into the monitoring area, wherein the light receiver (20) is configured to receive received light (12) reflected from the monitoring area, and wherein the evaluation unit (30) is configured to based on the received light (12) to obtain distance data over the monitoring area, wherein the distance data include intensity values ​​and associated distance values, for all those data components (21) in the distance data that do not originate from the object (40), and in particular for all those data components (21) whose intensity value is less than a first intensity limit, to determine a distance to the object (40), in particular to the edge of the object (40), based on the distance data, and in each case based on the determined distance of the relevant data component (21) to the object (40), based on a model (61, 62, 63) for the scattered light behavior of the optoelectronic sensor (100) and based on the intensity value of the relevant data component (21) to determine a probability value that the relevant data component (21) represents a disturbance, and to identify as disturbances all those data components (21) whose determined probability value is equal to or greater than a probability limit, wherein the evaluation unit (30) is designed to determine a distance, size, intensity and / or remission of the remitting object (40) based on the distance data, wherein the probability value that the relevant data component (21) represents a disturbance is determined as a function of the determined distance of the relevant data component to the object (40), the intensity value of the relevant data component (21), the maximum measurable intensity, and the size of the object (40). [2] Optoelectronic sensor (100) according to claim 1, wherein the evaluation unit (30) is configured to determine the size of the remitting object (40) based on those data components in the distance data whose intensity value is equal to or greater than the first intensity limit. [3] Optoelectronic sensor (100) according to claim 1 or 2, wherein the evaluation unit (30) is configured to perform the detection of those data components (21) in the distance data that represent a disturbance only if the determined size of the object (40) is equal to or greater than a predetermined size limit, if the determined remission of the object (40) is equal to or greater than a predetermined remission limit and / or if the determined intensity of the object (40) is equal to or greater than a second intensity limit. [4] Optoelectronic sensor (100) according to one of claims 1 to 3, wherein the distance data comprise a plurality of pixels, each with an intensity value and an associated distance value, and wherein the evaluation unit (30) is configured to to determine the size of the object (40) based on those pixels in the distance data whose intensity value is equal to or greater than the first intensity limit, wherein the size of the object (40) is preferably determined as the number of pixels, and for each pixel (21) whose intensity value is less than the first intensity limit, to determine a distance to the nearest pixel of the object (40), wherein the distance to the object (40) is preferably determined as a number of pixels, and to determine the probability value that the relevant pixel (21) represents a disturbance, depending on its intensity value, its determined distance to the nearest pixel of the object (40), the maximum measurable intensity and / or the intensity of the object (40), and the size of the object (40), and to identify all those pixels (21) as disturbances whose probability value is equal to or greater than the probability limit. [5] Optoelectronic sensor (100) according to one of the preceding claims, wherein the evaluation unit (30) is configured to determine a distance of the remitting object (40) based on the distance data, and to identify as disturbances all those data components (21) whose probability value is equal to or greater than the probability limit and whose associated distance value lies within a tolerance range around the determined distance of the object (40). [6] Optoelectronic sensor (100) according to claim 5, wherein the distance data comprise a plurality of pixels, each with an intensity value and an associated distance value, and wherein the evaluation unit (30) is configured to to plot each pixel in the distance data whose intensity value is equal to or greater than the first intensity threshold into a distance histogram, to identify a peak in the distance histogram, wherein the peak is preferably the largest peak in the distance histogram, to determine the size of the object (40) based on the number of pixels under the peak, and to determine the distance of the object (40) based on the position of the peak in the distance histogram. [7] Optoelectronic sensor (100) according to one of the preceding claims, wherein the data components (21) detected as interference in the distance data, in particular for controlling the movement of a robot, are removed from the distance data, marked as invalid, and / or ignored in a further evaluation of the distance data. [8] Optoelectronic sensor (100) according to one of the preceding claims, wherein the object (40) comprises a reflector, and preferably a retroreflector. [9] System comprising at least one optoelectronic sensor (100) according to one of claims 1 to 8 and at least one autonomous robot, wherein the optoelectronic sensor (100) is preferably mounted on the autonomous robot and can be moved by it. [10] Use of an optoelectronic sensor (100) according to any one of claims 1 to 8 for detecting at least one object (40) in a monitoring area. [11] Method for detecting at least one object (40), in particular a reflector, preferably a retroreflector, in a monitoring area, wherein transmitted light (11) is emitted into the monitoring area; wherein received light (12) reflected from the monitoring area is received; wherein distance data over the monitored area are obtained based on the received light (12), and in particular are measured using a time-of-flight method, wherein the distance data include intensity values ​​and associated distance values; where for all those data components in the distance data that do not originate from the object (40), and in particular for all those data components (21) whose intensity value is less than a first intensity limit, in each case a distance to the object (40), in particular to the edge of the object (40), is determined based on the distance data, and in each case a probability value is determined based on the determined distance of the relevant data component (21) to the object (40), based on a model (61, 62, 63) for the scattered light behavior of the optoelectronic sensor (100) and based on the intensity value of the relevant data component (21) that the relevant data component (21) represents a disturbance; and wherein all those data components (21) in the distance data are recognized as disturbances whose determined probability value is equal to or greater than a predetermined probability limit, wherein a distance, a size, an intensity and / or a remission of the remitting object (40) is determined on the basis of the distance data, wherein the probability value that the data component (21) in question represents a disturbance is determined as a function of the determined distance of the data component in question to the object (40), the intensity value of the data component in question (21), the maximum measurable intensity, and the size of the object (40). [12] Method according to claim 11, where a distance of the remitting object (40) is determined based on the distance data; and wherein all those data components (21) are recognized as disturbances whose probability value is equal to or greater than the probability limit and whose associated distance value lies within a tolerance range around the determined distance of the object (40).

Images