Optoelectronic sensor and method for optically detecting a surveillance area

Abstract

Optoelectronic sensor (10), in particular a 3D camera, for detecting a surveillance area (12), wherein the sensor (10) comprises an image sensor (16a-b), a lighting unit (20) for at least partially illuminating the surveillance area (12) with a lighting field (26), and a lighting control (28) designed to adjust the power of the lighting unit (20) to comply with protection requirements, wherein the sensor (10) further comprises a distance-measuring optoelectronic additional sensor (38) that checks the distance at which an object (42) is located in the lighting field (26), and wherein the lighting control (28) performs the power adjustment as a function of the distance measured by the additional sensor (38) and in accordance with a permissible maximum value, wherein the maximum value is a discrete function as a function of the measured distance, which has at least one maximum value for close distances,for distances in a range (34) in which an eye is hit with maximum power density, and for longer distances.

Description

[0001] The invention relates to an optoelectronic sensor and a method for optically monitoring a surveillance area.

[0002] Numerous optoelectronic sensors utilize their own laser illumination. Due to eye protection requirements, laser illumination can either only be operated with severe limitations in optical output power, or they must be classified in higher protection classes according to laser standards, for example, above Class 1M in 3R, 3B, or 4 according to EN 60825. The strict requirements for operating the device with higher protection classes are usually unacceptable. Similar requirements may also apply when using other light sources, such as LEDs in EN 62471.

[0003] 3D cameras capture image data that also includes distance information and is referred to as three-dimensional images or depth maps. Depending on the 3D capture, active illumination may be essential for the sensor function or at least lead to better image data quality.

[0004] Time-of-flight cameras evaluate the time-of-flight of their transmitted light in the pixels of their image sensor. A well-known method for corresponding time-of-flight image sensors is photonic detection.

[0005] Stereoscopic camera systems capture multiple two-dimensional images of a scene from slightly different perspectives. Identical structures are identified in the overlapping image areas, and distances are calculated using triangulation based on the disparity and the optical parameters of the camera system. Stereoscopy, in principle, operates passively without its own illumination. However, if the scene to be monitored has low contrast or contains areas with little structure, the stereoscopic evaluation is unreliable. At least two types of errors are conceivable: a failure to locate corresponding structural elements or incorrect assignment. The result is gaps in the three-dimensional images or incorrect distance calculations. This can be prevented by artificially structuring pattern illumination.In a variation of the stereoscopy principle, only one image is taken and correlated with a known projection pattern, ultimately evaluating the distortion of the illumination pattern caused by the contours in the scene.

[0006] To generate high-quality image data with 3D cameras, even at greater distances, the lighting must be powerful. This is especially true for safety applications where the 3D camera monitors a hazard and, if necessary, secures it. On the other hand, it is desirable to maintain a laser protection class that is safe for eye protection, for example, Type 1 or 1M according to EN 60825. These contradictory requirements are not easy to reconcile.

[0007] DE 10 2009 031 732 B3 describes a stereoscopic system that initially checks a preliminary working area with low optical output power. Only if no inadmissible objects are detected there does it switch to a higher laser power. The disadvantage of this is that a distinction must be made between switch-on and normal operation, which makes the process relatively complicated. DE 10 2010 037 744 B3 refines this process by checking the near-field during switch-on using a different method than the stereo algorithm of the subsequent normal operation. Of course, this does not prevent the switching process itself.

[0008] DE 10 2009 013 735 A1 discloses a sensor for monitoring a surveillance area, in which the sensor measures the incident power per area on an object. Upon detection of an object, the power is adjusted to prevent exceeding a specified value. This requires continuous measurement of the incident radiation power, which is complex and, due to its dependence on only partially known parameters such as the object distance and the object's remission properties, is therefore correspondingly unreliable.

[0009] US 2007 / 0 001 111 A1 discloses a laser projector that uses proximity sensors to detect people within a protection zone directly on the projector, allowing it to adjust the light output depending on the speed of the projector's scanning movement and the feedback from the proximity sensors. However, eye protection viewing is not suitable for a 3D camera.

[0010] GB 2 295 740 A discloses a laser-based rangefinder with a weak and a strong laser. The strong laser is only activated if no person has been previously detected with the weak laser. This eye protection consideration, which refers to a collimated laser, is also not applicable to 3D cameras.

[0011] US Pat. No. 6,661,820 B1 presents a structured light projector for use with an image sensor. While the safe laser power is maximized, it is not adjusted to the actual objects being detected. This allows only fixed assumptions to be made, and the scope for increasing the light output in a scene with more favorable conditions than these assumptions remains unused.

[0012] In US 8 290 208 B2 and similarly in US 9 201 501 B2, the power of a laser projector is adjusted when people are in the projection field. However, this requires an extremely complex image analysis.

[0013] DE 10 2009 029 233 A1 discloses a measuring station for vehicle measurement whose measuring devices use a laser to generate a laser light pattern. Sensors are provided to protect the system. These sensors detect when an object approaches the laser's exit aperture. The laser is then switched off. Depending on the design, a wide variety of sensors are used, including infrared sensors and sensors that measure the propagation time of the signal reflected in the monitoring area and, from this, the distance to the object.

[0014] EP 2 503 355 A1 discloses a laser control system for increased eye safety. If a distance measurement indicates that an object is outside a designated operating distance range, the laser beam is switched off or attenuated.

[0015] It is therefore an object of the invention to improve the power adjustment of an illumination of a generic optoelectronic sensor.

[0016] This object is achieved by an optoelectronic sensor and a method for optically monitoring a surveillance area according to claim 1 and 8, respectively. The sensor has a preferably divergent illumination unit for illuminating an extended scene of a 3D camera. In order to illuminate the surveillance area as well as possible and achieve a long range, while also meeting safety requirements such as eye protection, the illumination unit's power is adjusted to suit the situation. The invention is based on the basic idea of ​​using an additional sensor to obtain information about possible objects in the illumination field that should be taken into account. The additional sensor is a separate second sensor in addition to the image sensor and the illumination unit of the main sensor, but can also utilize common components such as the power supply, housing, and possibly some optical elements.The lighting control system adjusts the output to the measured distance of an object within the illumination field. This includes the case of an object-free illumination field, because in this case, the additional sensor provides distance information that any objects are farther away than its measuring range.

[0017] The invention has the advantage of responding to a situational risk assessment for protection against electromagnetic radiation with appropriate power adjustment. This avoids the conventional design based on worst-case assumptions, which unnecessarily limit the energy balance. This allows the lighting unit to be operated at high lighting power while maintaining, for example, the classification as a type 1M laser device according to DIN EN 60825-1. Additional safety measures, such as those required by higher laser protection classes, are not required. A switch-on phase with reduced power of the lighting unit is not necessary; the sensor operates directly in normal operation, provided the additional sensor, which measures the distance independently, does not detect a vulnerable object.There are inexpensive and compact additional distance measuring sensors that can be easily integrated into the sensor or even the lighting unit or even retrofitted.

[0018] The illumination field preferably has a region in which an eye is hit with maximum power density, with the additional sensor measuring the distance relative to this region. Contrary to first appearances, this region is by no means the shortest distance. Although the eye receives a large amount of light there, it is distributed over a larger retinal area. Therefore, it is useful to determine the most dangerous distance range and measure the distances used for power adjustment relative to it. Furthermore, the additional sensor preferably measures collinearly, or parallel, to the direction of propagation of the illumination, so that objects at the relevant location are detected in the relevant direction.

[0019] Power adjustment is preferably carried out according to a permissible maximum value. To ensure a good energy balance, it is important not only to ensure that the maximum value is not exceeded, but also that it is at least nearly reached, thus maximizing the potential lighting performance. The maximum value is preferably derived from eye protection regulations such as the EN 60825 standard.

[0020] According to the invention, the maximum value is adapted to the measured distance. This is done in the form of a discrete function of the maximum value as a function of the measured distance. In practice, even a few levels of a discrete function may suffice; at least one maximum value is provided for close distances, for distances in the range where an eye is hit with maximum power density, and for longer distances. For objects at certain distances, the appropriate response may also be to immediately switch off the illumination unit, i.e., the maximum value for this distance range is set to zero.

[0021] The lighting control is preferably designed to operate the lighting unit in a pulsed mode and to control the power adjustment via the pulse repetition frequency, the pulse amplitude, and / or the pulse length. The decisive factor for eye damage is not the instantaneous power, but rather the average integrated power. Therefore, the power adjustment does not necessarily have to adjust the pulse amplitude or only the pulse amplitude, but can also use the duration and frequency of the pulses as a control variable. The average optical output power can be controlled particularly easily via the pulse sequences, and the power can be better focused, possibly synchronized with reception time windows.

[0022] The additional sensor preferably has a SPAD light receiver. SPADs (single-photon avalanche diodes) are avalanche photodiodes operated in so-called Geiger mode, biased with a high bias voltage above the breakdown voltage. This allows even a single incident photon to trigger the avalanche breakdown and thus a detection signal. A rangefinder with a SPAD light receiver can be particularly inexpensive and compact, yet still provide sufficiently precise distance measurements.

[0023] The additional sensor preferably has its own illumination unit. This allows, for example, distance to be measured using a time-of-flight method. The dedicated illumination unit is preferably eye-safe across its entire illumination range and is not adjusted to the situation. Therefore, it is generally weaker than the illumination unit of the 3D camera. A shorter range is unproblematic, since at greater distances the illumination field is no longer hazardous anyway, especially if it is divergent.

[0024] The dedicated illumination unit preferably has an extended ring- or linear beam cross-section. With an alternative point-based measurement, the additional sensor monitors only a very small portion of the illumination field. This may be sufficient if the critical areas are concentrated locally within the illumination field and are monitored with one or a few point measurements. However, with a larger light spot and therefore a larger detection range, a correspondingly larger portion of the illumination field is monitored, so that the situation of an overlooked object may occur less frequently or even not at all.

[0025] Preferably, several additional sensors are provided. This is an alternative or additional measure to better cover the illumination field. The additional sensors can actually be several separate sensors, but an arrangement of a multiple light source, such as a laser line or a VCSEL array, with a receiver matrix can also be considered multiple additional sensors in this sense. Similarly, the evaluation for determining the distance can be separate or joint. The additional sensors can each measure either point-like or with an extended beam cross-section, just like the individual additional sensor.

[0026] The sensor, designed as a 3D camera, can use all known techniques for capturing depth maps, such as a time-of-flight camera or a stereo camera, as explained in the introduction.

[0027] In a preferred development, a shutdown device is provided, which is designed to output a shutdown signal to a monitored hazard source or machine if an unauthorized object intrusion occurs. An unauthorized object intrusion can be detected by the 3D camera itself, for example, because an unknown object is located in a protected area, particularly too close to the monitored machine. However, it may also be the case that an object detected by the additional sensor requires a power adjustment that no longer ensures a reliable monitoring function of the 3D camera, so that a safety-related shutdown is also performed here as a precautionary measure.

[0028] The lighting unit preferably comprises a laser light source. Laser light sources are particularly bright, and their coherence properties can be used to create structural patterns with high efficiency. Thus, a pattern generation element can project a bright, structured illumination pattern into the monitored area. Even with alternative light sources, such as LEDs, an output power that is potentially harmful to the eyes is possible, and therefore the invention can be used to comply with safety regulations, for example, according to the relevant standard DIN 62471.

[0029] The method according to the invention can be further developed in a similar manner and thereby exhibits similar advantages. Such advantageous features are described by way of example, but not exhaustively, in the subclaims following the independent claims.

[0030] The invention will be explained in more detail below with regard to further features and advantages, using exemplary embodiments and with reference to the accompanying drawings. The figures of the drawing show: Fig. 1 a schematic representation of a stereoscopic 3D camera; Fig. 2 an exemplary functional representation depending on the distance on the left of the diameter of the light spot of a laser source on the retina, in the middle of the intensity of the laser source and on the right of the resulting power density on the retina; Fig. 3 a schematic representation of the illumination field and the most dangerous point for the eye of a 3D camera; Fig. 4 a schematic representation of a 3D camera similar Fig. 3 with additional sensor; Fig. 5 is a schematic representation in which the illumination field of the 3D camera according to Fig. 4 an object is located; and Fig. 6 a schematic representation according to Fig. 5, whereby after detection of the object at a dangerous distance, the illumination field is switched off; and Fig. 7 a schematic block diagram of an additional sensor for measuring the distance of objects in the illumination field.

[0031] Fig. 1 shows a schematic representation of the general structure of a 3D camera 10 according to the stereoscopic principle for detecting a spatial area 12. However, the invention also includes other sensors, in particular other cameras and 3D cameras such as the time-of-flight cameras mentioned above or cameras that correlate a projection pattern with a recorded image.

[0032] Two camera modules 14a, 14b are mounted at a known fixed distance from each other and each capture images of the spatial area 12. Each camera contains an image sensor 16a, 16b, typically a matrix-shaped recording chip that captures a rectangular pixel image, for example, a CCD or CMOS sensor, possibly configured as a SPAD matrix. Each image sensor 16a, 16b is assigned a lens 18a, 18b with imaging optics.

[0033] An illumination unit 20 is provided in the center between the two image sensors 16a, 16b; this spatial arrangement is to be understood only as an example. The illumination unit 20 has a light source 22, for example, one or more lasers or LEDs, and a pattern generation element 24, which is designed, for example, as a mask, phase plate, microlens array, or diffractive optical element. Thus, the illumination unit 20 is capable of illuminating the spatial region 12 with an illumination field 26 having a structured pattern. An illumination controller 28 switches the light source 22 and determines its light output.

[0034] A controller 30 is connected to the two image sensors 16a, 16b and the illumination controller 28. The controller receives image data from the image sensors 16a, 16b and calculates three-dimensional image data (range image, depth map) of the spatial area 12 using stereoscopic disparity estimation. The structured illumination pattern ensures good contrast and a clearly identifiable structure of each image element in the illuminated spatial area 12. Accordingly, the structured pattern and thus the pattern generation element 24 are not required for a sensor with a different distance measurement, such as a time-of-flight camera.

[0035] Depending on the application of the 3D camera 10, the three-dimensional image data is output at an output 32 or further processed internally. For example, in a safety-related application, it is checked whether objects are located in a hazardous area and, if necessary, a safety-related shutdown signal is output to a hazard source. For this purpose, output 32 can be configured as a safety output (OSSD, Output Signal Switching Device). A sensor used in safety technology is designed to be fail-safe. For non-contact protective devices, the corresponding requirements are standardized in EN 61496-1 or IEC 61496, as well as in DIN EN ISO 13849 and EN 61508. A corresponding standard for safety cameras is currently being developed.

[0036] When operating an optoelectronic sensor with active illumination, such as the 3D camera 10, adequate protection against electromagnetic radiation must be ensured. The protection requirements are explained using the typical example of laser eye protection according to EN 60825. The eye is generally the most sensitive target, so other conceivable protection requirements are automatically met. However, other protection goals, such as skin protection or purely technical reasons such as avoiding excessive stray light, are also conceivable.

[0037] To assess the hazard and comply with a laser class such as 1M, all accessible distances between the eye and the location of the apparent source must be considered and evaluated with regard to the damaging power density, i.e. the ratio of incident power to the area of ​​the retinal image. The most unfavorable distance is relevant for the classification of the laser device. Due to a small exit pupil of the projection lens of the illumination unit 20 at a large field angle, the eye pupil acts as a field stop and crops the image of the source with increasing distance. At short distances, the retinal image becomes increasingly larger, so that the overall increasing amount of light is distributed over a larger retinal area within the iris, which acts as a measuring stop.At very large distances, however, the image of the source on the retina becomes very small, thus concentrating the received radiation. However, overall, very little light falls on the retina due to the high divergence of the illumination. The hazard is therefore maximum at a certain intermediate distance. This most unfavorable or dangerous distance is relevant for the classification of the laser device.

[0038] Fig. Figure 2 illustrates how the most dangerous distance can be determined. The eye is modeled as an auxiliary lens that captures part of the radiation from the illumination field 26. The left part shows Fig. 2 First, the radius of the laser source imaged onto the retina by the auxiliary lens is calculated as a function of the distance to the laser source. To account for the variable accommodation capacity of the human eye, lens focal lengths between f' = + 14.5 mm and f' = + 17 mm are also considered for each distance. The minimum focal length of f' = + 14.5 mm corresponds to an object distance of g = 100 mm, while the maximum value of f' = + 17 mm corresponds to an object distance of g = ∞.

[0039] In the middle part of the Fig. Figure 2 shows the distance-dependent intensity curve for a measuring aperture of 7 mm diameter corresponding to the iris. In the right part of the Fig. 2 shows the power density on the retina, which is relevant for eye damage, again plotted as a function of distance. For this purpose, the power incident on the retina is calculated according to Fig. 2 Center through the area of ​​the resulting retinal image according to the radius according to Fig. 2 split on the left.

[0040] How Fig. As can be seen on the right in Figure 2, the power density on the retina of the eye forms a sharp maximum depending on the observation distance from the hazard source. For different distances, the hazard would be lower. All limit value determinations for Class 1M are based on this scenario, which is the most dangerous for the human eye, with the maximum power density hitting the retina.

[0041] Fig. 3 illustrates again the most dangerous area 34 in the illumination field 26. The Fig. 1, the 3D camera 10 explained as an example is shown here only as a functional block. The detection of the most unfavorable distance on the optical axis 36 has just been described. Laterally, i.e. in the Fig. 3 up or down, the radiant power only decreases due to the edge drop of the projection optics.

[0042] With a situational risk assessment that takes into account the distance between a person actually present and the light source 22, the permissible lighting output can be adjusted in order to achieve a stronger lighting that still meets the required eye protection class for each distance.

[0043] Fig. 4 shows an extension of the 3D camera 10 by an additional sensor 38, which is itself a distance-measuring optical sensor. An electrical control circuit is provided, for example, within the lighting control 28, which prevents the light output limits determined for the 3D camera 10 depending on the application and hazard class from being exceeded. However, the light output limits are not statically determined based on worst-case assumptions, but are adapted to the situation. For this purpose, the additional sensor 38 checks whether there is an object in the beam path 40 of the additional sensor 38 in the most dangerous area 34 or determines the distance of an object detected in the illumination field 26 from the most dangerous area 34. This makes it possible to operate the lighting unit 20 at a higher power, at which the eye protection limits in the most dangerous area 34 are no longer met.This does not affect the classification because accessibility via the active sensor function of the additional sensor 38 is excluded.

[0044] Fig. 5 shows, by way of example, a first situation with an object 42 at a greater distance, in particular beyond the most dangerous area 34. The 3D camera 10 can remain in normal operation and increase the power of the lighting unit 20 depending on the distance of the object 42.

[0045] Fig. Figure 6 shows another example situation with an object 42 very close to and in front of the most dangerous area 34. The power of the lighting unit 20 must be throttled accordingly. A particularly drastic consequence is even shown: The lighting field 26 is switched off completely.

[0046] As illustrated by these two examples, the additional sensor 38 provides distance values ​​that can be used for power adjustment. The distance values ​​are preferably measured relative to the most dangerous area 34 and collinear, or parallel, to the propagation direction of the electromagnetic radiation of the illumination field 26. Furthermore, the measurement is preferably taken as close as possible to the optical axis 36 of the illumination unit 20. This ensures that the measurement is taken in the immediate vicinity of the hazard, and, for example, a person's head can be detected in this way before the human eye is exposed to the hazardous electromagnetic radiation.

[0047] Depending on the measured distance D to a detected object, new threshold values ​​S(D) are then set for the permissible limits depending on the protection class. This dependency can be stored as a continuous or discrete function. The threshold values ​​S(D) are transferred to the control circuit so that protection against electromagnetic radiation is always provided according to the currently prevailing hazard situation. When selecting the threshold, the latency of the affected overall system, consisting of the additional sensor 38, lighting control 28, and lighting unit 20, as well as the injury time, should be taken into account.

[0048] Fig.Figure 7 shows a very schematic block diagram of an exemplary structure of the additional sensor 38. Distance measurement is performed here using a time-of-flight (TOF) method, although other methods are also conceivable. A dedicated light transmitter 44 emits a light signal, which, after remission from an object, is registered by a light receiver 46, preferably a sensitive and compact SPAD light receiver. The light signal is modulated, either with short light pulses or a periodic signal, and in a time-of-flight unit 48, the time interval between the transmission and reception of a pulse, or a phase shift, is determined accordingly and converted into a distance using the constant speed of light.

[0049] The light transmitter 44 is preferably eye-safe. As a result, the range remains limited and can even be less than that of the illumination field 26. This only has an impact on a lighting unit 20 with collimated radiation, which would not be practical for a 3D camera 10. For divergent radiation, which is typically generated by the lighting unit 20 because surfaces are regularly illuminated, it is sufficient if the most dangerous area 34 is located within the range, possibly with a certain buffer. The light transmitter 44 is preferably separated from the illumination field 26, for example, in terms of time, coding, or wavelength, in order not to distort the three-dimensional image data. It is also conceivable for the additional sensor 38 to be deactivated as soon as the 3D camera 10 is in normal operation, and then to replace its function by evaluating the image data.It is advantageous to design the light transmitter 44 as a point source, as this beam cross-section promises the highest distance measurement accuracy. However, a larger portion of the illumination field 26 can be covered with an extended light spot, such as a line or ring. This effect can also be achieved by using multiple additional sensors 38.

[0050] It is possible to use status LEDs to indicate whether the lighting unit 20 is operating in a mode with a specific average optical radiation power. The additional sensor 38 can be an integral part of the lighting unit 20 or the 3D camera 10, or it can be retrofitted. The inventive power adjustment using an additional distance-measuring sensor 38 is useful not only in a sensor, specifically a 3D camera 10, but also, for example, in a laser device in science, an industrial laser for cutting or welding, or in telecommunications. This allows, for example, adjustment operation with low power and control operation with high power to be controlled.

Claims

[1] Optoelectronic sensor (10), in particular a 3D camera, for detecting a surveillance area (12), wherein the sensor (10) comprises an image sensor (16a-b), a lighting unit (20) for at least partially illuminating the surveillance area (12) with a lighting field (26), and a lighting control (28) which is designed to adjust the power of the lighting unit (20) to comply with protection requirements, wherein the sensor (10) further comprises a distance-measuring optoelectronic additional sensor (38) which checks the distance at which an object (42) is located in the lighting field (26), and wherein the lighting control (28) carries out the power adjustment as a function of the distance measured by the additional sensor (38) and in accordance with a permissible maximum value, wherein the maximum value is a discrete function as a function of the measured distance, which has at least one maximum value for close distances,for distances in a range (34) in which an eye is hit with maximum power density, and for longer distances. [2] Sensor (10) according to claim 1, wherein the additional sensor (38) measures the distance relative to the area (34). [3] Sensor (10) according to claim 1 or 2, wherein the illumination control (28) is designed to operate the illumination unit (20) in a pulsed manner and to control the power adjustment via the pulse repetition frequency, the pulse length and / or the pulse amplitude. [4] Sensor (10) according to one of the preceding claims, wherein the additional sensor (38) comprises a SPAD light receiver (46). [5] Sensor (10) according to one of the preceding claims, wherein the additional sensor (38) has its own lighting unit (44). [6] Sensor (10) according to claim 5, wherein the separate illumination unit (44) has an extended annular or linear beam cross-section. [7] Sensor (10) according to one of the preceding claims, wherein a plurality of additional sensors (38) are provided. [8] Method for the optical detection of a surveillance area (12) which is at least partially illuminated by a lighting unit (20) with a lighting field (26), wherein a power adjustment of the lighting unit (20) is carried out in order to comply with protection requirements, wherein a distance-measuring optoelectronic additional sensor (38) is used to check the distance at which an object (42) is located in the lighting field (26), and wherein the power adjustment is carried out as a function of the distance measured by the additional sensor (38) and in accordance with a permissible maximum value, wherein the maximum value is a discrete function as a function of the measured distance, which has at least one maximum value for close distances, for distances in a region (34) in which an eye is hit with maximum power density, and for greater distances.

Images