Calibration device and calibration procedure for a camera system

Abstract

Calibration device (50) for calibrating optical properties of a camera system, in particular a time-of-flight camera system (1), with an imaging unit (52) which provides a planar reference surface (54) on one side and a calibration unit (60) for calibrating the optical properties of the camera system (1) from images A of the reference surface (54) produced by means of this camera system (1), characterized in that the imaging unit (52) for creating patterns M on the reference surface (54) has switchable light sources (56), wherein the reference surface (54) has light emission openings (58) for the light of the light sources (56) and each light source (56) is assigned at least one light emission opening (58).

Description

[0001] The invention relates to a calibration device for calibrating the optical properties of a camera system, comprising an imaging unit that provides a planar reference surface on one side and a calibration unit for calibrating the optical properties of the camera system from images of the reference surface created by this camera system. The invention further relates to the use of such a device for calibrating the optical properties of a camera system and a corresponding method for calibrating the optical properties of a camera system, in particular a time-of-flight camera system, by means of an imaging unit that provides a planar reference surface on one side, wherein the optical properties of the camera system are calibrated via images of the reference surface of the imaging unit created by this camera system.

[0002] The term "camera system" encompasses not only 2D cameras but also, in particular, time-of-flight (TOF) camera systems that derive time-of-flight information from the phase shift of emitted and received radiation. PMD cameras with photomixing detectors (PMDs), such as those described in DE 197 04 496 A1 and available, for example, from companies like 'ifm electronic GmbH' or 'pmdtechnologies ag' as the Frame-Grabber O3D or the "CamBoard pico flexx," are especially suitable as time-of-flight or 3D TOF cameras. Camera systems that obtain distance information by detecting emitted structured light are also included.

[0003] German patent application DE 10 2011 122 335 A1 discloses a calibration device for a camera system, wherein this calibration device comprises planar test plates with test images and a calibration unit for calibrating the optical properties of the camera system from images of the test plates created by this camera system. The camera system is, in particular, a binocular camera module with a binocular image acquisition unit. Such a binocular camera module is also referred to as a type of 3D camera. The patent application further describes a corresponding procedure for calibrating the optical properties of the camera system.

[0004] From DE 10 2009 047 303 A1, a calibration method for a lidar sensor is known in which the calibration device is composed of a plurality of reflector elements. A diaphragm mask, for example an LCD screen, is proposed for structuring a calibration pattern; its transmission can be selectively adjusted. In addition to the reflector elements, light emitters are provided that can be triggered by the received light. The device is intended for calibrating the intensity and / or light transit time of the lidar system.

[0005] DE 195 36 297 A1 discloses a device and a method for calibrating optical 3D sensors, in which test patterns are projected into the camera's field of view or presented on a reference surface, wherein some of the marks have a code so that the orientation of the patterns in space is recognizable for the camera.

[0006] US patent 5,825,464 A deals with a calibration system for a lidar system. Light from the lidar system is directed into an integrating sphere and coupled into several optical fibers. After traveling a specific path, the light is reflected back into the integrating sphere via the optical fibers, and light with different travel times can be detected at the lidar sensor, allowing the sensor to be calibrated with respect to distance measurements.

[0007] EP 2 927 711 A1 addresses the problem that some laser scanners have a dead zone with a reference target for functional monitoring in which no environmental detection can be carried out, and proposes to further develop an evaluation unit in such a way that objects can also be detected in this area.

[0008] The object of the invention is to provide a calibration device and a method for calibrating the optical properties of a camera system, enabling the simplest possible calibration process. This calibration then makes it possible to improve the accuracy of the camera system.

[0009] The problem is advantageously solved by the calibration device and the method with the features of the independent claims.

[0010] According to the invention, the calibration device is provided that the imaging unit for creating patterns on the reference surface has switchable light sources, wherein the reference surface has light emission apertures for the light from the light sources, and each light source is assigned at least one, and in particular exactly one, light emission aperture. The light sources are preferably LEDs (LED: Light Emitting Diodes). In this way, switchable, self-illuminating markers can be created that form the patterns M. This measure makes it possible to change the pattern (in the simplest case: pattern on / pattern off) without mechanical intervention, which enables precise calibration.

[0011] The markers can, in principle, have two different functions. Firstly, they can be used purely as position markers, i.e., to determine the position and orientation of the reference surface relative to the camera optics of the camera system. The actual (test pattern) structures of the reference surface for calibration can be formed by other elements. Secondly, however, the patterns formed by the markers themselves can constitute these actual structures for calibration.

[0012] In principle, the light sources can be arranged away from the reference surface and their light can be guided, for example, via optical fibers to the corresponding light emission openings. However, according to a preferred embodiment of the invention, the switchable light sources are arranged in the area of ​​the light emission openings in such a way that light from these light sources can exit at the side of the reference surface.

[0013] In particular, it is provided that the switchable light sources are arranged on the side of the imaging unit opposite the planar reference surface.

[0014] According to a further preferred embodiment of the invention, the pattern-determining arrangement of the light emission apertures is designed such that the position of a light aperture can be uniquely determined via pattern recognition. The pattern recognition is preferably implemented in the calibration unit.

[0015] Advantageous is a calibration device for a camera system, with a planar reference surface, wherein the reference surface has several light emission apertures, with switchable light sources which are arranged in the area of ​​the light emission apertures in such a way that light from these light sources can exit at a top surface of the reference surface, wherein the arrangement of the light emission apertures is designed in such a way that the position of a light aperture can be uniquely determined by means of pattern recognition.

[0016] According to a further preferred embodiment of the invention, the calibration device comprises at least one diffuser arranged on the side of the reference surface. The diffuser is, for example, designed as a foil-shaped diffuser, i.e., as a diffusely scattering foil.

[0017] Furthermore, it is advantageous that the light exit aperture is smaller than the diameter of an exit lens of the light source. The size of the markers is determined by the size of the apertures, independent of the dimensions of the light sources.

[0018] According to a preferred embodiment of the invention, the calibration device further comprises a control unit for controlling the light sources. The control unit is preferably connected to the calibration unit via a signal connection or is even part of the calibration unit. The patterns are specified by the control unit.

[0019] In the use of the calibration device according to the invention for calibrating the optical properties of a camera system, it is provided that this calibration device is designed as the device described above.

[0020] According to a preferred embodiment of the inventive use, the camera system to be calibrated is brought into a unique measuring position in relation to the planar reference surface, and the measuring position is chosen such that the sensor of the camera system to be calibrated detects all light emission apertures of the reference surface.

[0021] In the inventive method for calibrating the optical properties of a camera system, in particular a time-of-flight camera system, it is provided that the imaging unit has switchable light sources for creating patterns on the reference surface and that the patterns created by the light of the light sources are used for calibration.

[0022] The invention will now be explained in more detail using exemplary embodiments and with reference to the drawings.

[0023] They show: Fig. 1 schematically a time-of-flight camera system, Fig. 2 a modulated integration of generated charge carriers, Fig. 3 an arrangement of the time-of-flight camera system and a calibration device according to a preferred embodiment of the invention , Fig. 4 a section of the reference surface of an imaging unit of the calibration device, Fig. 5 an LED pattern created on the reference surface and Fig. 6 a recording of the sample according to Fig. 5.

[0024] In the following description of preferred embodiments, identical reference numerals denote identical or comparable components.

[0025] Fig. Figure 1 shows a measurement situation for an optical distance measurement with a time-of-flight camera system 1, as is known, for example, from DE 197 04 496 A1.

[0026] The time-of-flight camera system 1 comprises a transmitter unit or illumination module 10 with an illumination 12 and an associated beam shaping optics 15, as well as a receiver unit or time-of-flight camera 20 with a camera optics (receiving optics) 25 and a time-of-flight sensor 22.

[0027] The time-of-flight sensor 22 has at least one time-of-flight pixel, preferably also a pixel array, and is in particular designed as a PMD sensor. The receiving optics 25 typically consist of several optical elements to improve the imaging properties. The beam-shaping optics 15 of the transmitting unit 10 can, for example, be designed as a reflector or lens optics. In a very simple embodiment, optical elements can optionally be omitted on both the receiving and transmitting sides.

[0028] The measurement principle of this arrangement is essentially based on the fact that, starting from the phase shift of the emitted and received light, the travel time and thus the distance traveled by the received light can be determined. For this purpose, the light source 12 and the light travel time sensor 22 are connected via a modulator 30 together with a specific modulation signal M o with a basic phase position φ0. In the example shown, a phase shifter 35 is also provided between the modulator 30 and the light source 12, with which the basic phase φ0 of the modulation signal M0 of the light source 12 is shifted by defined phase positions φ var can be shifted. For typical phase measurements, phase positions of φ are preferably used. var = 0°, 90°, 180°, 270° used.

[0029] Depending on the set modulation signal, the light source 12 sends an intensity-modulated signal S p1with the first phase position p1 or p1 = φ0 + φ var off. This signal S p1 or, in the case shown, the electromagnetic radiation is reflected by an object 40 and arrives with a corresponding phase shift Δφ(t) due to the distance traveled. L ) with a second phase position p2 = φ0 + φ var + Δφ(t L ) as a received signal S p2 on the light time-of-flight sensor 22. In the light time-of-flight sensor 22, the modulation signal M is o with the received signal S p2 mixed, whereby the phase shift or the object distance d is determined from the resulting signal.

[0030] Infrared LEDs are preferably suitable as the illumination source or light source 12. Of course, other radiation sources in other frequency ranges are also conceivable, in particular light sources in the visible frequency range.

[0031] The basic principle of phase measurement is schematically shown in Fig. Figure 2 shows the time course of the modulation signal M0, which controls the lighting 12 and the light-time-of-flight sensor 22. The light reflected from object 40 arrives as the received signal S. p2 according to its light travel time t L phase-shifted Δφ(t L ) to the time-of-flight sensor 22. The time-of-flight sensor 22 collects the photonically generated charges q over several modulation periods in the phase of the modulation signal M0 in a first accumulation gate Ga and in a phase shifted by 180° M0 + 180° in a second accumulation gate Gb. From the ratio of the charges qa, qb collected in the first and second gates Ga, Gb, the phase shift Δφ(t) can be determined. L ) and thus determine a distance d of the object.

[0032] The Fig. Figure 3 shows an arrangement of the already in Fig. Figure 1 shows a time-of-flight camera system 1 and a calibration device 50 for calibrating the optical properties of the camera system 1. This calibration device 50 comprises an imaging unit 52 which provides a planar reference surface 54 on one side. This imaging unit 52 has switchable light sources 56 for creating patterns M on the reference surface 54. The reference surface 54 in turn has (in Fig. The calibration device 50 has four light exit apertures 58 (shown in detail) for the light from the light sources 56, with each light source 56 having exactly one light exit aperture 58 assigned to it. The switchable light sources 56 are arranged in the area of ​​the light exit apertures 58 such that light from these light sources 56 can exit at the side of the reference surface 54. The calibration device 50 further comprises a calibration unit 60 for calibrating the optical properties of the camera system 1 from the images A of the reference surface 54 created by this camera system 1. The calibration unit 60 is connected to the time-of-flight camera system 1 via signal and data transmission. Finally, the calibration device 50 also includes a control unit 62 for controlling the light sources 56. The control unit 62 is in signal communication with the calibration unit 60.

[0033] The following function results: To calibrate the optical properties of camera system 1 (in short: to calibrate camera system 1) using the calibration device 50, a planar reference surface 54 with different patterns M is provided by the imaging unit 52. These patterns are created using switchable, self-illuminating markers. The patterns M are defined by the calibration device 50. To create the patterns M, the imaging unit 52 has the switchable light sources 56 and the openings 58 of the reference surface 54. The imaging unit 52 is controlled by the control unit (the control device of the imaging unit) 62. For the calibration of the camera system, these patterns M created by the light of the light sources 56 are used by taking photographs A of these patterns M with the camera 20 of camera system 1.Subsequently, the optical properties of the camera system 1 are calibrated using the calibration unit 60 from the images A of the reference surface 54 of the imaging unit 52. This calibration corrects imaging errors of the camera optics 25.

[0034] The light emitted through the openings 58 from the light sources 56 serves as a marker for calibration. In the example shown, LEDs form the switchable light sources 56. Switchable LEDs as markers can be detected very robustly using very simple algorithms. A unique assignment, necessary for lens calibration, can be ensured by using different LED switching groups. The intensity distributions of the individual LED spots in the image enable sub-pixel-accurate detection. A significant advantage arises for the calibration of 3D cameras that operate, for example, according to the "ToF" or "Structured Light" principle, since the actual 3D absolute coordinates can be precisely determined using LED markers, but these markers do not interfere with 3D calibration because they can be removed instantly without any mechanical modification.

[0035] Lens calibration is typically performed using markers in the image, such as black circles, checkerboard patterns, or lines. Calibration of 3D cameras with active lighting, such as Time-of-Flight or Structured Light, is negatively affected by markers in the scene.

[0036] To overcome these disadvantages, the invention provides a planar reference surface which is fitted with LEDs on its back side. These LEDs can be switched individually or in groups. The LEDs can shine through the reference surface 54 at precisely positioned points. This can be achieved through small openings 58, in particular CNC-drilled holes, in the reference surface 54. The openings / holes 58 can be closed with a layer acting as a diffuser 64 (for example, foil, paper, wallpaper, paint).

[0037] One or more camera systems 1 are mechanically aligned with precision with respect to the LED pattern. The pixels generated by the LEDs in the cameras 20 can be reliably separated from the background by two images (LED on / LED off) using very simple algorithms.

[0038] By switching individual LEDs or groups of LEDs, a robust, unambiguous assignment to a pattern with all reference points is simplified.

[0039] The projections of the LED spots can also be smaller than a single pixel can resolve, because the often-present residual blur, which can be attributed to various causes, often allows several pixels to detect light intensity. By calculating a local, intensity-weighted average, the LED projection in the image plane can be determined with a higher resolution than pixel-perfect (sub-pixel accuracy).

[0040] The colors of the LEDs (visible, infrared, UV) can be mixed as desired. This allows, for example, the calibration of an RGB camera together with an IR camera.

[0041] A significant advantage arises for the calibration of 3D cameras that operate according to the "ToF" or "Structured Light" principle, for example, because the actual 3D absolute coordinates can be precisely determined using LED markers, but these markers do not interfere with a 3D calibration, as they can be removed instantly without any mechanical change.

[0042] Drift effects have a significant impact on the data quality of 3D cameras. For example, temperature changes can alter the mechanical geometries as well as the electrical properties of individual camera components. The rigid mechanical design required for the methodology described above enables precise calibration of these drift effects. Data acquisition can be achieved either by utilizing the camera's self-heating or by actively controlling the camera's recording temperature.

[0043] Fig. Figure 4 shows a section of the reference plate of the imaging unit 52 in the area of ​​a light exit aperture 58 or LED position. The diameter of the light exit aperture 58 is significantly smaller than the dimensions or exit lens of the light source 56 (here, the LED). For an LED diameter of, for example, 5 mm, the bore of the light exit aperture 58 is preferably less than 3 mm and particularly preferably less than or equal to 1 mm.

[0044] A diffuser 64 is arranged above the light exit aperture 58. This film-like diffuser 64 is preferably designed, particularly in terms of thickness and / or scattering properties, such that the light exit apertures 58 are not visible from the camera 20 being calibrated when the light source 56 is switched off. Preferably, the diffuser 64 has a matte finish on its upper surface.

[0045] This design has the advantage that when (point) pattern M is switched off, the reference surface 54 can be used as a distance reference surface.

[0046] In Fig. Figure 5 shows a possible pattern M (LED pattern). The LEDs are positioned on the reference surface 54. The grid has a regular structure and is precisely drilled and populated, for example, using a CNC machine. In the illustrated case, the black dots represent an IR LED. To simplify the alignment of a camera system 1 to be calibrated, an LED or other light source 56 with a visible spectrum, for example red, is provided in the center of the pattern M, marked here with "x".

[0047] In general, any LED position can be used for this process. However, by cleverly selecting the LED pattern M, the one-to-one mapping relevant for calibration can be performed in a single image using pattern recognition algorithms. In the pattern shown here, the LED density decreases towards the edges. This ensures that the projected LEDs remain distinguishable in the image plane despite high distortion.

[0048] Fig. Figure 6 shows an example of a recording A of the according to Fig. 5 projected light patterns M, which were distorted by the recording camera 20. The LED positions can be uniquely assigned using standard pattern recognition algorithms (from the calibration unit 60), so that image and / or distance errors can be corrected accordingly. Reference sign 1 Light time-of-flight camera system 10 Lighting modules 12 Lighting 20 receivers, time-of-flight camera 22 Light time-of-flight sensor 25 Camera optics (time-of-flight camera) 30 Modulator 35 Phase shifters, lighting phase shifters 40 objects 50 Calibration device 52 units, imaging 54 Reference area, planar 56 Light source 58 Opening 60 calibration units 62 Control unit 64 Diffuser, foil-like φ, Δφ(t L ) runtime-related phase shift φ var Phase position φ0 basic phase M0 modulation signal p1 first phase p2 second phase Sp1 transmit signal with first phase SP2 Receive signal with second phase Ga, Gb Integration node d object distance q charge M Sample A figure

Claims

[1] Calibration device (50) for calibrating optical properties of a camera system, in particular a time-of-flight camera system (1), comprising an imaging unit (52) which provides a planar reference surface (54) on one side and a calibration unit (60) for calibrating the optical properties of the camera system (1) from images A of the reference surface (54) produced by means of this camera system (1), characterized by , that the imaging unit (52) for creating patterns M on the reference surface (54) has switchable light sources (56), wherein the reference surface (54) has light exit apertures (58) for the light from the light sources (56) and each light source (56) is associated with at least one light exit aperture (58). [2] Calibration device according to claim 1, wherein the switchable light sources (56) are arranged in the area of ​​the light exit openings (58) such that light from these light sources (56) can exit on the side of the reference surface (54). [3] Calibration device according to claim 1 or 2, wherein the switchable light sources (56) are arranged on the side of the imaging unit (52) opposite the planar reference surface (54). [4] Calibration device according to one of the preceding claims, wherein the arrangement of the light emission openings (58) determining the pattern M is designed such that the position of a light emission opening (58) can be uniquely determined via pattern recognition. [5] Calibration device according to one of the preceding claims, in which at least one diffuser (64) is arranged on the side of the reference surface (54). [6] Calibration device according to one of the preceding claims, wherein the light exit aperture (58) is smaller than the diameter of an exit lens of the light source (56). [7] Calibration device according to one of the preceding claims, comprising a control unit (62) for controlling the light sources (56), wherein the control unit (62) is in signal communication with the calibration unit (60). [8] Use of the calibration device (50) according to any of the preceding claims for calibrating optical properties of a camera system (1). [9] Use according to claim 8, wherein the camera system (1) to be calibrated is brought into a unique measuring position in relation to the planar reference surface (54), and the measuring position is selected such that the sensor (22) of the camera system (1) to be calibrated detects all light emission apertures (58) of the reference surface (54). [10] Method for calibrating the optical properties of a camera system (1), in particular a time-of-flight camera system, using an imaging unit (52) which provides a planar reference surface (54) on one side, wherein the optical properties of the camera system (1) are calibrated via images A of the reference surface (54) of the imaging unit (52) which are created using this camera system (1), characterized by , that the imaging unit (52) has switchable light sources (56) for creating patterns M on the reference surface (54) and that the patterns M created by the light of the light sources (56) are used for calibration.

Images