Diffractive optical element and method and device for calibrating a camera

Abstract

A diffractive optical element (104) for calibrating a camera (102) has a microstructure which is shaped so as to diffract a monochromatic plane wave (108) incident on the diffractive optical element (104) into a plurality of orders of diffraction which have different but known propagation directions and surround a zero order, each of the orders of diffraction being projectable as a point of a diffraction pattern onto a camera sensor (112) of the camera (102).

Description

[0001] Description

[0002] title

[0003] Diffractive optical element and method and device for calibrating a camera

[0004] State of the art

[0005] The invention is based on a device or method according to the class of the independent claims. The present invention also relates to a computer program.

[0006] Cameras used in vehicles, for example, must be calibrated.

[0007] US3912395A describes a geometric camera calibration using a diffractive optical element (DOE).

[0008] Disclosure of the invention

[0009] Against this background, the approach presented here provides a diffractive optical element, a method, and a device for calibrating a camera, as well as a corresponding computer program according to the main claims. The measures listed in the dependent claims allow advantageous refinements and improvements of the device specified in the independent claim.

[0010] The diffractive optical element has a microstructure that can project a diffraction pattern, advantageously enabling robust and fully automatic marker matching. Marker matching enables a precise assignment between pixel coordinates or detected image points and the diffraction order, and thus the object angle. To enable camera calibration, marker matching allows each marker in the image—i.e., each point of a diffraction pattern—to be aligned with the diffraction angle associated with the respective marker. Advantageously, automatic alignment of all markers is easy to perform when coded markers are used. Coded markers can be generated using a suitably shaped diffractive optical element.

[0011] A corresponding diffractive optical element for calibrating a camera has a microstructure that is shaped to diffract a monochromatic plane wave incident on the diffractive optical element into several diffraction orders with different but known propagation directions, wherein the diffraction orders surround a zero order, wherein each of the diffraction orders is projectable as a point of a diffraction pattern onto a camera sensor of the camera.

[0012] The camera can be an image capture device used, for example, to capture the interior or surroundings of a vehicle. Alternatively, the camera can be used in other applications, for example as a surveillance camera on a building. The camera can comprise the camera sensor and optics. Optical properties of essentially identical cameras can differ from one another, for example due to manufacturing tolerances. Such deviations can be detected by calibrating the camera and compensated for using the calibration rule. For example, the calibration rule can be used to ensure correct translation of pixel coordinates of an image captured by the camera into the viewing angle of the real world, and vice versa. Known procedures, such as geometric camera calibration, can be used to determine the calibration rule.The diffractive optical element can have a predetermined microstructure through which the wave incident on the diffractive optical element can be diffracted into several diffraction orders, thereby creating the diffraction pattern, for example a matrix of points, also referred to as markers. Due to the optical properties of the camera, the diffraction pattern captured by the camera typically deviates from a known, ideal reference pattern associated with the microstructure of the diffractive optical element. The calibration specification can be determined by comparing the captured diffraction pattern with the known reference pattern. Assigning points of the diffraction pattern to points of the reference pattern can be facilitated by allowing or intended different diffraction efficiencies of the orders of the diffraction pattern.

[0013] In the diffractive optical element, the microstructure can be designed to diffract the plane wave with respect to the intensities and / or angular separations of the diffraction orders with different but known propagation directions. If the diffraction orders differ in terms of their intensities, the points of the diffraction pattern captured by the camera sensor can also have different intensities. Using suitable image analysis, points with distinctive intensities can be easily detected and selected. If the diffraction orders differ in terms of their angular separations, the points of the diffraction pattern captured by the camera sensor can also have different separations. Using suitable image analysis, points with distinctive separations from one another can be easily detected and selected.

[0014] The microstructure can be shaped to diffract the monochromatic plane wave incident on the diffractive optical element into a first group of diffraction orders having a first intensity and into a second group of diffraction orders having a second intensity different from the first intensity. If it is known which of the diffraction orders are assigned to the first group and which of the diffraction orders are assigned to the second group, points of the diffraction pattern assigned to the respective groups can be easily distinguished from one another. The first group can comprise between 2 and 20, in particular between 2 and 10, diffraction orders. For example, the first group can comprise four diffraction orders. The second group can comprise the remaining diffraction orders. For example, the second group can comprise at least 90% of all diffraction orders.Since only a few diffraction orders have an intensity that differs from the other diffraction orders, points assigned to these specific diffraction orders can be easily identified in the diffraction pattern.

[0015] The diffraction orders of the first group can be distributed around the zero order. This can facilitate the identification of a zero point of the diffraction pattern associated with the zero order, as well as the detection of an orientation of the diffraction pattern.

[0016] For example, the diffraction orders of the first group can be projected as points of the diffraction pattern arranged on a quadrilateral enclosing a zero point of the diffraction pattern projectable by the zero order. A quadrilateral can be, for example, a square, a rectangle, a rhombus, or a trapezoid. Such a pattern is easily recognizable in an image. Advantageously, geometric shapes that are not rotationally symmetrical by 90° can be used.

[0017] The diffraction orders of the first group may have a greater intensity, for example, 50% greater, than the diffraction orders of the second group. A greater intensity may mean that a larger radiation component of the wave is radiated in a direction associated with the corresponding diffraction order than in a direction associated with a diffraction order with a lower intensity. A difference between the intensities of the diffraction orders of the first group and the intensities of the diffraction orders of the second group may be chosen such that the projected points of the diffraction pattern also have different intensities, which can be easily distinguished from one another using a threshold value. Alternatively, the diffraction orders of the first group may have a lower intensity, for example, 50% lower, than the diffraction orders of the second group.According to one embodiment, the low intensity may be close to or equal to zero. This allows a more uniform intensity distribution to be achieved compared to using a higher intensity of the diffraction orders of the first group.

[0018] According to one embodiment, the diffraction orders can be projected as points of a grating of the diffraction pattern spanned by a first axis and a second axis. The axes can be aligned perpendicularly or obliquely to each other. The distances between adjacent points along the first axis can differ from the second distances between adjacent points along the second axis. In this way, the orientation of a detected diffraction pattern can be easily identified.

[0019] To identify the orientation of the captured diffraction pattern, one embodiment performs a precise mapping between the pixel coordinates of detected image points and the corresponding diffraction orders, and thus the corresponding object angle. This is also referred to as marker matching. Once the mapping of image points to diffraction orders is known, the correct intrinsic value can be determined.

[0020] A corresponding procedure for calibrating a camera includes the following steps:

[0021] Providing a said diffractive optical element and the camera;

[0022] Applying a monochromatic plane wave to the diffractive optical element to diffract the monochromatic plane wave into several diffraction orders with different but known propagation directions, wherein the diffraction orders surround a zero order, wherein each of the diffraction orders is projectable as a point of a diffraction pattern; capturing the diffraction pattern using a camera sensor of the

[0023] camera; and

[0024] Determine a calibration rule for calibrating the camera using the diffraction pattern.

[0025] In the determination step, a precise mapping can first be performed between the coordinates of the points of the diffraction pattern and the diffraction orders corresponding to each point. The calibration specification can be determined using this precise mapping according to known methods.

[0026] The aforementioned diffractive optical element can advantageously be used in conjunction with a method and a device for calibrating a camera. The diffractive optical element can be used, for example, instead of a diffractive optical element previously used for geometric camera calibration.

[0027] The determining step can comprise a step of finding a zero point of the diffraction pattern, a step of selecting a set of points of the diffraction pattern surrounding the zero point, a step of matching the set of points with corresponding reference points of a reference pattern assigned to the diffractive optical element, a further step of selecting a further set of points of the diffraction pattern surrounding the set of points, and a further step of matching the further set of points with corresponding further reference points of the reference pattern. Such a procedure can be advantageous in the case of large deviations in the angle of rotation or focal length during camera production.The steps of further selection and further alignment can be performed repeatedly to select a further set of points of the diffraction pattern surrounding the previous set of points and to compare them with corresponding further reference points of the reference pattern. This method can be implemented, for example, in software or hardware, or in a hybrid form of software and hardware, for example, in a control unit.

[0028] The approach presented here further provides a device designed to perform, control, or implement the steps of a variant of a method presented here in corresponding devices. This embodiment of the invention in the form of a device also allows the problem underlying the invention to be solved quickly and efficiently.

[0029] For this purpose, the device can have at least one computing unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or an actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading or outputting data embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller, or the like, wherein the memory unit can be a flash memory or a magnetic storage unit.The communication interface can be designed to read in or output data wirelessly and / or wired, wherein a communication interface that can read in or output wired data can read this data, for example, electrically or optically from a corresponding data transmission line or output it to a corresponding data transmission line.

[0030] In this case, a device can be understood as an electrical device that processes sensor signals and outputs control and / or data signals depending on them. The device can have an interface, which can be implemented in hardware and / or software. In a hardware implementation, the interfaces can, for example, be part of a so-called system ASIC, which contains a wide variety of functions of the device. However, it is also possible for the interfaces to be separate integrated circuits or to consist at least partially of discrete components. In a software implementation, the interfaces can be software modules that are present, for example, on a microcontroller alongside other software modules.

[0031] Also advantageous is a computer program product or computer program with program code that can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used to carry out, implement and / or control the steps of the method according to one of the embodiments described above, in particular when the program product or program is executed on a computer or a device.

[0032] Examples of the approach presented here are shown in the drawings and explained in more detail in the following description. It shows:

[0033] Fig. 1 is a schematic representation of an embodiment of a device for calibrating a camera;

[0034] Fig. 2 shows a simulation of an embodiment of a diffraction pattern captured by a camera;

[0035] Fig. 3 shows an embodiment of a diffraction pattern with detectable zero order and four highlighted orders;

[0036] Fig. 4 shows an embodiment of a diffraction pattern with detectable zero order and four vacancies;

[0037] Fig. 5 shows an embodiment of a diffraction pattern with different angular distances in the x- and y-direction; and

[0038] Fig. 6 shows a flowchart of a method according to one embodiment. In the following description of advantageous embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the various figures and having a similar effect, whereby a repeated description of these elements is omitted.

[0039] Fig. 1 shows a schematic representation of an embodiment of a device 100 for calibrating a camera 102. The device 100 comprises a diffractive optical element 104 arranged in front of the camera 102. The diffractive optical element 104 is designed to generate a projection 106 of a diffraction pattern when the diffractive optical element 104 is exposed to a monochromatic plane wave 108. The projection 106 is composed of a plurality of diffracted beams. For example, the monochromatic plane wave 108 is depicted as a laser beam emitted by a laser 110. The laser 110 or another suitable radiation source is optionally part of the device 100. The monochromatic plane wave 108 impinges on a surface of the diffractive optical element 104 facing away from the camera 102.The diffraction pattern projected using the diffractive optical element 104 is captured by a camera sensor 112 of the camera 102.

[0040] For example only, the camera sensor 112 in the embodiment shown in Fig. 1 is aligned parallel to the diffractive optical element 104. A propagation direction of the monochromatic plane wave 108 follows, for example, an optical axis of the camera 102.

[0041] According to one embodiment, the device 100 comprises a determination device 116, which is designed to determine a calibration rule for calibrating the camera 102 using the detected diffraction pattern. For example, the determination device 116 is designed to first evaluate the detected diffraction pattern using an image processing algorithm, as described in more detail below, for example, in connection with Fig. 3. This allows image preprocessing for robust and accurate marker detection. This allows predetermined markers, which, for example, have an increased or decreased intensity compared to the other markers of the diffraction pattern, to be automatically detected and used for automatic camera calibration.

[0042] In order to identify individual points of the diffraction pattern in an image captured using the camera sensor 112, it is advantageous if the diffraction pattern exhibits an inhomogeneity. Such an inhomogeneity can consist, for example, in some points of the diffraction pattern having an intensity that differs from the remaining points or a characteristic distance from neighboring points. According to one embodiment, the inhomogeneity is selected such that it differs from deviations of the diffraction pattern from an ideal reference pattern that typically occur due to manufacturing deviations during the manufacture of the camera 102.

[0043] According to one embodiment, such inhomogeneity is achieved by selecting a microstructure of the diffractive optical element 104 used to diffract the monochromatic plane wave 108 such that the monochromatic plane wave 108 is diffracted with different but known propagation directions, thereby generating the diffraction pattern exhibiting the inhomogeneity. Each diffraction order defined by the microstructure projects a point of the diffraction pattern. Each point can be considered a marker. The microstructure can achieve a predetermined assignment of different intensities and / or angular separations of the diffraction orders.

[0044] The described approach thus enables a robust, fully automatic geometric camera calibration of the camera 102 with the diffractive optical element 104

[0045] The camera 102 can be used, for example, in vehicles for driver assistance or automated driving systems and can provide several functions, including the detection of objects in the scene using various object detection algorithms. For many of these algorithms, the world-to-camera or camera-to-world transformation must be known, which means that the optical parameters of the camera 102 should be known. These parameters include the focal length, the principal point, and optical distortion parameters, such as radial and non-radial distortion effects.

[0046] The intrinsic calibration of camera 102 and the optical distortion can be measured during the production phase of camera 102. A space- and cost-saving method for camera calibration is based on the use of the diffractive optical element (DOE) 104. An embodiment of a corresponding basic configuration of laser 110, diffractive optical element 104, and camera 102 for the geometric camera calibration of camera 102 is shown in Fig. 1.

[0047] When the monochromatic plane wave 108, e.g., a laser beam, strikes the diffractive optical element 104, the incident beam is diffracted into several diffraction orders by the precisely manufactured microstructure of the diffractive optical element 104. The Fraunhofer diffraction pattern is visible on a screen in the far field. By using powerful optical field simulation tools, it is possible to calculate the required microstructure to generate any desired diffraction pattern.

[0048] For camera calibration, the diffractive optical element 104 is placed directly in front of the camera 102. The camera optics then function as a Fourier lens. This results in the far-field diffraction pattern being projected onto the image plane. The plane waves of each diffraction angle are projected onto the image plane as individual points, as shown in Fig. 2. The pattern on the image sensor 112 is similar to a screen arranged at infinity. Of course, the diffraction pattern on the image plane, i.e., the image sensor 112, also referred to as the camera sensor, is altered by lens distortion. This fact is used for camera calibration.

[0049] Known methods can be used to perform the geometric camera calibration of the camera 102 using the diffractive optical element 104. Fig. 2 shows a simulation of an embodiment of a schematic diffraction pattern 200 captured with a camera. For example, the diffraction pattern 200 is a pattern projectable using the diffractive optical element described with reference to Fig. 1, wherein no inhomogeneity is shown in the schematic representation.

[0050] For example, the diffraction pattern 200 shown in Fig. 2 can be used as a reference pattern for the diffractive optical element used to calibrate the camera shown in Fig. 1.

[0051] To enable camera calibration, each marker in the image—that is, each point of the diffraction pattern 200—must be aligned with the corresponding diffraction angle. This can be quite difficult because the diffraction pattern 200 can be highly periodic and homogeneous. Automatic alignment of all markers in a single step is possible when using coded markers.

[0052] When using recognizable features in the image, the matching is usually performed in two steps: In a first step, the features are detected and matched. By matching these features, the camera orientation and important parameters such as the focal length can be estimated. Subsequently, all markers are matched using an optimized set of intrinsic and extrinsic parameters.

[0053] Advantageously, the diffractive optical element enables geometric camera calibration, in which the adjustment can be carried out automatically, for example using the zero order, which in most cases is the brightest marker.

[0054] Automated detection is an advantageous option for camera calibration in series production, as cycle time is critical and a high degree of automation is required. Automatic detection that simply relies on zero order is not very robust for several reasons: Depending on the DOE design, zero order may not be distinguishable from all other diffraction orders, as is the case in the merely schematically shown diffraction pattern 200.

[0055] Conversely, the intensity of the zero order may be too bright so that the neighboring diffraction orders cannot be detected.

[0056] A misalignment of the roll angle, i.e. a rotation around the optical axis, of ~ 45° can lead to a mismatch, since the diffraction orders in the diagonal can be interpreted as horizontal and vertical diffraction orders.

[0057] A misalignment in the roll angle of ~ 90° cannot be detected because the diffraction order for 90° is rotationally symmetric.

[0058] Even if neighboring markers are correctly detected, the distance between them may be too small to reliably estimate parameters such as roll angle or focal length. Large deviations in these parameters primarily affect the outer regions of the image, where mismatches can occur if these parameters are not corrected after the initial alignment.

[0059] Advantageously, the approach described here does not rely, or does not rely only on zero order, so that the problems mentioned can be avoided.

[0060] According to the approach described here, a diffraction pattern is used that allows for robust and fully automated marker matching. According to one exemplary embodiment, the diffraction pattern is characterized by the definition of the diffraction angles and the distribution of the intensity of the incident beam across the zero order and all diffraction orders.

[0061] Fig. 3 shows an embodiment of a diffraction pattern 300 with detectable zero order 320 and four highlighted orders 322, 323, 324, 325. The diffraction pattern 300 can be generated, for example, using the diffractive optical element described with reference to Fig. 1.

[0062] The diffraction pattern 300 comprises a plurality of dots arranged in a grid. The dots represent images of the diffraction orders of the diffractive optical element, as they can be captured, for example, by the camera sensor shown in Fig. 1 and output in the form of an image when the diffractive optical element is exposed to a laser beam, for example.

[0063] According to one embodiment, the microstructure of the diffractive optical element is configured to diffract the monochromatic plane wave incident on the diffractive optical element into a first group of diffraction orders 322, 323, 324, 325, which have a higher intensity, and into a second group of diffraction orders 327 (for the sake of clarity, only two diffraction orders of the second group are provided with the reference numeral 327) which have a lower intensity, which is lower than the higher intensity. By way of example, the second group of diffraction orders 327 includes all diffraction orders 322, 323, 324, 325 not included in the first group, with the exception of the zero order 320, which, by way of example, also has the higher intensity.

[0064] The higher intensity can, for example, be at least 25% or at least 50% higher than the lower intensity. According to one embodiment, a distance between the higher intensity and the lower intensity is selected such that automated image analysis enables a clear distinction between the diffraction orders 322, 323, 324, and 325 with the higher intensity and the diffraction orders 327 with the lower intensity.

[0065] In the illustration shown in Fig. 3, the zero order 320 is represented by a zero point surrounded by points of further diffraction orders 327 of the second group. The highlighted orders 322, 323, 324, 325 are represented by points distributed around the zero point. By way of example, four highlighted orders 322, 323, 324, 325 are shown. By way of example, the diffraction orders 322, 323, 324, 325 form corner points of a quadrilateral enclosing the zero point. Depending on the embodiment, the points representing the diffraction orders 322, 323, 324, 325 have the same or different distances from the zero point. As an alternative to four highlighted diffraction orders 322, 323, 324, 325, another suitable number of diffraction orders can also be used. In addition, an arrangement of the highlighted diffraction orders 322, 323, 324, 325 can be chosen differently.

[0066] During the design process of the diffraction pattern 300, it is possible to select individual diffraction orders 322, 323, 324, 325 to achieve greater diffraction efficiency. Figure 3 shows a DOE design that exhibits a uniform intensity distribution in most diffraction orders, with the zero order 320 and selected diffraction orders 322, 323, 324, 325 exhibiting higher intensity.

[0067] This enables marker alignment with "highlighted markers" that represent the selected diffraction orders 322, 323, 324, 325. For example, the highlighted markers are projected onto a camera sensor by the selected diffraction orders 322, 323, 324, 325 and imaged using the camera sensor.

[0068] With the help of image processing algorithms, it is possible to reliably detect the highlighted markings.

[0069] Since the diffraction angles of the highlighted markers are known, alignment is possible. For binary DOEs, the highlighted orders 322, 323, 324, and 325 are chosen symmetrically around the zero order 320, resulting in 180° symmetry while avoiding 90° rotational symmetry. This means that misalignment at any angle between 0° and 90° around the optical axis can be detected.

[0070] The number of diffraction orders between the zero order 320 and the highlighted orders 322, 323, 324, and 325 can be freely chosen. This distance allows for a comparatively robust initial estimation of the camera orientation, including the pitch, yaw, and roll angles, as well as the focal length, leading to a more robust assignment of all diffraction orders in a second step.

[0071] Even if the intensity distribution is intended to be uniform across all diffraction orders, some diffraction orders will be brighter than others due to manufacturing tolerances. According to one embodiment, the brightest diffraction orders are selected, and a first adjustment step is performed with these orders. This avoids the need to manufacture a completely new DOE.

[0072] Alternatively, in this and other embodiments, the zero order 320 may also be assigned the reduced intensity instead of the high intensity.

[0073] Fig. 4 shows an embodiment of a diffraction pattern 400 with detectable zero order 320 and four vacancies assigned to four diffraction orders 422, 423, 424, 425 with reduced intensity.

[0074] In contrast to the diffraction pattern shown in Fig. 3, the microstructure of the diffractive optical element underlying the diffraction pattern 400 shown in Fig. 4 is shaped to diffract the monochromatic plane wave incident on the diffractive optical element into a first group of diffraction orders 422, 423, 424, 425, which have a reduced intensity, and into a second group of diffraction orders 327, which have a low intensity but are higher than the reduced intensity. By way of example, the second group of diffraction orders 327 includes all diffraction orders 422, 423, 424, 425 not included in the first group, with the exception of the zero order 320, which, by way of example, has a high intensity that is higher than the low intensity.

[0075] The high intensity can, for example, be at least 25% or at least 50% higher than the low intensity. The low intensity can, for example, be at least 25% or at least 50% higher than the reduced intensity. For example, the reduced intensity can be close to zero. According to one embodiment, a distance between the high intensity and the low intensity and a distance between the low intensity and the reduced intensity is selected such that automated image analysis enables a clear distinction between the diffraction orders 422, 423, 424, 425 with the reduced intensity and the diffraction orders with the low intensity, as well as a clear distinction between the diffraction orders with the low intensity and the zero order with the high intensity.

[0076] The use of reduced intensity allows marker matching with “open spots”.

[0077] This is advantageous, for example, when a uniform intensity distribution is required. In this case, the selected diffraction orders

[0078] 422, 423, 424, 425 are designed to exhibit reduced intensity, for example, an intensity close to zero. Such a vacancy can be detected automatically, and the image coordinates of the vacancy can be estimated based on the coordinates of the neighboring markers, e.g., as their center for linear optics. Another possibility is to use the neighboring markers of a vacancy directly as feature markers, which are then aligned in a first step.

[0079] Alternatively, only some of the selected diffraction orders 422,

[0080] 423, 424, 425 with the reduced intensity and the remaining selected diffraction orders 422, 423, 424, 425 with the high intensity.

[0081] Fig. 5 shows an embodiment of a diffraction pattern 500 with different angular distances in the x- and y-direction.

[0082] The points encompassed by the diffraction pattern 500 are arranged in a grid spanned by a first axis, here, for example, in the x-direction, and a second axis, here, for example, in the y-direction. The axes are arranged orthogonally to each other merely by way of example. Distances between neighboring points along the first axis differ from distances between neighboring points along the second axis. For example, the distances in the x-direction are greater than those in the y-direction.

[0083] For example, the zero order 320 has a high intensity, and the remaining diffraction orders (327) of the diffraction pattern 500 have a low intensity. Alternatively, in accordance with Fig. 3 or Fig. 4, some selected diffraction orders can be realized with a high intensity or a reduced intensity.

[0084] The different distances shown are chosen merely as an example for realizing the diffraction pattern 500 exhibiting an asymmetry. Alternatively, the distances between adjacent points of the diffraction pattern 500 can be varied differently to generate a suitable asymmetry.

[0085] The asymmetric diffraction pattern enables automated marker alignment.

[0086] The asymmetric diffraction pattern 500 can be used in a marker matching algorithm. In a first step, the zero order 320 is determined. In a second step, the distances of the surrounding orders to the zero order 320 are calculated. Since the diffraction pattern 500 has different angular separations between the two axes, there is no 90° rotational symmetry, and the matching should be straightforward.

[0087] For large deviations in the rotation angle or focal length during camera production, the alignment is divided into smaller steps. For example, after detecting the zero order 320, the surrounding 8 markers are aligned. Since the accuracy of the focal length estimate may not be sufficient to match all markers in the image due to the relatively small distance between the zero order 320 and the aligned markers, a central area, e.g., 6 x 6 markers, can be selected and aligned. This larger radius allows for a more robust focal length estimate and should allow the alignment of all markers in the diffraction pattern 500. If the deviation is still too large, another matching step can be performed in a larger central area, e.g., 10 x 10 markers.

[0088] Fig. 6 shows a flowchart of an embodiment of a method for calibrating a camera. The method can be implemented, for example, using devices of a device as shown in Fig. 1. A diffractive optical element can be used, with which, for example, a diffraction pattern can be projected onto the image sensor, as shown in Figs. 3 to 5.

[0089] According to one embodiment, the method comprises a step 601 in which a corresponding diffractive optical element and the camera to be calibrated are provided, and a step 603 in which the diffractive optical element is exposed to a monochromatic plane wave. Using the diffractive optical element, the monochromatic plane wave with different but known propagation directions is diffracted into several diffraction orders surrounding a zero order. Each of the diffraction orders projects a point of the diffraction pattern onto the camera sensor.

[0090] In step 605, the diffraction pattern is captured using the camera's sensor. In step 607, a calibration rule for calibrating the camera is determined using the diffraction pattern.

[0091] Optionally, step 607 comprises a step 610 of finding a zero point of the diffraction pattern, a step 612 of selecting a set of points of the diffraction pattern surrounding the zero point, a step 614 of matching the set of points with corresponding reference points of a reference pattern assigned to the diffractive optical element, a further step 616 of selecting a further set of points of the diffraction pattern surrounding the set of points, and a further step 618 of matching the further set of points with corresponding further reference points of the reference pattern.

[0092] To determine the calibration specification, known procedures can be used that allow calibration based on a recorded diffraction pattern. The inhomogeneity of the diffraction pattern can support such procedures, as, for example, an orientation of the diffraction pattern can be reliably detected. This allows for a correct mapping between the points of the diffraction pattern and the points of the reference pattern, even with a rotated diffraction pattern.

[0093] The process enables automatic camera calibration without the need to manually select the diffraction orders in the image. This is very important for series production, as cycle times must be adhered to.

[0094] In addition, robust camera calibration is possible even if orientation or focal length may vary during camera production.

[0095] Multi-level phase DOEs can optionally be used to diffract the symmetry. If the phase mask of the DOE is designed to have more than two levels, i.e., as a non-binary phase mask, the diffraction pattern can be designed without symmetric constraints. This means that the arrangement of the diffraction order can be freely chosen, greatly simplifying the matching process.

Claims

Claims 1. Diffractive optical element (104) for calibrating a camera (102), wherein the diffractive optical element (104) has a microstructure which is shaped to diffract a monochromatic plane wave (108) incident on the diffractive optical element (104) into a plurality of diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) with different but known propagation directions, wherein the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) surround a zero order (320), wherein each of the diffraction orders is projected as a point of a diffraction pattern (300; 400; 500) onto a camera sensor (112) of the camera (102) is projectable 2. Diffractive optical element (104) according to claim 1, wherein the microstructure is shaped to diffract the plane wave (108) with respect to intensities and / or angular spacings of the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) with different but known propagation directions.

3. Diffractive optical element (104) according to one of claims 1 to 2, wherein the microstructure is shaped to diffract the monochromatic plane wave (108) incident on the diffractive optical element (104) into a first group of diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) having a first intensity and into a second group of diffraction orders (327) having a second intensity different from the first intensity.

4. Diffractive optical element (104) according to claim 3, wherein the first group comprises between 2 and 20, in particular between 2 and 10 diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) and / or wherein the second group comprises at least 90% of all diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425).

5. Diffractive optical element (104) according to one of claims 3 to 4, wherein the diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) of the first group are arranged distributed around the zero order (320).

6. Diffractive optical element (104) according to claim 5, wherein the diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) of the first group are projectable as points of the diffraction pattern (300; 400) arranged on a quadrilateral enclosing a zero point of the diffraction pattern (300; 400) projectable by the zero order (320).

7. Diffractive optical element (104) according to one of claims 3 to 6, wherein the diffraction orders (322, 323, 324, 325) of the first group have a greater intensity, in particular a 50% greater intensity, than the diffraction orders (327) of the second group.

8. Diffractive optical element (104) according to one of claims 3 to 6, wherein the diffraction orders (422, 423, 424, 425) of the first group have a lower intensity, in particular an intensity 50% lower, than the diffraction orders (327) of the second group.

9. Diffractive optical element (104) according to one of the preceding claims, wherein the diffraction orders (327) are projectable as points of a grating of the diffraction pattern (500) spanned by a first axis and a second axis, wherein first distances between adjacent points along the first axis differ from second distances between adjacent points along the second axis.

10. A method for calibrating a camera (102), the method comprising the following steps: Providing (601) a diffractive optical element (104) according to one of the preceding claims and the camera (102); Applying (603) a monochromatic plane wave (108) to the diffractive optical element (104) in order to diffract the monochromatic plane wave (108) into a plurality of diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) with different but known propagation directions, wherein the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) surround a zero order (320), wherein each of the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) is represented as a point of a diffraction pattern (300; 400; 500) is projectable; Capturing (605) the diffraction pattern (300; 400; 500) using a camera sensor (112) of the camera (102); and Determining (607) a calibration rule for calibrating the camera (102) using the diffraction pattern (300; 400; 500).

11. Method according to one of the preceding claims, wherein the step (607) of determining comprises a step (610) of finding a zero point of the diffraction pattern (300; 400; 500), a step (612) of selecting a set of points of the diffraction pattern (300; 400; 500) surrounding the zero point, a step (614) of matching the set of points with corresponding reference points of a reference pattern (200) assigned to the diffractive optical element (104), a further step (616) of selecting a further set of points of the diffraction pattern (300; 400; 500) surrounding the set of points, and a further step (618) of matching the further set of points with corresponding further reference points of the reference pattern (200).

12. Device (100) for calibrating a camera (102), wherein the device (100) is configured to carry out and / or control the steps of the method according to one of the preceding claims 10 to 11 in corresponding units.

13. Computer program configured to execute and / or control the steps of the method according to one of the preceding claims 10 to 11.

Images