Diffractive optical element and method and device for calibrating a camera

Abstract

A diffractive optical element (104) for calibrating a camera (102) has a microstructure shaped to diffract a monochromatic plane wave (108) incident on the diffractive optical element (104) into multiple diffraction orders with different but known propagation directions surrounding a zero order, wherein each of the diffraction orders can be projected as a point of a diffraction pattern onto a camera sensor (112) of the camera (102).

Description

Prior ArtThe invention is based on an apparatus or a method according to the preamble of the independent claims. The present invention also relates to a computer program.Cameras used in vehicles, for example, must be calibrated.US3912395A describes geometric camera calibration using a diffractive optical element (DOE).Disclosure of the InventionAgainst this background, the approach presented here is used to present a diffractive optical element and a method and a device for calibrating a camera, and finally a corresponding computer program according to the main claims. The measures listed in the dependent claims allow advantageous refinements and improvements of the device specified in the independent claim.The diffractive optical element has a microstructure with which it is possible to project a diffraction pattern which advantageously enables robust and fully automatic marker matching. The marker matching thereby enables an exact assignment between pixel coordinate or detected image point and diffraction order and thus object angle. In order to enable a camera calibration, each marking in the image, i.e. each point of a diffraction pattern, can be matched to the diffraction angle assigned to the respective marking by means of the marker matching. Advantageously, automatic matching of all markers is easily feasible when coded markers are used. Coded markers can be generated using a suitably shaped diffractive optical element.A corresponding diffractive optical element for calibrating a camera has a microstructure which is shaped to diffract a monochromatic planar wave incident on the diffractive optical element into a plurality of diffraction orders having different but known propagation directions, wherein the diffraction orders surround a zero order, wherein each of the diffraction orders can be projected as a point of a diffraction pattern onto a camera sensor of the camera.The camera can be an image capturing device used, for example, for capturing an interior or a surrounding area of a vehicle. Alternatively, the camera can be used in other fields of application, for example as a surveillance camera on a building. The camera may include the camera sensor and an optics. Optical properties on cameras which are identical per se can deviate from one another, for example, on account of manufacturing tolerances. Such deviations can be detected by calibrating the camera and compensated using the calibration rule. For example, the calibration rule can be used to ensure a correct translation of pixel coordinates of an image recorded by the camera into view angles of the real world, and vice versa. Known procedures, for example geometric camera calibration, can be used to determine the calibration rule. The diffractive optical element can have a predetermined microstructure, by means of which the wave incident on the diffractive optical element can be diffracted in a plurality of diffraction orders, as a result of which the diffraction pattern, for example a matrix of points, also referred to as markers, can arise. Due to the optical properties of the camera, the diffraction pattern captured by the camera typically deviates from a known ideal reference pattern assigned to the microstructure of the diffractive optical element. By matching the captured diffraction pattern with the known reference pattern, the calibration rule can be determined. An assignment of points of the diffraction pattern to points of the reference pattern can be facilitated in that the diffraction efficiencies of the orders of the diffraction pattern can or should be different.In the diffractive optical element, the microstructure can be shaped in order to diffract the planar wave with respect to intensities and / or angular spacings of the diffraction orders with different but known propagation directions. If the diffraction orders differ with respect to their intensities, the points of the diffraction pattern detected by the camera sensor can likewise have different intensities. Using a suitable image evaluation, points with prominent intensities can be easily recognized and selected. If the diffraction orders differ with respect to their angular distances, the points of the diffraction pattern detected by the camera sensor can likewise have different distances. Using a suitable image evaluation, points with marked distances from one another can be easily recognized and selected.The microstructure can be shaped to diffract the monochromatic planar wave incident on the diffractive optical element into a first group of diffraction orders which have a first intensity and into a second group of diffraction orders which have a second intensity different from the first intensity. If it is known which of the diffraction orders of the first group are assigned and which of the diffraction orders of the second group are assigned, points of the diffraction pattern assigned to the respective groups can easily be distinguished from one another.The first group can comprise between 2 and 20, in particular between 2 and 10, diffraction orders. For example, the first group can comprise four diffraction orders. The second group can comprise the remaining diffraction orders. For example, the second group can comprise at least 90% of all diffraction orders. Since only a few diffraction orders have an intensity differing from the other diffraction orders, points assigned to these special diffraction orders can be easily identified in the diffraction pattern.The diffraction orders of the first group can be arranged distributed around the zero order. This can facilitate identification of a zero point of the diffraction pattern associated with the zero order and detection of an orientation of the diffraction pattern.For example, the diffraction orders of the first group may be projectable as points of the diffraction pattern that are arranged on a quadrilateral enclosing a zero point of the diffraction pattern that is projectable by the zero order. A square can be, for example, a square, a rectangle, a diamond or a trapezoid. Such a pattern can be easily recognized in an image. Geometric shapes that are not 90° rotationally symmetrical can advantageously be used.The diffraction orders of the first group can have a greater intensity, for example an intensity that is greater by 50% than the diffraction orders of the second group. A greater intensity can mean that a greater radiation component of the wave is emitted in a direction assigned to the corresponding diffraction order than in a direction assigned to a diffraction order with a lower intensity. A difference between the intensities of the diffraction orders of the first group and the intensities of the diffraction orders of the second group can be selected such that the projected points of the diffraction pattern likewise have different intensities, which can be easily distinguished from one another using a threshold value.Alternatively, the diffraction orders of the first group can have a lower intensity, for example an intensity which is lower by 50% than the diffraction orders of the second group. The low intensity may be near or equal to zero, according to an embodiment. As a result, a more uniform intensity distribution can be achieved in comparison with the use of a greater intensity of the diffraction orders of the first group.According to one embodiment, the diffraction orders can be projectable as points of a grating of the diffraction pattern spanned by a first axis and a second axis. The axes can be oriented perpendicular or oblique to one another. The distances of adjacent points along the first axis may be different from second distances of adjacent points along the second axis. In this way, an alignment of a detected diffraction pattern can be easily recognized.In order to recognize the alignment of the captured diffraction pattern, according to one embodiment, an exact assignment between pixel coordinates of detected pixels and corresponding diffraction orders and thus corresponding object angles is carried out. This is also referred to as marker matching. If the assignment of image point to diffraction order is known, the correct intrinsics can be determined.A corresponding method for calibrating a camera comprises the following steps:providing said diffractive optical element and said camera;applying a monochromatic plane wave to the diffractive optical element in order to diffract the monochromatic plane wave into a plurality of diffraction orders having different but known propagation directions, wherein the diffraction orders surround a zero order, wherein each of the diffraction orders can be projected as a point of a diffraction pattern;detecting the diffraction pattern using a camera sensor of the camera; anddetermining a calibration rule for calibrating the camera using the diffraction pattern.In the determination step, an exact assignment between the coordinates of the points of the diffraction pattern and the diffraction orders respectively corresponding to the points can first be carried out. The calibration rule can be determined using the exact assignment according to known methods.The named diffractive optical element can advantageously be used in conjunction with a method and a device for calibrating a camera. In this case, the diffractive optical element can be used, for example, instead of a diffractive optical element used previously for geometric camera calibration.The step of determining can thereby comprise a step of finding a zero point of the diffraction pattern, a step of selecting a set of points of the diffraction pattern surrounding the zero point, a step of matching the set of points with corresponding reference points of a reference pattern assigned to the diffractive optical element, a further step of selecting a further set of points of the diffraction pattern surrounding the set of points, and a further step of matching the further set of points with corresponding further reference points of the reference pattern. Such a procedure may be advantageous in the case of large deviations of the angle of rotation or the focal length during the camera production. The steps of further selection and further matching can be carried out repeatedly in order to respectively select a further set of points of the diffraction pattern surrounding the preceding set of points and to match it with corresponding further reference points of the reference pattern.This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control device.The approach presented here furthermore creates a device which is designed to carry out, actuate or implement the steps of a variant of a method presented here in corresponding devices. This embodiment variant of the invention in the form of a device also enables the object on which the invention is based to be achieved quickly and efficiently.For this purpose, the device can have at least one arithmetic unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or an actuator for reading in sensor signals from the sensor or for outputting data or control signals to the actuator, and / or at least one communication interface for reading in or outputting data which are embedded in a communication protocol. The computing unit can be, for example, a signal processor, a microcontroller or the like, wherein the memory unit can be a flash memory or a magnetic memory unit. The communication interface can be designed to read in or output data wirelessly and / or in a wired manner, wherein a communication interface that can read in or output wired data can read in this data, for example electrically or optically, from a corresponding data transmission line or output it into a corresponding data transmission line.In the present case, a device can be understood to mean an electrical device which processes sensor signals and outputs control and / or data signals as a function thereof. The device can have an interface which can be designed as hardware and / or software. In the case of a hardware configuration, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible for the interfaces to be dedicated, integrated circuits or to consist at least partially of discrete components. In the case of a software configuration, the interfaces can be software modules which are present, for example, on a microcontroller in addition to other software modules.A computer program product or computer program with program code which can be stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical memory and is used for carrying out, implementing and / or controlling the steps of the method according to one of the embodiments described above is also advantageous, in particular if the program product or program is executed on a computer or a device.Exemplary embodiments of the approach presented here are illustrated in the drawings and explained in more detail in the following description. It shows: FIG. 1 shows a schematic illustration of an exemplary embodiment of a device for calibrating a camera; FIG. 2 shows a simulation of an exemplary embodiment of a diffraction pattern captured with a camera; FIG. 3 shows an embodiment of a diffraction pattern with detectable zero order and four emphasized orders; FIG. 4 shows an embodiment of a diffraction pattern with detectable zero order and four vacancies; FIG. 5 shows an embodiment of a diffraction pattern with different angular distances in the x- and y-directions; and FIG. 6 shows a flow diagram of a method according to an exemplary embodiment.In the following description of advantageous exemplary embodiments of the present invention, the same or similar reference numerals are used for the elements shown in the different figures and acting in a similar manner, wherein a repeated description of these elements is omitted.FIG. 1 shows a schematic illustration of an exemplary embodiment of a device 100 for calibrating a camera 102. The apparatus 100 comprises a diffractive optical element 104 arranged in front of the camera 102. The diffractive optical element 104 is configured to generate a projection 106 of a diffraction pattern when a monochromatic plane wave 108 is applied to the diffractive optical element 104. The projection 106 is composed of a plurality of diffracted beams. For example, monochromatic plane wave 108 is shown as a laser beam emitted by laser 110. The laser 110 or other suitable radiation source is optionally part of the apparatus 100. The monochromatic plane wave 108 impinges on a surface of the diffractive optical element 104 facing away from the camera 102. The diffraction pattern projected using diffractive optical element 104 is detected by a camera sensor 112 of camera 102.Merely by way of example, the camera sensor 112 is oriented parallel to the diffractive optical element 104 in the exemplary embodiment illustrated in FIG. 1. A propagation direction of the monochromatic planar wave 108 follows here, by way of example, an optical axis of the camera 102.According to one exemplary embodiment, the apparatus 100 comprises a determination device 116 which is designed to determine a calibration rule for calibrating the camera 102 using the captured diffraction patterns. For example, the determination device 116 is designed to initially evaluate the detected diffraction pattern using an image processing algorithm, as is described in more detail below, for example in connection with FIG. 3. This allows image preprocessing to be carried out for robust and accurate marker recognition. As a result, predetermined markers, which have an increased or reduced intensity compared to the remaining markers of the diffraction pattern, for example, can be automatically detected and used for automatic camera calibration.In order to identify individual points of the diffraction pattern in an image captured using the camera sensor 112, it is advantageous if the diffraction pattern has an non-homogeneity. Such an non-homogeneity can consist in that, for example, some points of the diffraction pattern have an intensity differing from the other points or a characteristic distance from adjacent points. In this case, the non-homogeneity is selected according to an exemplary embodiment such that they differ from deviations of the diffraction pattern from an ideal reference pattern that typically occur on account of manufacturing deviations during the production of the camera 102.According to one exemplary embodiment, such an inhomogeneity is achieved in that a microstructure of diffractive optical element 104 used for diffracting monochromatic planar wave 108 is selected such that monochromatic planar wave 108 diffracts with different but known propagation directions, as a result of which the diffraction pattern having the inhomogeneity is generated. Each diffraction order defined by the microstructure projects a point of the diffraction pattern. Each point can be understood as a marker. A predetermined assignment of different intensities and / or angular distances of the diffraction orders can be achieved by the microstructure.The described approach thus enables a robust, fully automatic geometric camera calibration of the camera 102 with the diffractive optical element 104The camera 102 may be used in vehicles for driver assistance or automated driving systems, for example, and provide multiple functions, including detecting objects in the scene using various object detection algorithms. For many of these algorithms, the world-to-camera or camera-to-world transformation must be known, meaning that the optical parameters of the camera 102 should be known. These parameters include focal length, principal point, and optical distortion parameters, such as radial and non-radial distortion effects.The intrinsic calibration of the camera 102 and the optical distortion may be measured in the production phase of the camera 102. A space and cost saving method for camera calibration is based on the use of diffractive optical element (DOE) 104. An exemplary embodiment of a corresponding basic structure of laser 110, diffractive optical element 104 and camera 102 for the geometric camera calibration of camera 102 is shown in FIG. 1.If the monochromatic planar wave 108, e.g. a laser beam, impinges on the diffractive optical element 104, the incident beam is diffracted into a plurality of diffraction orders by the precisely manufactured microstructure of the diffractive optical element 104. On a screen in the far field, the Fraunhofer diffraction pattern is visible. By using powerful optical field simulation tools, it is possible to calculate the required microstructure for generating arbitrary diffraction patterns.For camera calibration, diffractive optical element 104 is placed directly in front of camera 102. The camera optics then functions as a Fourier lens. This results in the far field diffraction pattern being projected onto the image plane. The plane waves of each diffraction angle are projected as individual points onto the image plane as shown in Fig. 2. The pattern on the image sensor 112 is similar to a screen disposed at infinity. Of course, the diffraction pattern on the image plane, i.e., the image sensor 112, also referred to as a camera sensor, is changed by the lens distortion. This fact is used for camera calibration.Known methods can be used to perform the geometric camera calibration of the camera 102 with the aid of the diffractive optical element 104.FIG. 2 shows a simulation of an exemplary embodiment of a schematic diffraction pattern 200 captured by a camera. For example, the diffraction pattern 200 is a pattern projectable using the diffractive optical element described with reference to FIG. 1, wherein no non-homogeneity is shown in the schematic illustration.For example, the diffraction pattern 200 illustrated in FIG. 2 can be used as a reference pattern for the diffractive optical element used for calibrating the camera shown in FIG. 1.To enable camera calibration, each mark in the image, i.e. each point of the diffraction pattern 200, needs to be matched with the corresponding diffraction angle, which may be quite difficult, as the diffraction pattern 200 may be very periodic and homogeneous. Automatic matching of all markers in a single step is possible when coded markers are used.When using recognizable features in the image, the matching is usually carried out in two steps: in a first step, the features are recognized and matched. By matching these features, camera orientation and important parameters such as focal length can be estimated. Subsequently, matching of all markers is carried out with an optimized intrinsic and extrinsic parameter set.Advantageously, the diffractive optical element enables geometric camera calibration, in which the matching is carried out automatically, for example using the zero order, which is the brightest marker in most cases.Automated detection is an advantageous option for camera calibration in series production, since the cycle time is decisive and a high degree of automation is required. Automatic detection simply relying on zero order is not very robust for several reasons:Depending on the DOE design, it may be that the zero order cannot be distinguished from all other diffraction orders, as is also the case in the merely schematically illustrated diffraction pattern 200.Conversely, the intensity of the zero order can also be too bright, so that the adjacent diffraction orders cannot be recognized.Misalignment of the roll angle, i.e. a rotation about the optical axis of ~45°, may lead to a mismatch, since the orders of diffraction in the diagonal may be interpreted as horizontal and vertical orders of diffraction.A misalignment in the roll angle of ~ 90° cannot be recognized, since the diffraction order is rotationally symmetrical for 90°.Even if adjacent markers are correctly detected, the distance between them may be too small to make a reliable estimate of e.g. the roll angle or the focal length. Large deviations in these parameters have an effect above all on the outer regions of the image, where a mismatch can occur if these parameters are not corrected after the first comparison.Advantageously, the approach described here is not or not just simply based on the zero order, so that the mentioned problems can be avoided.According to the approach described here, a diffraction pattern is used with which robust and fully automatic marker matching can be carried out. According to one exemplary embodiment, the diffraction pattern is distinguished in the definition of the diffraction angles and in the distribution of the intensity of the incident beam to the zero order and all diffraction orders.FIG. 3 shows an embodiment of a diffraction pattern 300 with detectable zero order 320 and four emphasized orders 322, 323, 324, 325. The diffraction pattern 300 can be generated, for example, using the diffractive optical element described with reference to FIG. 1.The diffraction pattern 300 includes a plurality of points arranged in a lattice-like manner. The points represent images of the diffraction orders of the diffractive optical element, as captured, for example, by the camera sensor shown in FIG. 1 and can be output in the form of an image when a laser beam is applied to the diffractive optical element, for example.According to one exemplary embodiment, the microstructure of the diffractive optical element is shaped in order to diffract the monochromatic planar wave incident on the diffractive optical element into a first group of diffraction orders 322, 323, 324, 325 which have a higher intensity and into a second group of diffraction orders 327 (for the sake of clarity, only two diffraction arrangements of the second group are provided with the reference symbol 327) which have a lower intensity which is lower than the higher intensity. By way of example, the second group of diffraction orders 327 comprises all diffraction orders 322, 323, 324, 325 not included in the first group, with the exception of the zero order 320, which likewise has the higher intensity, for example.The higher intensity can be, for example, at least 25% or at least 50% higher than the lower intensity. According to one exemplary embodiment, a distance between the higher intensity and the lower intensity is selected such that an automated image evaluation enables a clear distinction between the diffraction orders 322, 323, 324, 325 with the higher intensity and the diffraction orders 327 with the low intensity.In the illustration shown in FIG. 3, the zero order 320 is represented by a zero point which is surrounded by points of further diffraction orders 327 of the second group. The emphasized orders 322, 323, 324, 325 are represented by points distributed around the zero point. Four highlighted orders 322, 323, 324, 325 are shown by way of example. By way of example, the diffraction orders 322, 323, 324, 325 form corner points of a quadrilateral enclosing the zero point. Depending on the embodiment, the points representing the diffraction orders 322, 323, 324, 325 have the same or different distances from the zero point. As an alternative to four emphasized diffraction orders 322, 323, 324, 325, another suitable number of diffraction orders can also be used. In addition, an arrangement of the emphasized diffraction orders 322, 323, 324, 325 can be chosen differently.During the design process of the diffraction pattern 300, it is possible to select individual diffraction orders 322, 323, 324, 325 in order to achieve a greater diffraction efficiency. FIG. 3 shows a DOE design which has a uniform intensity distribution in most diffraction orders, wherein the zero order 320 and selected diffraction orders 322, 323, 324, 325 have a higher intensity.This allows for mark matching with "highlighted marks" that depict the selected diffraction orders 322, 323, 324, 325. For example, the highlighted markings have been projected onto a camera sensor by the selected diffraction orders 322, 323, 324, 325 and are pictorially depicted using the camera sensor.With the aid of image processing algorithms, it is possible to reliably recognize the highlighted markings.Since the diffraction angles of the emphasized markings are known, a matching is possible. In binary DOEs, the emphasized orders 322, 323, 324, 325 are chosen symmetrically around the zero order 320, which results in 180° symmetry, while 90° rotational symmetry can be avoided. This means that misalignment by any angle between 0° and 90° about the optical axis can be detected.The number of diffraction orders between the zero order 320 and the emphasized orders 322, 323, 324, 325 can be freely selected.This distance enables a comparatively robust first estimate of the camera alignment, including the angle of inclination, yaw and roll, as well as the focal length, which in a second step results in a more robust assignment of all diffraction orders.Even if the intensity distribution is to be uniform over all diffraction orders, some diffraction orders will be brighter than others due to production tolerances. According to an embodiment, the brightest diffraction orders are selected and a first matching step is performed with these orders. This makes it possible to avoid producing a completely new DOE.Alternatively, in this and other embodiments, the reduced intensity may also be assigned to the zero order 320 instead of the high intensity.FIG. 4 shows an embodiment of a diffraction pattern 400 with detectable zero order 320 and four vacancies which are assigned to four diffraction orders 422, 423, 424, 425 with reduced intensity.In contrast to the diffraction pattern shown in FIG. 3, the microstructure of the diffractive optical element on which the diffraction pattern 400 shown in FIG. 4 is based is shaped in order to diffract the monochromatic planar wave incident on the diffractive optical element into a first group of diffraction orders 422, 423, 424, 425, which have a reduced intensity, and into a second group of diffraction orders 327, which have a low intensity but which is higher than the reduced intensity. By way of example, the second group of diffraction orders 327 comprises all diffraction orders 422, 423, 424, 425 not comprised by the first group, with the exception of the zero order 320, which for example has a high intensity that is higher than the low intensity.The high intensity can be higher than the low intensity by at least 25% or by at least 50%, for example. The low intensity can be, for example, at least 25% or at least 50% higher than the reduced intensity. For example, the reduced intensity may be close to zero. According to one exemplary embodiment, a distance between the high intensity and the low intensity and a distance between the low intensity and the reduced intensity are selected such that an automated image evaluation enables a clear distinction between the diffraction orders 422, 423, 424, 425 with the reduced intensity and the diffraction orders with the low intensity and enables a clear distinction between the diffraction orders with the low intensity and the zero order with the high intensity.The use of the reduced intensity allows for marker matching with "open spots".This is advantageous, for example, if a uniform intensity distribution is required. In this case, the selected diffraction orders 422, 423, 424, 425 are designed to have the reduced intensity, for example an intensity close to zero. Such a void may be automatically recognized and the image coordinates of the void may be estimated from the coordinates of the adjacent markers, e.g., as their center for the linear optic. Another possibility is to use the adjacent markers of a blank directly as feature markers, which are matched in a first step.Alternatively, only some of the selected diffraction orders 422, 423, 424, 425 can be realized with the reduced intensity and the remaining ones of the selected diffraction orders 422, 423, 424, 425 can be realized with the high intensity.FIG. 5 shows an embodiment of a diffraction pattern 500 with different angular distances in the x- and y-directions.The points encompassed by the diffraction pattern 500 are arranged in a grating which is spanned by a first axis, here for example in the x direction, and a second axis, here for example in the y direction. The axes are arranged here merely by way of example orthogonally to one another.Distances of adjacent points along the first axis differ from distances of adjacent points along the second axis. For example, the distances in the x direction are greater than in the y direction.By way of example, the zero order 320 has a high intensity and the remaining diffraction orders ( 327) of the diffraction pattern 500 have a low intensity. Alternatively, in addition to FIG. 3 or FIG. 4, some selected diffraction orders may be realized with a high intensity or a reduced intensity.The different distances shown are chosen merely as an example for realizing the diffraction pattern 500 having an asymmetry. Alternatively, distances between adjacent points of the diffraction pattern 500 may be varied differently to generate a suitable asymmetry.The asymmetric diffraction pattern allows automated marker matching.The asymmetric diffraction pattern 500 may be used in a marker matching algorithm. In a first step, the zero order 320 is determined. In a second step, the distances of the surrounding orders from the zero order 320 are calculated. Since the diffraction pattern 500 has different angular pitches of the two axes, there is no 90° rotational symmetry, and the matching should be simple.In the case of large deviations of the angle of rotation or the focal length during the camera production, the adjustment is divided into smaller steps. For example, after zero order 320 is detected, the surrounding 8 markers are aligned. Because the accuracy of the focal length estimate may not be sufficient to align all of the markers of the image, as the distance between the zero order 320 and the aligned markers is quite small, a central region, e.g., 6 x 6 markers, may be selected and aligned. With this larger radius, a more robust estimation of the focal length is possible and should allow the adaptation of all markers of the diffraction pattern 500. If the deviation is still too large, a further matching step can be carried out in a larger central region, for example 10 x 10 markers.FIG. 6 shows a flow diagram of an exemplary embodiment of a method for calibrating a camera. The method can be implemented, for example, using devices of an apparatus as shown with reference to FIG. 1. In this case, a diffractive optical element can be used, with which, for example, a diffraction pattern can be projected onto the image sensor, as is illustrated with reference to FIGS. 3 to 5.According to an embodiment, the method comprises a step 601 in which a corresponding diffractive optical element and the camera to be calibrated are provided and a step 603 in which a monochromatic plane wave is applied to the diffractive optical element. Using the diffractive optical element, the monochromatic plane wave is diffracted with different but known propagation directions into a plurality of diffraction orders, which surround a zero order. Each of the diffraction orders projects a point of the diffraction pattern onto the camera sensor of the camera.In a step 605, the diffraction pattern is detected using the camera sensor of the camera. In a step 607, a calibration rule for calibrating the camera is determined using the diffraction pattern.Optionally, the step 607 comprises a step 610 of finding a zero point of the diffraction pattern, a step 612 of selecting a set of points of the diffraction pattern surrounding the zero point, a step 614 of matching the set of points with corresponding reference points of a reference pattern assigned to the diffractive optical element, a further step 616 of selecting a further set of points of the diffraction pattern surrounding the set of points, and a further step 618 of matching the further set of points with corresponding further reference points of the reference pattern.Known procedures can be used to determine the calibration rule, which procedures allow calibration based on a detected diffraction pattern. Such procedures can be supported by the non-uniformity of the diffraction pattern, since, for example, an alignment of the diffraction pattern can be reliably detected. As a result, even in the case of a rotated diffraction pattern, a correct assignment can be made between the points of the diffraction pattern and the points of the reference pattern.The method enables automatic camera calibration without the diffraction orders in the image having to be selected manually. This is very important for series production, since the cycle times must be maintained.In addition, robust camera calibration is possible even if orientation or focal length can vary during camera production.Optionally, multi-level phase DOEs are used to diffract symmetry. If the phase mask of the DOE is designed to have more than two stages, i.e. is designed as a non-binary phase mask, the diffraction pattern can be designed without symmetrical restrictions. This means that the arrangement of the diffraction order can be freely selected and the adaptation process is greatly simplified.References included in the specificationThis list of documents cited by the applicant has been produced in an automated manner and is only included for the better information of the reader. The list is not part of the German patent application or utility model application. The DPMA does not take any adhesion for any faults or omissions.Patent Literature citedUS 3912395A

[0003]

Claims

Diffractive optical element (104) for calibrating a camera (102), wherein the diffractive optical element (104) has a microstructure which is shaped to diffract a monochromatic planar wave (108) incident on the diffractive optical element (104) into a plurality of diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) having different but known propagation directions, wherein the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) surround a zero order (320), wherein each of the diffraction orders is projectable as a point of a diffraction pattern (300; 400; 500) onto a camera sensor (112) of the camera (102).Diffractive optical element (104) according to claim 1, wherein the microstructure is shaped to diffract the planar wave (108) with different but known propagation directions with respect to intensities and / or angular spacings of the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425).Diffractive optical element (104) according to one of Claims 1 to 2, in which the microstructure is shaped to diffract the monochromatic planar wave (108) incident on the diffractive optical element (104) into a first group of diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) which have a first intensity and into a second group of diffraction orders (327) which have a second intensity different from the first intensity.Diffractive optical element (104) according to claim 3, wherein the first group comprises diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) between 2 and 20, in particular between 2 and 10, diffraction orders, and / or wherein the second group comprises at least 90% of all diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425).Diffractive optical element (104) according to one of Claims 3 to 4, in which the diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) of the first group are arranged distributed around the zero order (320).Diffractive optical element (104) according to claim 5, wherein the diffraction orders (322, 323, 324, 325; 422, 423, 424, 425) of the first group are projectable as points of the diffraction pattern (300; 400) arranged on a quadrilateral enclosing a null point of the diffraction pattern (300; 400) projectable by the null order (320).Diffractive optical element (104) according to one of Claims 3 to 6, in which the diffraction orders (322, 323, 324, 325) of the first group have a greater intensity, in particular an intensity which is greater by 50% than the diffraction orders (327) of the second group.Diffractive optical element (104) according to one of Claims 3 to 6, in which the diffraction orders (422, 423, 424, 425) of the first group have a lower intensity, in particular an intensity which is lower by 50% than the diffraction orders (327) of the second group.Diffractive optical element (104) according to one of the preceding claims, in which the diffraction orders (327) can be projected as points of a grating of the diffraction pattern (500) spanned by a first axis and a second axis, wherein first distances of adjacent points along the first axis differ from second distances of adjacent points along the second axis.A method for calibrating a camera (102), the method comprising the steps of: providing (601) a diffractive optical element (104) according to any one of the preceding claims and the camera (102); applying (603) a monochromatic plane wave (108) to the diffractive optical element (104) in order to diffract the monochromatic plane wave (108) into a plurality of diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) with different but known propagation directions, wherein the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) surround a zero order (320), wherein each of the diffraction orders (322, 323, 324, 325, 327; 422, 423, 424, 425) is projectable as a point of a diffraction pattern (300; 400; 500); detecting (605) the diffraction pattern (300; 400; 500) using a camera sensor (112) of the camera (102); and determining (607) a calibration specification for calibrating the camera (102) using the diffraction pattern (300; 400; 500).Method according to one of the preceding claims, wherein the step (607) of determining comprises a step (610) of finding a zero point of the diffraction pattern (300; 400; 500), a step (612) of selecting a set of points of the diffraction pattern (300; 400; 500) surrounding the zero point, a step (614) of matching the set of points with corresponding reference points of a reference pattern (200) assigned to the diffractive optical element (104), a further step (616) of selecting a further set of points of the diffraction pattern (300; 400; 500) surrounding the set of points, and a further step (618) of matching the further set of points with corresponding further reference points of the reference pattern (200).Device (100) for calibrating a camera (102), wherein the device (100) is configured to execute and / or control the steps of the method according to one of the preceding claims 10 to 11 in corresponding units.Computer program which is configured to execute and / or control the steps of the method according to one of the preceding claims 10 to 11.

Images