Methods for 3D measurement of objects

Abstract

Method for 3D measurement of objects (1) using a stereo system, In a device for the location-different detection and three-dimensional evaluation of stereo image sequences, a statistical optical pattern (3) is projected onto the object (1) via a deflecting mirror (5), wherein the deflecting mirror (5) is attached to a motor (6) such that an axis (7) of the motor (6) intersects a plane of the deflecting mirror (5) almost, but not quite, perpendicularly, and wherein the deflecting mirror (5) is set in motion by rotation of the motor (6), and a wobbling motion of the deflecting mirror (5) is achieved by the slight tilting of the deflecting mirror plane normal to the axis (7) of the motor (6). wherein the statistical optical pattern (3) mapped onto the object (1) is modified in any way with respect to position and / or shape without a defined pattern sequence and / or without an exact pattern position, wherein, at an arbitrary recording rate, stereo image sequences from different locations are recorded by cameras (8, 9) synchronized with each other in a camera system, which have been pre-calibrated with respect to the internal and external parameters of the stereo system, and are transmitted to a computer, and during the evaluation of the stereo image sequences detected from different locations, homologous image points are assigned to each other from the stereo image sequences and spatial information is calculated from these, where, during the computational evaluation of the stereo image sequences detected at different locations, the homologous image points are assigned to each other using a time correlation method, and 3D points in space are determined from these using the calibration parameters.

Description

[0001] The invention relates to a method for the fastest possible and highly accurate 3D measurement of objects, in which statistical patterns are projected onto the object to be measured. These patterns are detected as corresponding image patterns of the object from different viewpoints, for example by cameras. Spatial information for the three-dimensional reconstruction of the object is obtained from the comparison of these different image patterns.

[0002] In many fields, fast-measuring optical 3D measurement systems are required. Optical methods are already used for analyzing airbag deployments, damage analysis of accident scenarios, and vehicle crashes. However, these methods track only a few target points and thus 3D points in the scene, or, with dense measurement methods, only very imprecise 3D data can be obtained. For quality control of industrial goods in assembly line production, a high measurement rate and tolerance to object movements are crucial. High-precision 3D measurement methods could not previously be used for these tasks because the required short measurement times were technically unfeasible. For medical purposes, measuring moving body parts is helpful for diagnosing misalignments.In sports science, the analysis of body part and / or individual movement can be used to optimize movement sequences. However, until now, only target markers could be used, resulting in simplified models that could only be fed with data. The same problem exists with the digitization of moving scenes for multimedia use, whether the movement of actors or moving objects. Particularly with the increasing prevalence of 3D television, 3D digitization will gain importance in the near future, thereby increasing the demands on the quality of 3D recordings. Furthermore, high-resolution, high-speed cameras will become available in the coming years, as the current generation of interfaces (e.g., USB 3.0, LightPeak) allows for higher frame rates (up to 1000 Hz at VGA resolution), thus significantly reducing the currently high cost of camera systems.In this context, the development of a fast and highly accurate measuring system for many application areas is therefore desirable.

[0003] Known methods exist for highly accurate (relative measurement uncertainty < 10) measurements. -4 ) and dense 3D measurement of objects using structured lighting. This includes, for example, fringe projection methods (W. Schreiber and G. Notni: Theory and arrangements of self-calibrating whole-body three-dimensional measurement systems using fringe projection technique, Optical Engineering 39, 2000, 159-169; J. Gühring, Dense 3-D surface acquisition by structured light using off-the-shelf components, video-metrics and optical methods for 3D shape measurement 4309, 2001, 220-231) or methods using statistical patterns (DE 196 23 172 C1; A. Wiegmann, H. Wagner, R. Kowarschik: Human face measurement by projecting bandlimited random patterns, Optics Express 14, 2006, 7692-7698).

[0004] German patent DE 196 23 172 C1 discloses a method in which patterns with a stochastic structure are projected onto the objects being measured, and in which two matrix cameras each record image sequences. Between each image capture, the projected pattern is shifted and / or rotated by predetermined values. Homologous pixels are assigned to each other by means of a similarity analysis of the temporal intensity profiles in the recorded image sequences, i.e., by a time correlation. Spatial coordinates are calculated from this. While this enables precise and dense three-dimensional measurement, the measurement time is significantly limited by the speed at which the projected stochastic pattern can be shifted by a predetermined value between image captures. Implementing such a method requires a digital video projector or a mechanical adjustment device with which the pattern can be shifted in a defined manner.The problem of a short measurement time, which formed the basis of our invention, cannot be solved with such a technique. The projection rate severely limits the possible recording rate of the cameras and thus also the measurement time of such a method.

[0005] Methods that meet the highest accuracy requirements need longer image sequences (between ten and 50 images per camera) to achieve the required measurement accuracy for any object, including discontinuous and separate objects. No highly accurate and dense measurement methods capable of operating at a recording and projection rate exceeding 15 Hz are known from the literature, with the projection technology being the limiting factor (M. Schaffer, M. Große, and R. Kowarschik: High-speed pattern projection for three-dimensional shape measurement using laser speckles, Applied Optics 49(18), 2010, 3622-3629; S. Zhang: Recent progresses on real-time 3d shape measurement using digital fringe projection techniques, Optics and Lasers in Engineering 48, 2010, 149-158).

[0006] Methods for precise 3D measurement are also known (relative measurement uncertainty 10). -3 up to 10 -4), which allow for dense reconstructions with sequence lengths of five to twenty images. Projection rates of up to 180 Hz were achieved here using specially adapted hardware (S. König and S. Gumhold: Image-based motion compensation for structured light scanning of dynamic surfaces, EG Workshop on Dynamic 3D Imaging, 2007; Z. Wang, H. Du, S. Park and H. Xie: Three-dimensional shape measurement with a fast and accurate approach, Appl. Opt. 48(6), 2009, 1052-1061), since losses in the quality of the pattern structure are tolerable given the aforementioned relative uncertainty.

[0007] German patent application DE 2006 001 634 B3 discloses a method and a device for creating a distance image, in which a statistical optical pattern is projected onto an object and moved translationally or rotationally on the object by a projection device. The measurement principle is based on a spatial correlation to which the projection and movement of the statistical pattern are assigned. This spatial correlation is limited with regard to reproducible accuracy and resolution.

[0008] It is also known to project periodic (striped) patterns onto the object to be evaluated three-dimensionally and to shift them in a defined manner (for example, US 6,700,669 B1). In practice, however, the assignment between camera and projector is not unambiguous, especially when using periodic patterns. As a result, discontinuous or spatially separated objects, or objects with a large depth, cannot be successfully measured. Furthermore, the acquisition rates are limited by the defined mechanical pattern shifts.

[0009] German patent DE 10 2009 040 981 A1 presents a method in which independent statistical patterns are projected onto the object as a series of optical pattern sequences. Corresponding pixels are found by comparing different pattern views. A combination of spatial and temporal correlation is intended to improve the three-dimensional reconstruction of objects and reduce the procedural effort. Since a conventional video projector is required for projecting the pattern sequences, the frame rates are also limited while maintaining high measurement accuracy.

[0010] In US 2004 / 0105580 A1, locally unique patterns (LUPs), which can also be statistical in nature, are mapped onto the objects to be evaluated. The LUP consists of several pattern layers that are projected and recorded sequentially. This also corresponds to the aforementioned sequence of pattern sequences to be detected.

[0011] WO 2005 / 010825 A2 discloses the projection of a uniquely coded pattern, which can also be statistical in nature, where the point assignment is carried out using spatial coding, i.e., spatial correlation, based on individual image pairs. For precise three-dimensional measurement, a line grid is projected in a second step, and another image pair is acquired. To solve the problem of uniqueness, the data obtained in the aforementioned first step using spatial correlation are used. In this way, a precise three-dimensional measurement is possible, but not a dense measurement. Only the object areas illuminated by the lines of the grid projected in the second step can be measured precisely.

[0012] Furthermore, more recent work on high-speed measurement is known (Y. Gong and S. Zhang: Ultrafast 3-d shape measurement with an off-the-shelf dlp projector, Optics Express 18(19), 2010, 19743-19754; Y. Wang and S. Zhang: Superfast multifrequency phaseshifting technique with optimal pulse width modulation, Optics Express 19, 2011, 5149-5155; SS Gorthi and P. Rastogi: Fringe projection techniques: Whither we are?, Optics and Lasers in Engineering 48, 2010, 133-140; J. Salvi, S. Femandez, T. Pribanic, and X. Llado: A state of the art in structured light patterns for surface profilometry. Pattern Recognition 43(8), 2010, 2666-2680), which is achieved through the use of special control software. and / or pattern generation units enable projection rates of up to 10,000 Hz.Due to the technology used, only binary images can be displayed at these projection rates, so conventional methods must be adapted or completely new methods for structured illumination developed. The relative measurement accuracies achieved so far (10. -2 up to 10 -3 However, these methods are too inaccurate for many applications, and often discontinuous objects cannot be measured in their complete form.

[0013] All described methods require various pattern structures for signaling the object surface of complex objects, making the use of digital projectors such as DMD or LCD projectors essential. Consequently, the maximum projection rate is technically limited to 255 Hz for high measurement accuracy and to 10,000 Hz for lower accuracy due to the projection of binary images. Therefore, with this current state of the art, highly accurate, dense 3D measurements with short measurement times are not yet achievable.

[0014] The invention is based on the objective of measuring the object three-dimensionally with minimal effort, as quickly as possible and with high accuracy.

[0015] This is intended to achieve high measurement accuracies (relative measurement accuracy better than 1.0*10). -4 ) very fast 3D recording rates (higher than 200 Hz, i.e. more than 200 3D recordings per second) can be achieved.

[0016] This problem is solved by a method for 3D measurement of objects, in which at least one statistical optical pattern for location-different detection and three-dimensional evaluation is projected onto the object and arbitrarily changed there in position and / or shape according to the teaching of independent claim 1. Further embodiments are defined in the dependent claims.

[0017] In a device for carrying out this method, at least one light source (constant light source or controllable pulsed light source) is provided for generating the at least one statistical optical pattern that can be detected differently depending on the location, wherein at least one element that changes the beam path is arranged in the beam path of the light source to the object.

[0018] In contrast to all methods described in the prior art, measurement accuracy is achieved using a single statistical pattern structure, the shape and / or position of which is continuously changed on the object. By eliminating the need for a defined sequence of different pattern structures, a flexible projection unit is not required. This circumvents all limitations imposed by the image composition and projection rate of conventional projection units. In particular, this means that any acquisition rate can be used, since, for example, the pattern can be moved across the object at a sufficiently high speed, thus resolving the most significant immediate problem of fast-measuring systems.Furthermore, the projection method ensures that even with high-speed measurement systems, a grayscale pattern structure is always generated, thus significantly improving the measurement accuracy of previous high-speed measurement methods using high-frequency binary images (by approximately a factor of 10). Additionally, the usual synchronization between the cameras and the projection unit is unnecessary, as no exact image sequence or pattern position needs to be maintained. Only the synchronization of the cameras themselves needs to be ensured. This increases the flexibility of possible measurement setups, as no connection or direct information exchange between the projection source and the recording devices is required.Since high-quality projection devices, such as slide projectors, can be used to project the fixed pattern, and these still offer the highest contrast range and resolution compared to other projectors, especially modern DLP projectors, the described method can also improve the measurement accuracy of slower measurement methods. Furthermore, no correction of the projection device's gamma function is necessary, as is required with digital projection devices. Additionally, no control computer or control electronics for the projection unit are needed, which further reduces the complexity of the process.

[0019] The invention will be explained in more detail below with reference to a device for fast and highly accurate 3D measurement of objects shown in the drawing as an exemplary embodiment.

[0020] The surface 2 of an object 1 is to be measured and reconstructed three-dimensionally. For this purpose, a statistical pattern from a photograph 3 in a projector 4 is projected onto the surface 2 via a deflecting mirror 5. The deflecting mirror 5 is attached to a motor 6 such that its axis 7 intersects the plane of the deflecting mirror 5 almost, but not quite, perpendicularly. Rotation of the motor 6 sets the deflecting mirror 5 in motion, and the slight tilt of the mirror plane normal to the axis 7 of the motor 6 results in a wobbling motion of the deflecting mirror 5. Due to this wobbling motion of the deflecting mirror 5, the projected image of the photograph 3 also moves in a wobbling manner across the surface 2 of the object 1 to be measured. The area that remains illuminated during a complete rotation of the mirror represents the boundary of the measurement volume.

[0021] Using two synchronized cameras 8, 9, which have been pre-calibrated with respect to the internal and external parameters of the stereo system, a number of images, for example 12, are captured from different locations. This stereo image sequence is transmitted to a computer (not shown for clarity). During the computational evaluation of the stereo image sequence, homologous points are assigned to each other using the established method of time correlation. From these, 3D points in space are determined using the also known calibration parameters, which can then be further processed depending on the application. The motor-controlled movement of the deflecting mirror 5 enables a very rapid change in the projected statistical pattern on the surface 2 to be measured, and thus a very fast high-resolution reconstruction of the surface 2.Instead of a motorized mirror, an automatic zoom lens, a slide-shifting mechanism, or a light-modifying element, such as a diffracting or refractive element, could be used for pattern variation.

[0022] In particular, diffractive optical elements (DOEs) could be used as light-diffusing elements, either by using digitally switchable spatial light modulators or, in the simplest case, by mechanically shifting the DOE, in each case using a coherent light source. For example, a rotatable wedge could be used to achieve beam deflection by means of a light-refracting element.

Claims

[1] Method for 3D measurement of objects (1) using a stereo system, In a device for the location-different detection and three-dimensional evaluation of stereo image sequences, a statistical optical pattern (3) is projected onto the object (1) via a deflecting mirror (5), wherein the deflecting mirror (5) is attached to a motor (6) such that an axis (7) of the motor (6) intersects a plane of the deflecting mirror (5) almost, but not quite, perpendicularly, and wherein the deflecting mirror (5) is set in motion by rotation of the motor (6), and a wobbling motion of the deflecting mirror (5) is achieved by the slight tilting of the deflecting mirror plane normal to the axis (7) of the motor (6). wherein the statistical optical pattern (3) mapped onto the object (1) is modified in any way with respect to position and / or shape without a defined pattern sequence and / or without an exact pattern position, wherein, at an arbitrary recording rate, stereo image sequences from different locations are recorded by cameras (8, 9) synchronized with each other in a camera system, which have been pre-calibrated with respect to the internal and external parameters of the stereo system, and are transmitted to a computer, and during the evaluation of the stereo image sequences detected from different locations, homologous image points are assigned to each other from the stereo image sequences and spatial information is calculated from these, where, during the computational evaluation of the stereo image sequences detected at different locations, the homologous image points are assigned to each other using a time correlation method, and 3D points in space are determined from these using the calibration parameters. [2] Method according to claim 1, characterized by , that the statistical optical pattern (3) on the object (1) is continuously changing. [3] Method according to claim 1 or 2, characterized by , that the statistical optical pattern (3) is shifted over the object (1). [4] Method according to claim 3, characterized by , that the statistical optical pattern (3) is moved in rotational motion on the object (1). [5] Method according to claim 1 or 2, characterized by , that the statistical optical pattern (3) on the object (1) is moved arbitrarily and without a predetermined coordinate direction. [6] Method according to claim 5, characterized by , that the statistical optical pattern (3) is moved back and forth on the object (1). [7] Device for carrying out the method according to one or more of claims 1 to 6, wherein at least one light source (4) is set up to generate the statistical and location-differentiated optical pattern (3), wherein a deflecting mirror (5) is arranged in the beam path of the light source (4) to the object (1), wherein the deflecting mirror (5) is attached to a motor (6) such that an axis (7) of the motor (6) intersects a plane of the deflecting mirror (5) almost, but not quite, perpendicularly, and wherein the deflecting mirror (5) is set in motion by rotation of the motor (6), and a wobbling motion of the deflecting mirror (5) is achieved by the slight tilting of the deflecting mirror plane normal to the axis (7) of the motor (6), and wherein a camera system with mutually synchronized cameras (8, 9) is used to record image sequences from different locations at any recording rate and to transfer them to a computer. [8] Device according to claim 7, wherein at least one light source (4) is a constant light source.

Images