Three-dimensional shape measurement method

Abstract

Provided is a three-dimensional shape measurement method that enables a high-speed three-dimensional shape measurement in which a phase-coordinate table is referred to on a per-pixel basis and a phase-shift method is used. The three-dimensional shape measurement method uses a three-dimensional measurement device 1 which is provided with a projection unit 2, an imaging unit 3 that has a prescribed number of pixels and that captures the image of an article M being measured to obtain a captured image, and a support 4. During calibration, a phase-shift method is used to derive a phase value for each pixel at positions in the z axis direction and create a phase-coordinate table in which the phase value and a z axis coordinate value are in one-to-one correspondence, and camera parameters of a pinhole camera model are calibrated. During three-dimensional shape measurement, the phase-shift method is used to derive a phase value for each pixel from captured images, the phase-coordinate table is referred to in order to derive a z axis coordinate value which corresponds to the phase value, and for each pixel, x axis and y axis coordinate values which correspond to the z axis coordinate value are calculated via the pinhole camera model having the calibrated camera parameters. Thus, the three-dimensional shape of said article is measured.

Description

3D shape measurement method

[0001] The present invention relates to a three-dimensional shape measurement method using a phase shift method.

[0002] A three-dimensional shape measurement method using a phase shift method is known as a method for measuring the three-dimensional shape of an object without contact. In this three-dimensional shape measurement method, a sinusoidally varying stripe pattern is projected onto the surface of the object by a projection unit and then photographed by an imaging unit, and the phase of the stripe pattern is sequentially shifted by a predetermined angle (e.g., π / 2 or 2π / 3) to obtain a set of photographed images (e.g., four or three). The set of photographed images is then processed to derive phase values ​​for each pixel and three-dimensional coordinate values ​​corresponding to the phase values, thereby measuring the three-dimensional shape of the object.

[0003] Among such three-dimensional shape measurement methods, as disclosed in Patent Documents 1 and 2, there is known a method in which a phase-coordinate table showing three-dimensional coordinate values ​​corresponding to phase values ​​for each pixel of a captured image is created during calibration, and the phase value is calculated for each pixel using a phase shift method or the like during three-dimensional shape measurement, thereby enabling high-precision and high-speed measurement during three-dimensional shape measurement. This phase-coordinate table is created by gradually moving a reference plate, on which a two-dimensional pattern such as a two-dimensional grid on which x-axis and y-axis coordinate values ​​of any location can be identified, in the z-axis direction (the normal direction of the reference plate), calculating the x-axis and y-axis coordinate values ​​for each pixel of the captured image at each position in the z-axis direction, and then calculating the phase value using a phase shift method or the like, thereby establishing a one-to-one correspondence between the phase value and the three-dimensional coordinate values ​​(x-axis, y-axis, and z-axis coordinate values) for each pixel.

[0004] Known projection units include those that use a projector as described in Patent Document 1, and those that use a grid-pattern plate (a stripe pattern forming optical element) that forms a grid pattern (stripe pattern) and passes light from multiple light sources arranged in a row as described in Patent Document 2.

[0005] JP 2008-281491 A JP 2011-242178 A

[0006] Incidentally, a phase-coordinate table, which associates phase values ​​with three-dimensional coordinate values ​​(x-axis, y-axis, and z-axis coordinate values) for each pixel on a one-to-one basis, can have a large number of pixels depending on the size of the object to be measured, the required accuracy, etc., which can result in an enormous amount of data. On the other hand, if the amount of data in the phase-coordinate table becomes enormous, when determining the three-dimensional coordinate values ​​of a product (object to be measured) placed on an object inspection line to measure its three-dimensional shape (during three-dimensional shape measurement), the time required to refer to the phase-coordinate table becomes long, which can impair the high-speed performance of the inspection.

[0007] The present invention has been made in view of the above circumstances, and its object is to provide a three-dimensional shape measurement method that is capable of performing high-speed three-dimensional shape measurement using a phase shift method by referencing a phase-coordinate table for each pixel, even if the number of pixels in a captured image is large.

[0008] In order to achieve the above object, a three-dimensional shape measurement method according to an embodiment of the present invention uses a three-dimensional measurement device including a projection unit that projects a stripe pattern, the phase of which is sequentially shifted by a predetermined angle, onto an object, a photographing unit that has a predetermined number of pixels and photographs the object to obtain photographed images, and a support to which the projection unit and the photographing unit are fixed, and uses a calibration object as the object during calibration, which can be switched between a surface on which a two-dimensional pattern, on which x-axis and y-axis coordinate values ​​of any location can be specified, and a plain surface, and at each of the positions to which the support unit or the calibration object is moved to a plurality of positions in a predetermined z-axis direction, derives a phase value for each pixel from a set of photographed images in which each of the stripe patterns is projected onto the plain surface of the calibration object using a phase shift method, creates a phase-coordinate table in which the phase values ​​and z-axis coordinate values ​​correspond one-to-one, and photographs the two-dimensional pattern of the calibration object at a plurality of positions. The X-axis and Y-axis coordinate values ​​of a plurality of coordinate value extraction points are extracted on a photographed image of a surface on which a stripe is displayed, and the camera parameters are calibrated by performing an iterative method using the camera parameters of the pinhole camera model as variables so as to reduce errors between the X-axis and Y-axis coordinate values ​​of the plurality of coordinate value extraction points and the X-axis and Y-axis coordinate values ​​calculated using a pinhole camera model. When measuring a three-dimensional shape, a measurement object is used as the object, the position of the support in the z-axis direction is fixed, and a phase value for each pixel is derived using a phase shift method from a set of photographed images of the measurement object onto which each of the stripe patterns is projected, and a z-axis coordinate value corresponding to the phase value is derived by referring to the phase-coordinate table. Finally, x-axis and y-axis coordinate values ​​corresponding to the z-axis coordinate value for each pixel are calculated using the pinhole camera model of the calibrated camera parameters, thereby measuring the three-dimensional shape of the measurement object.

[0009] The projection unit has a plurality of light sources arranged in a row, each of which has its lighting and light radiation intensity controlled, and a stripe pattern forming optical element that passes the light from the light sources in a stripe pattern, and by sequentially lighting up each of the plurality of light sources, a stripe pattern is projected onto the object, the phase of which is sequentially shifted by a predetermined angle, and the z-axis direction can be approximately vertical during the calibration and the three-dimensional shape measurement.

[0010] The calibration item can be a flat plate that can be slid to switch between a surface on which the two-dimensional pattern is displayed and a plain surface, or a flat plate that can be switched between a surface on which a two-dimensional pattern whose x-axis and y-axis coordinate values ​​of any location are identified and a plain surface, or a flat plate that has a surface on which a two-dimensional pattern whose x-axis and y-axis coordinate values ​​of any location are identified and a plain surface on both sides.

[0011] The calibration article may be arranged on an article inspection line, and the measurement article may be arranged on the article inspection line.

[0012] According to the three-dimensional shape measurement method of the present invention, even if the number of pixels in a captured image is large, high-speed three-dimensional shape measurement using the phase shift method is possible by referring to a phase-coordinate table for each pixel during three-dimensional shape measurement.

[0013] FIG. 1 is a reduced-scale front view showing an outline of a three-dimensional shape measurement device used in a three-dimensional shape measurement method according to an embodiment of the present invention during three-dimensional shape measurement. FIG. 1 is a front view showing a projection unit, an imaging unit, and a support of the three-dimensional shape measurement device. FIG. 1 shows each part constituting the projection unit in the three-dimensional shape measurement device, where (a) is a bottom view of a plurality of light sources, and (b) is a bottom view of a projection unit case accommodating a plurality of light sources and a stripe pattern forming optical element. FIG. 2 is a circuit diagram of a plurality of light sources of the projection unit in the three-dimensional shape measurement device. FIG. 3 is a front-view cross-sectional view of a stripe pattern forming optical element in the three-dimensional shape measurement device, cut horizontally near the vertical center, where (a) is an example in which a plurality of passing portions are formed regularly in a stripe pattern, and (b) is an example in which a plurality of cylindrical lenses are formed regularly. FIG. 4 is a waveform diagram showing, in simplified form, the timing of projection and imaging of the three-dimensional shape measurement device. FIG. 5 is a block diagram of a three-dimensional shape measurement processing unit of the three-dimensional shape measurement device implemented by a computer. FIG. 6 is a reduced-scale front view showing an outline of a three-dimensional shape measurement device used in the three-dimensional shape measurement method during calibration. 1 is a photograph of a surface on which a two-dimensional pattern of a calibration object is displayed in the three-dimensional shape measurement device of the same; a photograph of a set of photographs of a plain surface of a calibration object in the same three-dimensional shape measurement device; and a photograph of a set of photographs of a measurement object in the same three-dimensional shape measurement device.

[0014] An embodiment of the present invention will now be described. As shown in Figures 1 and 2, a three-dimensional shape measuring device 1 used in a three-dimensional shape measuring method according to an embodiment of the present invention includes a projection unit 2, an imaging unit 3, and a support 4. Furthermore, as shown in Figure 1, the three-dimensional shape measuring device 1 includes a projection control unit 5 and a three-dimensional shape measurement processing unit 6.

[0015] The projection unit 2 projects stripe patterns P1, P2, P3, and P4, each having a phase sequentially shifted by a predetermined angle, onto the article M. The stripe patterns P1, P2, P3, and P4 emitted from the projection unit 2 have light radiation intensities that vary sinusoidally. The predetermined angle of the phase shift is π / 2 in this embodiment, but it can also be 2π / 3, for example.

[0016] Although a projector can be used as the projection unit 2, in this embodiment, the projection unit 2 has a plurality of (specifically, four) light sources 21A, 21B, 21C, and 21D and a stripe pattern forming optical element 22. Unless otherwise specified, the projection unit 2 is assumed to have a plurality of light sources 21A, 21B, 21C, and 21D and a stripe pattern forming optical element 22. The plurality of light sources 21A, 21B, 21C, and 21D and the stripe pattern forming optical element 22 can be housed in a projection unit case 23. The number of light sources depends on the predetermined angle of the phase shift described above. In this embodiment, the predetermined angle of the phase shift is π / 2, so the number is four. If the predetermined angle of the phase shift is 2π / 3, the number of light sources is three.

[0017] The plurality of light sources 21A, 21B, 21C, and 21D are arranged in a row (in a row in the horizontal direction). Each of the plurality of light sources 21A, 21B, 21C, and 21D can be configured with a plurality of light-emitting elements 210 (six in this embodiment) arranged in a row in the vertical direction, as shown in Fig. 3(a) . The plurality of light-emitting elements 210 are connected in parallel, in series, or in a combination thereof in terms of a circuit. For example, as shown in Fig. 4 , all of the light sources 21A, 21B, 21C, and 21D can be connected in series.

[0018] 4, the terminal designated by the reference numeral 211 is a current limiting resistor. The terminals designated by the reference numerals 21Aa, 21Ba, 21Ca, and 21Da are current inflow terminals, and the terminals designated by the reference numerals 21Ab, 21Bb, 21Cb, and 21Db are current outflow terminals. It is possible to combine a plurality of the current inflow terminals 21Aa, 21Ba, 21Ca, and 21Da and the current outflow terminals 21Ab, 21Bb, 21Cb, and 21Db into a single common terminal.

[0019] The stripe pattern forming optical element 22 passes light from the multiple light sources 21A, 21B, 21C, and 21D in a stripe pattern. As shown in FIGS. 3B and 5A, the stripe pattern forming optical element 22 has multiple passing portions 22a formed in a regular stripe pattern (grid pattern). The multiple passing portions 22a pass light from the multiple light sources 21A, 21B, 21C, and 21D. The areas other than the multiple passing portions 22a are blocking portions 22b that do not pass light. In this embodiment, the stripe pattern forming optical element 22 is disposed close to and behind the front surface of the projection unit case 23, and an opening 23a is formed in the front surface of the projection unit case 23. The light that passes through the multiple passing portions 22a of the stripe pattern forming optical element 22 is emitted to the outside of the projection unit 2 through the opening 23a. In addition, in FIG. 3(b) and FIG. 5(a), the shielding portion 22b of the stripe pattern forming optical element 22 is painted black for ease of understanding.

[0020] The stripe pattern forming optical element 22 may have a plurality of passing portions 22a arranged in a regular stripe pattern, or may have a plurality of cylindrical lenses 22c arranged in a regular pattern, as shown in FIG. 5( b). Each of the plurality of cylindrical lenses 22c is a convex lens that refracts and focuses light and radiates it to the outside. As a result, light that passes through this stripe pattern forming optical element 22 forms a stripe pattern. In a stripe pattern forming optical element 22 having such cylindrical lenses 22c, all light passes through, thereby increasing the amount of light that is radiated.

[0021] The projection unit 2 sequentially lights up a plurality of light sources 21A, 21B, 21C, and 21D, thereby projecting stripe patterns P1, P2, P3, and P4, the phases of which are sequentially shifted by a predetermined angle, onto the object M. Since the operation of the projection unit 2 simply involves sequentially lighting up a plurality of light sources 21A, 21B, 21C, and 21D, the projection unit 2 is capable of high-speed projection of phase-shifted stripe patterns and can be made smaller and lighter.

[0022] The photographing unit 3 has a predetermined number of pixels and photographs the item M.

[0023] The support 4 is a member to which the projection unit 2 and the imaging unit 3 are fixed and which supports them. The shape of the support 4 is not particularly limited, and for example, a plate material large enough to support the projection unit 2 and the imaging unit 3 can be used. The support 4 can be freely moved in the z-axis direction relative to the item M (i.e., the item placement member 4C on which the item M is placed) during calibration, which will be described later. For example, the support 4 can be fixed to a slider member 4A, and the slider member 4A can be freely moved electrically or manually in the z-axis direction of a linear guide member 4B extending in the z-axis direction, and can be fixed at any position.

[0024] Here, if the projection unit 2 uses multiple light sources 21A, 21B, 21C, and 21D and the stripe pattern forming optical element 22 as described above, it can be made smaller and lighter, and this smaller size and lighter weight allows the projection unit 2 to be supported on a support 4 together with the imaging unit 3, with the z-axis direction being the approximately vertical direction (height direction), and the support 4 can be easily moved in the z-axis direction relative to the article M. In this way, as will be described later, it becomes possible to make the article placement member 4C on which the calibration article M1 is placed as the article M during calibration the same as the article inspection line L on which the measurement article M2 is placed as the article M during three-dimensional shape measurement. In this case, naturally, the article inspection line L as the article placement member 4C is stationary during calibration.

[0025] Note that even when the projection unit 2 uses a projector, or when the projection unit 2 uses a plurality of light sources 21A, 21B, 21C, and 21D and the stripe pattern forming optical element 22, if the size or weight is not sufficiently reduced to allow the projection unit 2 and the image capture unit 3 to move relative to the object M (calibration object M1) along the substantially vertical z-axis direction during calibration, the following can be done. That is, it is possible to set a direction other than the substantially vertical direction (e.g., the substantially horizontal direction) as the z-axis direction, support the projection unit 2 and the image capture unit 3 on a support 4, and move the support 4 along the z-axis direction relative to the object M. Alternatively, instead of moving the support 4, it is also possible to move the object M (calibration object M1) (more specifically, for example, an object placement member 4C on which the calibration object M1 is placed) along the z-axis direction, with the substantially vertical direction or a direction other than the substantially vertical direction (e.g., the substantially horizontal direction) as the z-axis direction, and treat the position from the moved position to the support 4 as the same as the position to which the support 4 has moved.

[0026] The projection control unit 5 controls the projection and light radiation intensity of the projection unit 2. When a plurality of light sources 21A, 21B, 21C, and 21D are used, the projection control unit 5 controls the lighting and light radiation intensity of each of the light sources.

[0027] In detail, as shown in FIG. 6, during a projection period Ta synchronized with a trigger signal TR, a current I is sequentially supplied to each of the plurality of light sources 21A, 21B, 21C, and 21D. Ａ , I Ｂ , I Ｃ , I Ｄ (See FIG. 4) is supplied to emit light. The projection period Ta has a preset length. The current can be a PWM pulse current, a constant current, or a constant voltage current. For example, in the case of a PWM pulse current, the PWM period Tb has a preset length. The PWM pulse width Tc Ａ , Tc Ｂ , Tc Ｃ , Tc Ｄcan be controlled for each of the light sources 21A, 21B, 21C, and 21D, and by widening the width, the light radiation intensity increases, and by narrowing the width, the light radiation intensity decreases, thereby making it possible to adjust the light radiation intensity of the plurality of light sources 21A, 21B, 21C, and 21D. The projection period Ta is synchronized with the exposure period Td (signal EX in FIG. 6 indicates the timing of exposure) in the photographing unit 3. In addition, in FIG. 6, the above-mentioned period Tb and pulse width Tc Ａ , Tc Ｂ , Tc Ｃ , Tc Ｄ is shown enlarged for ease of understanding.

[0028] The three-dimensional shape measurement processing unit 6 performs image processing on the captured image obtained by the imaging unit 3 .

[0029] Specifically, the three-dimensional shape measurement processing unit 6 can be realized by a computer comprising, for example, a CPU 6a, an auxiliary storage device 6b such as a hard disk or nonvolatile memory, a main memory (work memory) 6c, and an input / output interface 6d (input / output terminals are omitted from FIG. 7), as shown in FIG. 7. A calibration image processing program 6ba for performing image processing during calibration and a three-dimensional shape measurement image processing program 6bb for performing image processing during three-dimensional shape measurement are stored in the auxiliary storage device 6b, and can be transferred to the main memory 6c and executed when each program is executed. In addition, a storage area 6bc for a phase-coordinate table, which will be described later, and a storage area 6bd for camera parameters can be provided in the auxiliary storage device 6b.

[0030] Next, we will explain the calibration of the three-dimensional shape measuring device 1 in the three-dimensional shape measurement method according to the embodiment of the present invention. Here, calibration of the three-dimensional shape measuring device 1 refers to creating a phase-coordinate table to be referenced when deriving the z-axis coordinate value (the z-axis coordinate value among the three-dimensional coordinate values ​​(x-axis, y-axis, and z-axis coordinate values) of the object M (measurement object M2)) corresponding to a pixel from the phase value θ obtained for each pixel of the captured image using a phase shift method during three-dimensional shape measurement, which will be described later, and calibrating the camera parameters used when deriving the x-axis and y-axis coordinate values ​​from the derived z-axis coordinate value.

[0031] During calibration, as shown in Fig. 8, a calibration article M1 is used as the article M. The calibration article M1 is a flat plate that can be switched between a surface M1a on which a two-dimensional pattern is displayed, allowing the x-axis and y-axis coordinate values ​​of any location to be identified, and a plain surface M1b (no pattern is displayed), relative to the imaging unit 3. The two-dimensional pattern can be a regular pattern such as a checkered pattern (in which two-colored squares of a constant size are alternately arranged in a two-dimensional pattern), a dot pattern (in which dots are arranged in a two-dimensional pattern at a constant pitch), or a two-dimensional grid pattern (in which multiple parallel lines are arranged at regular intervals in the x-axis and y-axis directions).

[0032] It is preferable that the calibration article M1 be a flat plate having a surface M1a on which a two-dimensional pattern is drawn and a plain surface M1b on its front and back, and that the front and back surfaces can be manually reversed. In this way, the thickness of the flat plate is necessarily constant between the surface M1a on which the two-dimensional pattern is drawn and the plain surface M1b, and the surface M1a on which the two-dimensional pattern is drawn and the plain surface M1b can be easily and precisely flattened. Furthermore, the two-dimensional pattern can be easily and clearly drawn. In this way, high-precision calibration is possible.

[0033] Furthermore, the calibration object M1 can be composed of two components: a flat plate with a surface M1a on which a two-dimensional pattern is drawn and a flat plate with a plain surface M1b, and these components can be manually switched between them. Alternatively, a flat plate can be used that has a portion with a surface M1a on which a two-dimensional pattern is drawn and a portion with a plain surface M1b on the same plane, and these portions can be switched by sliding electrically or manually. In this way, the thickness of the flat plate can be easily and precisely constant between the surface M1a on which the two-dimensional pattern is drawn and the plain surface M1b, and the surface M1a on which the two-dimensional pattern is drawn and the plain surface M1b can be easily and precisely flattened. Furthermore, the two-dimensional pattern can be easily and clearly drawn. In this way, high-precision calibration is possible.

[0034] It is also possible to use a liquid crystal display device that can display a two-dimensional pattern on the surface M1a and that can prevent any pattern from being displayed on the surface M1b.

[0035] The support 4 (or the calibration object M1) is moved to a plurality of positions in the predetermined z-axis direction in order from the initial position (z = 0). For example, the initial position may be the position where the support 4 is furthest from the calibration object M1, and the z-axis direction value may then increase as the support 4 approaches the calibration object M1.

[0036] The phase-coordinate table is created as follows.

[0037] At a plurality of positions in the z-axis direction, including the initial position (for example, positions every 1 mm), sinusoidally varying stripe patterns P1, P2, P3, and P4 are sequentially projected by the projection unit 2 onto the plain surface M1b of the calibration object M1, and these are photographed by the photographing unit 3 to obtain a set of photographed images. In this embodiment, there are four photographed images. If the number of light sources is three, there will be three photographed images. In this set of photographed images, as shown in Figures 10(a) to (d), each of the stripe patterns P1, P2, P3, and P4 is regular and without distortion.

[0038] The three-dimensional shape measurement processing unit 6 then applies the phase shift method to the set of captured images to derive the phase value θ for each pixel. Since the phase shift method is a well-known method, a detailed description will be omitted. For example, in the case where there are four light sources, if I1, I2, I3, and I4 are the luminance values ​​of the stripe patterns P1, P2, P3, and P4, respectively, the phase value θ for each pixel can be derived using the following formula: tan θ=(I4-I2) / (I1-I3).

[0039] In this way, a phase-coordinate table can be created in which the phase value θ and the z-axis coordinate value are associated one-to-one for each pixel for a predetermined number of z-axis coordinate values. The phase-coordinate table can be stored in the auxiliary storage device 6b of the three-dimensional shape measurement processing unit 6.

[0040] Next, the calibration of camera parameters will be described. Camera parameters are variables that represent the mathematical relationship between the position (actual position) (three-dimensional coordinate values) of the item M and its position on the captured image (two-dimensional coordinate values). The mathematical relationship depends on the camera model used. Among these, the pinhole camera model is known as a typical camera model.

[0041] Specifically, the camera parameters can be calibrated, for example, as follows.

[0042] By photographing the surface M1a, on which the two-dimensional pattern of the calibration object M1 is displayed, at a plurality of positions using the photographing unit 3, data on the X-axis and Y-axis coordinate values ​​on the photographed image corresponding to the actual x-axis and y-axis coordinate values ​​of the calibration object M1 at a plurality of points (coordinate value extraction points) is extracted in order to calibrate the camera parameters. At this time, the projection unit 2 is not operated. Note that the actual x-axis and y-axis coordinate values ​​of the calibration object M1 at the coordinate value extraction points are known. In this embodiment, these multiple positions (multiple positions photographed by the photographing unit 3) are multiple positions in different z-axis directions. The multiple positions in the z-axis direction can include an initial position, and do not need to coincide with the multiple positions in the z-axis direction when creating the phase-coordinate table; for example, they can be positions spaced every 5 mm.

[0043] The coordinate value extraction points can be, for example, corners of a rectangle if the two-dimensional pattern of the calibration object M1 photographed by the photographing unit 3 is a checkered pattern as shown in Fig. 9(a), centers of dots if the two-dimensional pattern is a dot pattern as shown in Fig. 9(b), or intersections of parallel lines if the two-dimensional grid pattern is as shown in Fig. 9(c). Note that although detailed methods for determining the X-axis and Y-axis coordinate values ​​of the coordinate value extraction points on the photographed image are not the gist of the present application and will not be described here, they can be obtained with sub-pixel accuracy by gradient processing of pixel values. The X-axis and Y-axis coordinate values ​​of the multiple coordinate value extraction points on the photographed image can be numbered for each of multiple positions (i.e., z-axis coordinate values) in the z-axis direction and temporarily stored in the auxiliary storage device 6b or the main memory 6c.

[0044] In the pinhole camera model, the relationship between the X-axis and Y-axis coordinate values ​​on the captured image and the actual x-axis, y-axis, and z-axis coordinate values ​​of the calibration object M1 is expressed by the following equation (1) using the projection matrix P, and the function g Ｘ , g Ｙ These can be expressed by the following equations (2(1)) and (2(2)) using In equation (1), P is a 3x4 matrix, and s is a coefficient determined so that the third component on the left side becomes 1. The function g in equations (2(1)) and (2(2)) Ｘ , g Ｙ is a function that represents the distortion of the lens.

[0045] If P=AB, equation (1) can be expressed as the following equation (3). In equation (3), A is a matrix of internal camera parameters representing the optical center and focal length of the imaging unit 3, and B is a matrix of external camera parameters representing the positional relationship, including translation and rotation, between the article M and the imaging unit 3. A is a 3x4 matrix, and B is a 4x4 matrix.

[0046] B can be expressed by, for example, the following equation (4). In formula (4), a １１ , a １２ , a １３ , a ２１ , a ２２ , a ２３ , a ３１ , a ３２ , a ３３ is an external camera parameter that represents the positional relationship of rotation, and a １４ , a ２４ , a ３４ is a matrix of external camera parameters that represents the positional relationship of translation.

[0047] The function g in equations (2(1)) and (2(2)) Ｘ , g Ｙ is a nonlinear function. Ｘ , g Ｙ is not particularly limited, but for example, when only the radial distortion of the lens is taken into consideration, it can be given as a simple function by the following equations (5(1)) and (5(2)). where κ is the camera parameter of the distortion coefficient.

[0048] The calibration of the camera parameters in the pinhole camera model can be performed by performing an iterative method using the camera parameters as variables so as to reduce the error (root mean square (RMS) value) between the X-axis and Y-axis coordinate values ​​on the captured image (specifically, the z-axis coordinate values ​​that are numbered and temporarily saved) and the X-axis and Y-axis coordinate values ​​on the captured image calculated using the pinhole camera model (specifically, equations (1), (2(1)), and (2(2))) at a plurality of coordinate value extraction points. In this case, if the magnitude of change in all camera parameters during the iteration falls below a specified accuracy value, the iteration can be terminated as convergence has occurred.

[0049] The camera parameters calibrated in this way can be stored in the auxiliary storage device 6b.

[0050] In addition to the above, various other methods are possible for calibrating the camera parameters, such as a method of obtaining the camera parameters by capturing images at a plurality of positions including rotation and translation (a method known as the Zhang method). In this case, as in the above, the camera parameters can be calibrated by extracting the X-axis and Y-axis coordinate values ​​of a plurality of coordinate value extraction points on a captured image of the surface on which the two-dimensional pattern of the calibration object M1 is displayed at a plurality of positions, and performing an iterative method using the camera parameters of the pinhole camera model as variables so as to reduce errors between the X-axis and Y-axis coordinate values ​​of the plurality of coordinate value extraction points and the X-axis and Y-axis coordinate values ​​calculated by the pinhole camera model.

[0051] Next, three-dimensional shape measurement in a three-dimensional shape measurement method according to an embodiment of the present invention will be described.

[0052] When measuring the three-dimensional shape, a measurement object M2 is used as the object M (see FIG. 1). When measuring the three-dimensional shape, the calibrated three-dimensional shape measuring device 1 is placed on the object inspection line L and inspects the measurement objects M2 that flow one after another on the object inspection line L. The support 4 has a fixed position in the z-axis direction (for example, fixed at a position z = 0). Typically, the object inspection line L is a belt conveyor, and the approximately vertical direction (height direction) is set as the z-axis direction, allowing the measurement objects M2 that are placed on the object inspection line L and flow one after another to be inspected. Note that the phase-coordinate table and camera parameters are usually transferred from the auxiliary storage device 6b to the main memory 6c at the start of the three-dimensional shape measurement.

[0053] Then, in the same manner as with the plain surface M1b of calibration object M1 during calibration, sinusoidally varying stripe patterns P1, P2, P3, and P4 are projected sequentially by projection unit 2 onto measurement object M2, and these are photographed by photographing unit 3 to obtain a set of photographed images. In this set of photographed images, as shown in Figures 11(a) to 11(d), each of the stripe patterns P1, P2, P3, and P4 is distorted in accordance with the three-dimensional shape of measurement object M2 (a gable shape in Figures 11(a) to 11(d)).

[0054] The three-dimensional shape measurement processing unit 6 then applies the phase shift method to derive a phase value θ for each pixel from a set of captured images, in the same manner as for the plain surface M1b of the calibration item M1 during calibration. Then, the z-axis coordinate value corresponding to that phase value θ is derived by referring to the phase-coordinate table. At this time, the z-axis coordinate value corresponding to the phase value θ may be a value obtained by interpolating between z-axis coordinate values ​​corresponding to large and small phase values ​​close to the phase value θ in the phase-coordinate table, or may simply be the z-axis coordinate value corresponding to the phase value closest to the phase value θ in the phase-coordinate table.

[0055] Then, the x-axis and y-axis coordinate values ​​corresponding to the z-axis coordinate value are calculated for each pixel using a pinhole camera model with calibrated camera parameters, thereby enabling measurement of the three-dimensional shape of the measurement object M2.

[0056] The x-axis and y-axis coordinate values ​​can be calculated as follows.

[0057] For example, the above formulas (5(1)) and (5(2)) can be transformed into the following formulas (5(1)') and (5(2)') using an inverse function. In this way, equations (2(1)) and (2(2)) are functions g Ｘ －１ , g Ｙ －１ can be transformed into the following equations (2(1)') and (2(2)') using

[0058] The above equation (3) is a specific z-axis coordinate value z ０ can be expressed as (3') below. In equation (3'), B' is a 4 × 3 matrix. B' can be expressed as the following equation (4') by modifying equation (4) above.

[0059] Therefore, if P' = AB', the above equation (1) is ０ can be expressed by the following equation (1'). In equation (1'), P' is a 3x3 square matrix. P' is the inverse matrix P' －１ Since there may exist, equation (1') can be transformed into equation (1'') below.

[0060] Therefore, by using equations (2(1)') and (2(2)') (for example, equations (5(1)) and (5(2))) and equation (1"), it is possible to quickly calculate the x-axis and y-axis coordinate values ​​for each pixel.

[0061] As described above, according to the three-dimensional shape measurement method of the present invention, during three-dimensional shape measurement, the z-axis coordinate value corresponding to each pixel of the captured image is derived from the phase value θ obtained for that pixel using the phase shift method. Then, the x-axis and y-axis coordinate values ​​corresponding to the z-axis coordinate value are calculated for each pixel using a pinhole camera model with calibrated camera parameters. In this case, compared to a method in which the x-axis and y-axis coordinate values ​​corresponding to the z-axis coordinate value for each pixel are stored and then referenced to derive the x-axis and y-axis coordinate values, the total time required to access the main memory 6c can be shortened, thereby reducing the total time required to calculate the x-axis and y-axis coordinate values. Therefore, high-speed three-dimensional shape measurement is possible even when the number of pixels in the captured image is large.

[0062] Furthermore, it is possible to reduce the area of ​​the auxiliary storage device 6b and the main memory 6c used for three-dimensional shape measurement.

[0063] Although the three-dimensional shape measurement method according to the embodiment of the present invention has been described above, the present invention is not limited to the embodiment described above, and various design modifications are possible within the scope of the claims. For example, it is also possible to provide two projection units 2 so as to project stripe patterns P1, P2, P3, and P4 onto the object M from different directions, and to fix the two projection units 2 to the support body 4.

[0064] 1 Three-dimensional shape measurement device 2 Projection unit 21A, 21B, 21C, 21D Light source 21Aa, 21Ba, 21Ca, 21Da Current inflow terminal 21Ab, 21Bb, 21Cb, 21Db Current outflow terminal 210 Light emitting element 211 Resistor 22 Stripe pattern forming optical element 22a Passing portion 22b Shielding portion 22c Cylindrical lens 23 Projection unit case 23a Opening of projection unit case 3 Photography unit 4 Support 4A Slider member 4B Linear guide member 4C Item placement member 5 Projection control unit 6 Three-dimensional shape measurement processing unit 6a CPU 6b Auxiliary storage device 6ba Calibration image processing program 6bb Three-dimensional shape measurement image processing program 6bc Phase-coordinate table storage area 6bd Camera parameter storage area 6c Main memory 6d Input / output interface EX Signal indicating exposure timing I Ａ , I Ｂ , I Ｃ , I Ｄ Current flowing through the light source L: Product inspection line M: Product M1: Calibration product M1a: Surface on which the two-dimensional pattern is displayed M1b: Plain surface M2: Product to be measured P1, P2, P3, P4: Stripe pattern Ta: Projection period Tb: PWM cycle Tc Ａ , Tc Ｂ , Tc Ｃ , Tc Ｄ PWM pulse width Td Exposure period TR Trigger signal

Claims

1. Using a three-dimensional measuring device comprising: a projection unit that projects a stripe pattern, the phase of which is sequentially shifted by a predetermined angle, onto an object; a photographing unit that has a predetermined number of pixels and photographs the object to obtain photographed images; and a support to which the projection unit and the photographing unit are fixed, during calibration, a calibration object is used as the object, which can be switched between a surface on which a two-dimensional pattern, on which x-axis and y-axis coordinate values ​​of any location can be identified, and a plain surface; at each of the positions to which the support unit or the calibration object is moved to a plurality of positions in a predetermined z-axis direction, a phase value for each pixel is derived using a phase shift method from a set of photographed images in which each of the stripe patterns is projected onto the plain surface of the calibration object, and a phase-coordinate table in which the phase values ​​and z-axis coordinate values ​​correspond one-to-one; at a plurality of positions, X-axis and Y-axis coordinate values ​​of a plurality of coordinate value extraction points on the photographed image of the surface of the calibration object on which the two-dimensional pattern is displayed are extracted; the camera parameters are calibrated by performing an iterative method using the camera parameters of the pinhole camera model as variables so as to reduce errors between the X-axis and Y-axis coordinate values ​​of the plurality of coordinate value extraction points and the X-axis and Y-axis coordinate values ​​calculated by the pinhole camera model; and when measuring the three-dimensional shape, a measurement object is used as the object, the position of the support in the z-axis direction is fixed, and a phase value for each pixel is derived using a phase shift method from a set of captured images of the measurement object onto which each of the stripe patterns is projected, and the z-axis coordinate value corresponding to the phase value is derived by referring to the phase-coordinate table, and the x-axis and y-axis coordinate values ​​corresponding to the z-axis coordinate value for each pixel are calculated using the pinhole camera model of the calibrated camera parameters, thereby measuring the three-dimensional shape of the measurement object.

2. A three-dimensional shape measurement method as claimed in claim 1, wherein the projection unit has a plurality of light sources arranged in a row, each of which has its lighting and light radiation intensity controlled, and a stripe pattern forming optical element that passes the light from these light sources in a stripe pattern, and projects a stripe pattern onto the object whose phase is sequentially shifted by a predetermined angle as each of the plurality of light sources is sequentially lit, and wherein the z-axis direction is approximately vertical during the calibration and the three-dimensional shape measurement.

3. A three-dimensional shape measurement method according to claim 1 or 2, wherein the calibration item is a flat plate that can be slid to switch between a surface on which the two-dimensional pattern is displayed and a plain surface, or an item that can be switched between a flat plate with a surface on which a two-dimensional pattern whose x-axis and y-axis coordinate values ​​of any location can be identified and a plain surface, or a flat plate that has a surface on which a two-dimensional pattern whose x-axis and y-axis coordinate values ​​of any location can be identified and a plain surface on both sides.

4. A three-dimensional shape measurement method according to claim 2, wherein the calibration object is arranged on an object inspection line, and the measurement object is arranged on the object inspection line.

Images