Three-dimensional measuring device

Abstract

To provide a configuration capable of executing accurate three-dimensional measurement by improving the identification accuracy of stripe numbers.SOLUTION: A stripe pattern SP projected onto a measurement object R by a projection unit 20 is generated such that stripe regions in which luminance periodically varies in the horizontal direction and the luminance does not vary in the vertical direction are arranged along the horizontal direction, and the pattern is projected multiple times by shifting a phase. Furthermore, a stripe-number identification pattern NP projected by the projection unit 20 for identifying a stripe number SN is generated such that the luminance varies sinusoidally in one cycle along the horizontal direction and the luminance does not vary in the vertical direction, and the pattern is projected multiple times by shifting the phase. A measuring unit 40 identifies the stripe number SN for each pixel on the basis of a phase value θ obtained according to the three-dimensional shape of the measurement object R from an image captured during projection of the stripe pattern SP, and a stripe phase value φ obtained from an image captured during projection of the stripe-number identification pattern NP, and measures the three-dimensional shape of the measurement object R on the basis of the stripe number SN and the phase value θ.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a three-dimensional measuring device for measuring the three-dimensional shape of an object to be measured. [Background technology]

[0002] Conventionally, a known technology related to a three-dimensional measurement device for measuring the three-dimensional shape of a measurement target is a three-dimensional information input camera disclosed in Patent Document 1 below. This three-dimensional information input camera is configured to obtain three-dimensional information of a subject based on a phase value determined according to the three-dimensional shape of the subject from an image captured when stripe pattern light is projected and a so-called stripe number identified from an image captured when stripe-specific light is projected. In this case, a gray code that functions as stripe-specific light for identifying the stripe number is added to both the top and bottom ends of the stripe pattern light and projected from the stripe pattern projection unit together with the stripe pattern light. This allows the stripe pattern light and stripe-specific light to be captured simultaneously, thereby improving the accuracy of identifying the stripe number. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2000-287735 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a configuration in which stripe numbers are identified by projecting a gray code, which is generated so that the combination of white and black changes in the stripe region, as a stripe number identifying pattern (stripe identifying light), as in Patent Document 1, an incorrect stripe number may be identified for a pixel located near the boundary of the stripe region (the boundary between a light color region and a dark color region). Furthermore, the incorrect stripe number may be identified due to the influence of reflectance caused by the color scheme or shape of the measurement object. If an incorrect stripe number is identified in this way, even if a correct phase value is obtained from the captured image when the stripe pattern is projected, an incorrect distance value will be calculated, making accurate three-dimensional measurement difficult.

[0005] The present invention has been made to solve the above-mentioned problems, and its purpose is to provide a configuration that can perform accurate three-dimensional measurements by improving the accuracy of identifying fringe numbers. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention provides a projection unit (20) that projects a predetermined stripe pattern (SP) onto a measurement object (R, R1, R2a to R2c); an imaging unit (30) that images the measurement object onto which the predetermined stripe pattern is projected; a measurement unit (40) that measures the three-dimensional shape of the measurement object imaged by the imaging unit using a phase shift method; A three-dimensional measuring device (10) comprising: the predetermined stripe pattern is generated so that a plurality of stripe regions, each of which has a luminance that periodically changes in a first direction and a luminance that does not change in a second direction orthogonal to the first direction, are arranged along the first direction, and the predetermined stripe pattern is projected a plurality of times with a phase shift; The projection unit further projects stripe number identification patterns (NP, NP1 to NP3) for identifying a stripe number (SN) that distinguishes the stripe region from other stripe regions, the stripe number specifying pattern is generated so that the luminance varies in a sinusoidal manner in one period or less along the first direction and does not vary in the luminance along the second direction, and is projected a plurality of times with a phase shift; The measurement unit identifies the stripe number for each pixel based on a first phase value (θ) obtained from an image captured when the specified stripe pattern is projected according to the three-dimensional shape of the object to be measured and a second phase value (φ) obtained from an image captured when the stripe number identification pattern is projected, and measures the three-dimensional shape of the object to be measured based on the stripe number and the first phase value. The symbols in parentheses above indicate the correspondence with the specific means described in the embodiments to be described later. [Effects of the Invention]

[0007] In the present invention, the predetermined stripe pattern projected onto the measurement object by the projection unit is generated so that multiple stripe regions, whose brightness changes periodically in a first direction and whose brightness does not change in a second direction orthogonal to the first direction, are arranged along the first direction, and the pattern is projected multiple times with a phase shift. Furthermore, the stripe number identification pattern projected by the projection unit to identify stripe numbers is generated so that the brightness changes sinusoidally in one period or less along the first direction and whose brightness does not change in the second direction, and the pattern is projected multiple times with a phase shift. The measurement unit identifies stripe numbers for each pixel based on a first phase value calculated according to the three-dimensional shape of the measurement object from the image captured when the predetermined stripe pattern is projected and a second phase value calculated from the image captured when the stripe number identification pattern is projected, and measures the three-dimensional shape of the measurement object based on the stripe numbers and the first phase value.

[0008] For example, by projecting a stripe number identifying pattern whose luminance varies sinusoidally over one period along the first direction three times with a phase shift of 2π / 3, a second phase value that linearly increases along the first direction can be obtained from three captured images. In this case, for stripe numbers that increase along the first direction, the second phase value obtained as described above increases for pixels onto which stripe regions with larger stripe numbers are projected. Using a linearly varying second phase value for stripe number identification can reduce erroneous identification of stripe numbers near the boundaries of stripe regions, unlike a Gray code in which the luminance for identifying stripe numbers changes suddenly near the boundaries of stripe regions. In particular, using multiple phase-shifted stripe number identifying patterns can reduce the influence of the reflectivity of the measurement object when acquiring the second phase value. Therefore, a three-dimensional measuring device capable of performing accurate three-dimensional measurements can be realized by improving the accuracy of identifying stripe numbers.

[0009] The projection unit is configured to control the gradation of light according to the light emission time within a unit time for each pixel, and the imaging unit may be an event camera that includes an imaging element that outputs event data including two-dimensional point data that identifies the position of a pixel corresponding to a pixel that has experienced a change in brightness when receiving light, and is capable of generating an imaged image from the event data output from the imaging element.

[0010] An event camera is a brightness value differential output camera developed based on the structure of the retina of living organisms. It detects brightness changes at each pixel and outputs their coordinates, time, and the polarity of the brightness change. Unlike conventional frame cameras, it does not output pixel information without brightness changes, i.e., redundant data, allowing for faster acquisition of brightness change data. With this type of event camera, when a brightening brightness change occurs at a pixel-by-pixel level, positive-polarity (positive brightness change) event data is output. When the light disappears, a darkening brightness change occurs and negative-polarity (negative brightness change) event data is output. In other words, the time difference between the output timing of positive-polarity event data and the output timing of negative-polarity event data corresponds to the brightness value, and the ability to measure this time difference more quickly shortens the processing time required to acquire first and second phase values, i.e., shortens the measurement time required for three-dimensional measurement.

[0011] The stripe number identifying pattern may be generated so that the luminance varies sinusoidally in less than one cycle along the first direction. When the stripe number identifying pattern is generated so that the luminance varies sinusoidally in one cycle along the first direction, if the second phase value at the left end of the stripe number identifying pattern is 0° and the second phase value at the right end is 360°, for example, the stripe numbers may be confused between the left and right ends. The same is true when the second phase value at the left end of the stripe number identifying pattern is 10°, changing in one cycle. Therefore, by generating the stripe number identifying pattern so that the luminance varies sinusoidally in less than one cycle along the first direction, the second phase value at the left end and the second phase value at the right end are different, preventing the above-mentioned confusion between stripe numbers.

[0012] The measurement section may specify a measurement target range from a measurement result of the three-dimensional shape based on the second phase value, and measure the three-dimensional shape of the measurement target range based on the fringe number and the first phase value.

[0013] Since the second phase value corresponds to the first phase value determined according to the three-dimensional shape from the captured image when a stripe pattern with one stripe is projected, using the second phase value makes it possible to obtain a simplified measurement result of the three-dimensional shape of the measurement target, etc. Although the measurement accuracy decreases by using the second phase value, the measurement target range to be measured can be identified from the simplified measurement result. Therefore, by measuring the three-dimensional shape of the measurement target range based on the identified stripe number and first phase value as described above, measurement processing is not performed on ranges that do not need to be measured, and therefore the measurement time can be shortened and the measurement processing load can be reduced. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a schematic configuration of a three-dimensional measuring apparatus according to a first embodiment. [Figure 2] FIG. 1 is an explanatory diagram illustrating a state in which a stripe pattern for a general phase shift method is projected onto a measurement object. [Figure 3] Figure 3(A) is an explanatory diagram explaining each stripe number identifying pattern and stripe phase value, and Figure 3(B) is an explanatory diagram illustrating a stripe number identifying pattern NP1 projected to match the projection range of the stripe pattern. [Figure 4] FIG. 1 is an explanatory diagram illustrating three-dimensional measurement by a phase shift method. [Figure 5] Figure 5(A) is an explanatory diagram explaining the positional relationship between a conventional stripe number identification pattern and stripe numbers, Figure 5(B) is an explanatory diagram explaining the phase value obtained per stripe area, and Figure 5(C) is an explanatory diagram explaining the stripe brightness and stripe brightness range obtained when projecting the stripe number identification pattern of Figure 5(A). [Figure 6] Figure 6(A) is an explanatory diagram explaining the phase value obtained per stripe area in this embodiment, and Figure 6(B) is an explanatory diagram explaining the stripe phase value and stripe phase value range obtained when projecting the stripe number identification pattern of Figure 3. [Figure 7] 10A and 10B are explanatory diagrams illustrating specific examples of ideal fringe phase values and fringe phase value ranges obtained for each fringe number for each phase value. [Figure 8]FIG. 11 is an explanatory diagram illustrating the relationship between the time difference between the output timings of two event data and the luminance value in the second embodiment. [Figure 9] 13 is an explanatory diagram illustrating each stripe number specifying pattern and stripe phase value in the third embodiment. FIG. [Figure 10] Figure 10(A) is an explanatory diagram explaining a rectangular parallelepiped-shaped measurement object to be measured in the fourth embodiment, Figure 10(B) is an explanatory diagram explaining an image of a simplified measurement result for the measurement object etc. of Figure 10(A) and the measurement target range, and Figure 10(C) is an explanatory diagram explaining an image of a detailed measurement result within the measurement target range of Figure 10(B). [Figure 11] Figure 11(A) is an explanatory diagram illustrating three cylindrical measurement objects to be measured in the fourth embodiment, Figure 11(B) is an explanatory diagram illustrating an image of the simplified measurement results for the measurement objects in Figure 11(A) and the measurement target range, and Figure 11(C) is an explanatory diagram illustrating an image of the detailed measurement results within the measurement target range in Figure 11(B). [Figure 12] Figure 12(A) is an explanatory diagram illustrating a pattern for identifying stripe numbers that is generated so that the stripe phase value decreases linearly from the left end to the right end of the projection range, and Figure 12(B) is an explanatory diagram illustrating a pattern for identifying stripe numbers that is generated so that the stripe phase value changes suddenly within a predetermined range. DETAILED DESCRIPTION OF THE INVENTION

[0015] [First embodiment] A first embodiment of a three-dimensional measuring device according to the present invention will be described below with reference to the drawings. A three-dimensional measuring device 10 according to this embodiment is an apparatus for measuring the three-dimensional shape of a measurement object R. As shown in FIGS. 1 and 2 , the three-dimensional measuring device 10 is configured to include a control unit 11 that handles overall control, a projection unit 20 that projects a predetermined stripe pattern (hereinafter also referred to as stripe pattern SP) for the phase shift method onto the measurement object R, an imaging unit 30 that captures an image of the measurement object R onto which the stripe pattern SP is projected, and a measurement unit 40 that measures the three-dimensional shape of the measurement object R from the captured image using the phase shift method. The three-dimensional measuring device 10 configured in this manner is attached to, for example, a robot hand, and measures the three-dimensional shape of the measurement object R, such as a workpiece, that moves relatively to the hand at high speed. Here, relative movement refers to the relative movement between the movement of the three-dimensional measuring device 10 attached to the robot hand and the high-speed movement of the measurement object R. If the position of the three-dimensional measuring device 10 is fixed, the relative movement corresponds to the movement of the measurement object R.

[0016] For convenience, Fig. 2 shows a simplified stripe pattern SP with 13 stripe regions, each consisting of a pair of light and dark regions. More specifically, because typical stripe patterns are represented as sine wave patterns, the light and dark regions of the stripe pattern SP have similar widths. However, for convenience, Fig. 2 shows the dark regions with smaller widths as lines. Furthermore, although the number of stripe regions is 13 or more in this embodiment, it is abbreviated to 13.

[0017] The control unit 11 is mainly composed of a microcomputer and has a CPU, a system bus, an input / output interface, etc., and constitutes an information processing device together with a storage unit consisting of a ROM, RAM, non-volatile memory, etc. In addition to programs related to robot control, the storage unit pre-stores programs such as programs related to control of the projection unit 20 and programs for executing control processing using the three-dimensional measurement results from the measurement unit 40 so that they can be executed by the control unit 11.

[0018] The projection unit 20 is a so-called DLP projector, controlled by the control unit 11, which projects a stripe pattern SP and a stripe number identification pattern NP as described below by reflecting light from a light source using a DMD element. The DMD element is an array of minute mirrors, each corresponding to a pixel in the image projected onto the screen. The DMD element is configured to turn the light emitted to the screen on and off in microsecond increments by changing the angle of each mirror. Therefore, each mirror is switched from reflection-off to reflection-on to a light-projecting state, and from reflection-on to reflection-off to a light-off state. That is, the projection unit 20 functions to project the stripe pattern SP and the like by controlling, for each mirror, the on / off reflection of incident light by the DMD, which is an array of multiple mirrors, with the control unit 11 controlling this mirror-by-mirror control. Therefore, the gradation (brightness) of the reflected light can be changed depending on the ratio of the on time to the off time of each mirror within a unit time, thereby enabling a gradation display based on the image data of the image to be projected. In this embodiment, the projection unit 20 includes mirrors corresponding to k×l pixels (for example, 1140×912) with the upper left being (1, 1) and the lower right being (k, l).

[0019] In this configuration, the longer the emission time (the time from reflection ON to reflection OFF) of the single-pulse emission emitted once within the unit time secured for each emission state, the brighter the emission state becomes, so the emission state can be identified according to the emission time. For example, if R (red), G (green), and B (blue) colors are prepared as light incident on the DMD element, an R-color emission state in which R is emitted by reflecting off a mirror, a G-color emission state in which G is emitted by reflecting off a mirror, and a B-color emission state in which B is emitted by reflecting off a mirror are repeated at a predetermined short interval, and the emission times of each state are individually adjusted, thereby enabling the projection of a color image. For this reason, the control unit 11 functions to set the reflection ON / OFF timing within the unit time for each mirror according to the stripe pattern SP to be projected, the stripe number identification pattern NP, etc.

[0020] The imaging unit 30 in this embodiment is a known camera (a so-called frame camera) in which solid-state imaging elements (light receiving elements) such as C-MOS and CCD are arranged two-dimensionally on the light receiving surface, and is configured to be able to generate an image using the output data from each imaging element.

[0021] The measurement unit 40 is controlled by the control unit 11 and measures the three-dimensional shape of the measurement object R using a phase shift method based on multiple captured images of the measurement object R captured by the imaging unit 30 while a stripe pattern SP is projected from the projection unit 20 and an captured image of the measurement object R captured by the imaging unit 30 while a stripe number identification pattern NP is projected from the projection unit 20.

[0022] In this embodiment, the stripe pattern SP projected from the projection unit 20 for the phase shift method is generated by generating multiple stripe regions arranged along the left-right direction, in which the brightness periodically changes in a sine curve (sinusoidal) in the left-right direction (first direction) and the brightness does not change in the up-down direction (second direction perpendicular to the first direction), and projected multiple times with the phase shifted. For this reason, a sine wave pattern specified by the brightness value I(x, y, n) in the following equation (1) is adopted as the stripe pattern SP. That is, when the number of phase shifts is N, the brightness value I(x, y, n) of N phase-shifted grid images (stripe images) is expressed by equation (1). I(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N}+b(x,y) ···(1) Here, point (x, y) is a point in the grid image, a(x, y) is the luminance amplitude, b(x, y) is the background luminance, and θ(x, y) is a phase value corresponding to a value distorted according to the surface shape of the measurement object R in the grid where n=0, and can be calculated from the luminance values I(x, y, n) of N grid images. The phase value θ can be an example of a "first phase value."

[0023] In addition, in this embodiment, in order to determine which stripe region the measured phase value θ belongs to, the projection unit 20 further projects a stripe number identification pattern NP to identify the stripe number SN that distinguishes the stripe region from other stripe regions, in accordance with the projection range of the stripe pattern SP.

[0024] The stripe number identification pattern NP is generated so that the brightness varies sinusoidally in one cycle or less in the left-right direction (first direction) and does not vary in the up-down direction (second direction), and is projected multiple times with a phase shift.

[0025] Specifically, as can be seen from Fig. 3A, three stripe number identifying patterns NP in this embodiment are generated so that the brightness varies sinusoidally in one cycle from the left end to the right end of the projection range. The first stripe number identifying pattern NP1 and the second stripe number identifying pattern NP2 are projected three times in total, with a phase shift of 2π / 3 between the first stripe number identifying pattern NP1 and the second stripe number identifying pattern NP2, and with a phase shift of 2π / 3 between the second stripe number identifying pattern NP2 and the third stripe number identifying pattern NP3. Fig. 3B illustrates the stripe number identifying pattern NP1 projected to fit within the projection range of the stripe pattern SP.

[0026] From the three captured images captured when each stripe number identifying pattern NP1, NP2, and NP3 is projected, a phase value (hereinafter also referred to as the stripe phase value φ) that increases linearly from the left end to the right end of the projection range along the left-right direction can be obtained for each pixel by performing calculations similar to those performed with the phase shift method, as shown in Figure 3(A). In this way, the stripe phase value φ increases toward the right end of the pixel, and for stripe numbers SN that increase along the left-right direction (first direction), the stripe phase value φ obtained as described above increases for pixels onto which stripe areas with larger stripe numbers SN are projected. Therefore, the stripe phase value φ can be used to identify the stripe number SN. Note that the stripe phase value φ can be an example of a "second phase value."

[0027] For this reason, in the three-dimensional measurement process performed by the measurement unit 40, the stripe number SN is identified for each pixel based on the phase value θ calculated from the captured image when the stripe pattern SP is projected in accordance with the three-dimensional shape of the measurement object R and the stripe phase value φ calculated from the captured image when the stripe number identifying pattern NP is projected. Then, based on the stripe number SN thus determined and the phase value θ calculated as described above, the distance z of each point (x, y) of the captured measurement object R is measured, thereby making it possible to measure the three-dimensional shape of the measurement object R.

[0028] For example, when determining the distance z of point P1 in Fig. 4, the stripe number SN is identified based on the phase value θ (see symbol θ1 in Fig. 4) of point P1 determined from N images captured by the imaging unit 30 while the stripe pattern SP is projected from the projection unit 20 after being shifted N times, and the stripe phase value φ determined from the images captured when the stripe number identifying pattern NP is projected. From this stripe number SN and phase value θ1, the angle θp1 at the projection unit 20 and the angle θc1 at the imaging unit 30 shown in Fig. 4 can be determined. Once the angle θp1 at the projection unit 20 and the angle θc1 at the imaging unit 30 are determined in this way, the distance z of point P1 can be determined by triangulation because the distance (parallax L) between the projection unit 20 and the imaging unit 30 is known. Similarly, the distance z of point P2 in Fig. 4 can be determined by identifying the stripe number SN based on the phase value θ (see symbol θ2 in Fig. 4) of point P2 found when projecting the stripe pattern SP and the stripe phase value φ found when projecting the stripe number identifying pattern NP, and then calculating the angle θp2 at the projection unit 20 and the angle θc2 at the imaging unit 30 from this stripe number SN and phase value θ2, and then performing triangulation based on these angles θp2 and θc2. By performing this calculation over the entire measurement area, three-dimensional measurement can be performed.

[0029] Here, the reason for identifying the fringe number SN based on the phase value θ and the fringe phase value φ as described above will be explained. For example, consider a case where a conventional stripe number identification pattern NPo, as illustrated in Figure 5(A), is projected onto a measurement object where the phase value θ, as illustrated in Figure 5(B), is obtained for each stripe area when a stripe pattern SP is projected.

[0030] In this case, the luminance value (hereinafter also referred to as stripe luminance SL) obtained for each pixel when the stripe number identifying pattern NPo is projected increases stepwise for each stripe number, as shown in Fig. 5(C), and the stripe number to which the pixel belongs is identified depending on which stripe luminance range (ΔSL1 to ΔSL4) the obtained stripe luminance SL falls within. Note that in Fig. 5 and Fig. 6 described below, the horizontal axis indicates pixels in the left-right direction, and the boundaries of the stripe regions are indicated by dashed lines.

[0031] In this case, for example, as shown in Figure 5(C), the stripe intensity SL at a pixel Xk near the boundary of the stripe region is included in the stripe intensity range ΔSL2, so stripe number "2" is identified, and the phase value θ at that pixel Xk may be calculated as θko, which is slightly different from the originally calculated θk (see arrow F1 in Figure 5(B)). In other words, what should be calculated as phase value θko at stripe number "3" may be calculated as phase value θko at stripe number "2" (see arrow F2 in Figure 5(B)). Note that, for convenience, Figure 5(B) and other figures exaggerate the difference between θk and θko.

[0032] In this case, a pixel with a phase value θko is searched for within the pixel range specified by stripe number "2", resulting in an incorrect distance value being calculated, which deteriorates the measurement accuracy of the three-dimensional measurement.

[0033] In contrast, the fringe phase value φ obtained by projecting the stripe number identifying pattern NP in this embodiment linearly increases in the left-right direction as shown in Fig. 3(A), so that the fringe phase value φ increases toward the right (the pixel onto which the stripe area with the larger stripe number is projected). As a result, the range of fringe phase values φ (hereinafter also referred to as the fringe phase value range Δφ) assumed for the pixel for which the phase value θ is obtained if it is in the k-th stripe area increases stepwise for each stripe number.

[0034] Therefore, in this embodiment, a fringe phase value range Δφ is set based on the ideal fringe phase value φi calculated for each fringe number SN for the calculated phase value θ, and the fringe number SN is identified based on this fringe phase value range Δφ. Specifically, the ideal fringe phase value φi and the fringe phase value range Δφ are calculated for each fringe number SN for each calculated phase value θ using the following equations (2) to (4) by setting the maximum fringe phase value φ, which is the maximum value of the fringe phase value φ, to φmax and the total number of fringe regions (total number of fringes) to N. When θ-180°≧0° φi=φmax / N×{(SN-1)+(θ-180°) / 360°} ···(2) When θ-180°<0° φi=φmax / N×{(SN-1)+(360°-(θ-180°)) / 360°} ···(3) Δφ=φi±(φmax / N×1 / 2) ···(4)

[0035] For example, when the maximum fringe phase value φmax is 360° and the total number of fringes N is 4, the ideal fringe phase value φi and fringe phase value range Δφ are calculated for each calculated phase value θ and fringe number SN, as shown in Figure 7.

[0036] Here, let us assume a case in which the stripe number identifying pattern NP according to this embodiment is projected onto a measurement object where the phase value θ is determined for each stripe area when the stripe pattern SP is projected, as shown in FIG. 6A. In this case, the ideal stripe phase value φi obtained for each pixel when the stripe number identifying pattern NP is projected increases at a constant rate toward the right, as shown in FIG. 6B. In this case, the stripe phase value range Δφ centered on the ideal stripe phase value φi is also set to increase at a constant rate together with the ideal stripe phase value φi.

[0037] 6A, when the phase value θ at pixel Xk is calculated as θko, which is slightly shifted from the originally calculated θk (see arrow F1 in FIG. 6A), the fringe phase value φ at pixel Xk is compared with the fringe phase value range Δφ1 to Δφ4 calculated as described above for each fringe region from that phase value θko. From the comparison result, it is determined that the fringe phase value φ at pixel Xk is included in the fringe phase value range Δφ3, and the fringe number "3" is identified for that pixel Xk.

[0038] In this way, the stripe number SN can be identified from the fringe phase value range Δφ (Δφ1 to Δφ4) associated with each provisional fringe number from the phase value θ and the fringe phase value φ obtained when projecting the fringe number identifying pattern NP. Therefore, by setting the fringe phase value range Δφ so that it does not overlap with those of other fringe numbers (see equation (3) above), the fringe number SN can be identified with high accuracy.

[0039] As described above, in the three-dimensional measurement apparatus 10 according to this embodiment, the stripe pattern SP projected onto the measurement object R by the projection unit 20 is generated so that multiple stripe regions whose brightness periodically changes in the left-right direction (first direction) and whose brightness does not change in the up-down direction (second direction perpendicular to the first direction) are arranged along the left-right direction, and the pattern is projected multiple times with a phase shift. Furthermore, the stripe number identification pattern NP projected by the projection unit 20 to identify the stripe number SN is generated so that its brightness changes sinusoidally in one period along the left-right direction and whose brightness does not change in the up-down direction, and the pattern is projected multiple times with a phase shift. The measurement unit 40 identifies the stripe number SN for each pixel based on a phase value θ calculated according to the three-dimensional shape of the measurement object R from the image captured when the stripe pattern SP is projected and a stripe phase value φ calculated from the image captured when the stripe number identification pattern NP is projected, and measures the three-dimensional shape of the measurement object R based on the stripe number SN and the phase value θ.

[0040] As a result, by using the linearly varying fringe phase value φ to identify the fringe number as described above, it is possible to suppress erroneous identification of the fringe number SN near the boundary of a fringe area, unlike a Gray code in which the brightness for identifying the fringe number changes suddenly near the boundary of the fringe area. In particular, because multiple phase-shifted fringe number identifying patterns NP (NP1 to NP3) are used, the acquisition of the fringe phase value φ is less susceptible to the influence of the reflectance of the measurement object R. Therefore, by improving the identification accuracy of the fringe number SN, a three-dimensional measuring device 10 can be realized that can perform accurate three-dimensional measurements.

[0041] [Second embodiment] Next, a three-dimensional measuring apparatus according to a second embodiment of the present invention will be described with reference to the drawings. The second embodiment differs from the first embodiment in that three-dimensional measurement is performed using an event camera. Therefore, components that are substantially the same as those in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted.

[0042] In this embodiment, an event camera is used as the imaging unit 30. The event camera includes an imaging element that outputs event data (specifically, two-dimensional point data, time, and polarity of brightness change) including two-dimensional point data that identifies the position of a pixel corresponding to a pixel that has experienced a brightness change when light is received, and is configured to be able to generate a captured image from the event data output from the imaging element.

[0043] An event camera is a brightness value differential output camera developed based on the retinal structure of living organisms. It is configured to sense changes in brightness at each pixel and output the coordinates, time, and polarity of the brightness change. Unlike conventional frame cameras, it does not output pixel information without brightness changes, i.e., redundant data, and therefore can acquire data related to brightness changes more quickly.

[0044] In the imaging unit 30 configured in this way, when a brightness change that makes the pixel brighter occurs, positive polarity (positive brightness change) event data is output, and when the light goes out, a brightness change that makes the pixel darker occurs, and negative polarity (negative brightness change) event data is output. In other words, the time difference between the output timing of the positive polarity event data and the output timing of the negative polarity event data (hereinafter also referred to as the event time difference) becomes information equivalent to the brightness value.

[0045] Therefore, when three images for the phase shift method are captured consecutively during projection of the stripe pattern SP, and event data is output at a pixel (x, y) as illustrated in FIG. 8, the luminance value I(x, y, 0) of the first image can be obtained according to the time difference between time t2 and time t1. Similarly, the luminance value I(x, y, 1) of the second image can be obtained according to the time difference between time t4 and time t3, and the luminance value I(x, y, 2) of the third image can be obtained according to the time difference between time t6 and time t5. The phase value θ can be obtained from the luminance value I(x, y, n) thus obtained using the above formula (1). Note that in FIG. 8, the unit time (in this embodiment, the time required to capture one image for the phase shift method) is illustrated as T / 3.

[0046] Similarly, for the captured images captured at the unit time when each stripe number specifying pattern NP (NP1 to NP3) is projected, the luminance value of each pixel can be obtained based on the event time difference. Then, the stripe phase value φ can be obtained from the luminance value of each pixel thus obtained as described above.

[0047] As described above, in the three-dimensional measuring device 10 according to this embodiment, the projection unit 20 is configured as a DLP projector that controls the gradation of light according to the light emission time per unit time for each pixel, and the imaging unit 30 is configured as an event camera that includes an imaging element that outputs event data including two-dimensional point data that identifies the position of a pixel that has experienced a change in brightness when it receives light, and that can generate an image from the event data output from the imaging element.

[0048] In an event camera configured in this manner, the time difference (event time difference) between the output timing of positive polarity event data and the output timing of negative polarity event data corresponds to the brightness value, and since this event time difference can be measured more quickly, it is possible to shorten the processing time required to obtain the phase value θ and the fringe phase value φ, i.e., to shorten the measurement time for three-dimensional measurement.

[0049] In particular, since the stripe number identification pattern NP (NP1 to NP3) is captured by the imaging unit 30 configured as an event camera, the number of pixels that light up simultaneously during projection is reduced compared to when the same imaging unit 30 captures a conventional gray code pattern that is a stripe number identification pattern, and therefore the number of events output from the imaging unit 30 can be reduced.

[0050] [Third embodiment] Next, a three-dimensional measuring apparatus according to a third embodiment of the present invention will be described with reference to the drawings. In the third embodiment, the stripe number specifying pattern is generated so that the luminance varies in less than one cycle in a sinusoidal manner along the left-right direction (first direction), which is a difference from the second embodiment. Therefore, the same reference numerals are used to designate components that are substantially the same as those in the second embodiment, and their description will be omitted.

[0051] When the stripe number identifying pattern NP is generated so that the brightness varies sinusoidally in one cycle along the left-right direction (first direction), if the stripe phase value φ at the left end of the stripe number identifying pattern NP is 0° and the stripe phase value φ at the right end is 360°, the stripe numbers SN may be confused between the left and right ends. The same applies when the stripe phase value φ at the left end of the stripe number identifying pattern NP varies in one cycle at 10°.

[0052] For this reason, in this embodiment, the stripe number specifying pattern NP is generated so that the luminance varies in less than one period in a sinusoidal manner along the left-right direction.

[0053] 9, an area is assumed that is longer in both the left and right directions than the projection range H1 by a predetermined width H2, and patterns that change sinusoidally in one cycle are generated for this area (H2+H1+H2), and the predetermined width H2 is removed from these patterns to generate stripe number identification patterns NP (NP1 to NP3). This makes it possible to prevent the above-mentioned mix-up of stripe numbers SN, since the stripe phase value φ at the left end and the stripe phase value φ at the right end are different.

[0054] The above-mentioned predetermined width H2 can be set, for example, as follows. First, a phase shift pattern with an appropriate period is projected onto a plane whose distance (planar distance) from the three-dimensional measurement device 10 (imaging unit 30) is known. The phase value is calculated for each point in the captured image, and the deviation between the calculated phase value and the theoretical phase value can be confirmed. On the other hand, when a phase shift pattern with an appropriate period is projected onto a plane whose planar distance is unknown, all phase values are projected, making it impossible to determine the theoretical value of light entering a specific captured image (camera pixel). In contrast, if only one arbitrary projection pixel (projector pixel) in the projection unit 20 is illuminated, the theoretical value is the phase value assigned to the arbitrarily illuminated projection pixel, and is known. When this is photographed, a spot somewhere in the captured image will be illuminated, and the phase value of that spot can be obtained. The error can be determined by comparing this acquired value with the known theoretical value. By calculating the error for each of a predetermined number of phase-value pixels as described above, the deviation from the theoretical phase value can be confirmed. The deviation from the theoretical phase value can also be confirmed in a similar manner, not just for a single pixel as described above, but also for a vertical line, for example. For example, if the variation is less than ±10°, setting the predetermined width H2 to a pixel width equivalent to a deviation of 10° prevents the predetermined width H2 from becoming unnecessarily large. The larger the predetermined width H2, the narrower the range of stripe phase values φ that can be used to identify the stripe number SN. This prevents confusion between stripe numbers SN and suppresses a decrease in the accuracy of identifying the stripe number SN that would be caused by generating the stripe number identification pattern NP in less than one period.

[0055] In addition, the stripe number identification pattern NP is not limited to being generated as described above by assuming an area that is longer by a predetermined width H2 in both the left and right directions relative to the projection range H1 of the stripe pattern SP, but may also be generated by assuming an area that is longer by a predetermined width only in the left direction relative to the projection range H1 of the stripe pattern SP, or may be generated by assuming an area that is longer by a predetermined width only in the right direction.

[0056] Furthermore, the characteristic configuration of this embodiment etc., in which the stripe number specifying pattern is generated so that its brightness changes in a sine wave pattern in less than one cycle, can also be applied to other embodiments such as the first embodiment etc.

[0057] [Fourth embodiment] Next, a three-dimensional measuring apparatus according to a fourth embodiment of the present invention will be described with reference to the drawings. The fourth embodiment differs from the second embodiment mainly in that a measurement target range is identified from the measurement results of the three-dimensional shape based on the fringe phase value φ, and the three-dimensional shape of the measurement target range is measured based on the fringe number SN and the phase value θ. Therefore, components that are substantially the same as those in the second embodiment are given the same reference numerals, and their description will be omitted.

[0058] The fringe phase value φ obtained as described above is calculated from multiple projection patterns, thereby alleviating the problem of difficulty in identifying projected pixel values related to the projection unit 20 due to the gradual change in brightness in the left-right direction (first direction) caused by the increased number of pixels. This can correspond to the phase value θ obtained according to the three-dimensional shape from an image captured when a single stripe pattern is projected. For example, focusing on a single pattern, the brightness changes twice as much as a simple gradation pattern because the brightness changes from 0 to 255 twice (goes back and forth). Furthermore, while the difference between the left and right pixels is small in areas with a gentle slope, other projection patterns in those areas have a steep slope, making it easier to identify the projected pixel. From another perspective, since projection is performed using a projection unit 20 with pixels, quantization error (quantization distortion) that occurs when converting an analog signal to a digital signal can be reduced. Therefore, the fringe phase value φ obtained as described above can correspond to the phase value θ obtained according to the three-dimensional shape from an image captured when a single stripe pattern is projected. This is because, assuming a single phase-shift pattern, it is necessary to project a sine wave pattern that draws a clean curve. However, the vertical brightness and horizontal projection pixels are digital values, making it impossible to project an ideal shape. While a sine wave pattern is usually reproduced with approximately 20 pixels, this embodiment, assuming the use of an event camera, can reproduce it with approximately 1000 pixels. This increases the horizontal resolution, improving the reproducibility of the sine wave pattern and improving distance accuracy. Therefore, assuming that the stripe number identification pattern NP is projected and captured before the stripe number identification pattern SP is projected and captured, the fringe phase value φ obtained by projecting and capturing the stripe number identification pattern NP during the projection and capture of the stripe number identification pattern SP can be used to obtain simple measurements of the three-dimensional shape of the measurement target, although the measurement accuracy will be lower.

[0059] Therefore, the measurement range to be measured can be identified from the above simplified measurement results, and by measuring the three-dimensional shape limited to the measurement range based on the fringe number SN and phase value θ identified as described above, measurement processing is not performed on ranges that do not need to be measured, thereby shortening the measurement time and reducing the measurement processing load.

[0060] Specifically, assume that a rectangular parallelepiped measurement object R1, as shown in FIG. 10A, is measured from above for picking. In this case, a simplified measurement using the fringe phase value φ obtained by projecting and capturing a stripe number identification pattern NP yields the point cloud data shown in FIG. 10B as a simplified measurement result. This allows the periphery of the edge portion required for picking to be identified as the measurement target range E1 to be measured. Then, by limiting the measurement target range E1 to the fringe number SN based on the phase value θ obtained by projecting and capturing a stripe pattern SP and the fringe phase value φ obtained during the simplified measurement, the fringe number SN is identified. Then, by performing three-dimensional measurement processing based on this fringe number SN and the phase value θ, the point cloud data shown in FIG. 10C can be obtained as a detailed measurement result. In other words, point cloud data for the top surface portion not required for picking is not measured, allowing for faster measurement of the detailed three-dimensional shape of the edge portion required for picking. In Figures 10(B) and (C), the image of the measured point cloud data is shown with a white circle, the part corresponding to the edge of the rectangular parallelepiped measurement object R1 when viewed from above is shown with a dotted dash line, and the outer and inner frames of the measurement object range E1 are shown with thick dashed lines.

[0061] Also, for example, assume that three cylindrical measurement targets R2a, R2b, and R2c placed on a tray, as shown in FIG. 11A, are measured from above for picking. In this case, a simplified measurement using the fringe phase value φ obtained by projecting and capturing the stripe number identification pattern NP can be performed to obtain the point cloud data shown in FIG. 11B as a simplified measurement result. This allows multiple measurement target ranges E2a, E2b, and E2c to be identified around the cylindrical portion required for picking. Then, by limiting the measurement target ranges E2a, E2b, and E2c to the fringe number SN, the fringe number SN is identified based on the phase value θ obtained by projecting and capturing the stripe pattern SP and the fringe phase value φ obtained during the simplified measurement. Then, by performing three-dimensional measurement processing based on the fringe number SN and the phase value θ, the point cloud data shown in FIG. 11C can be obtained as a detailed measurement result. That is, point cloud data for the edge portions of the tray, etc., that are not necessary for the picking operation are not measured, and the detailed three-dimensional shape of the cylindrical portion that is necessary for the picking operation can be measured more quickly. Note that in Figures 11(B) and 11(C), the point cloud data measured at a height corresponding to the top surfaces of the measurement objects R2a, R2b, and R2c and the point cloud data measured at a height corresponding to the top surface of the tray are illustrated in different images, and the portions corresponding to the edge portions of the cylindrical measurement objects R2a, R2b, and R2c and the tray as viewed from above are illustrated by dashed dotted lines, and the outer frames of the measurement target ranges E2a, E2b, and E2c are illustrated by thick dashed lines.

[0062] If the required three-dimensional shape can be measured using the simplified measurement results, the three-dimensional measurement may be performed using the simplified measurement results without projecting and capturing the stripe pattern SP.

[0063] Furthermore, the characteristic configuration of this embodiment and the like, which specifies the measurement target range from the simplified measurement result based on the fringe phase value φ, can also be applied to other embodiments such as the first embodiment.

[0064] The present invention is not limited to the above-described embodiments, and may be embodied as follows, for example. (1) The stripe number specifying patterns NP (NP1 to NP3) are not limited to being generated and projected so that the stripe phase value φ increases linearly from the left end to the right end of the projection range, but may also be generated and projected so that it decreases linearly from the left end to the right end of the projection range, as shown in Fig. 12(A) for example. Furthermore, when there is a pixel range Ps where it is desired to improve the measurement accuracy in the above-mentioned simplified measurement, the stripe number specifying patterns NP (NP1 to NP3) may be generated and projected so that the gradient of the stripe phase value φ in that pixel range Ps suddenly decreases (increases), as shown in Fig. 12(B) for example.

[0065] (2) The number of stripe regions constituting the stripe pattern SP is not limited to four, but may be two, three, or five or more.

[0066] (3) The three-dimensional measuring device 10 is not limited to being attached to the hand of a robot and moving to measure the three-dimensional shape of a measurement object that moves relative to the robot. For example, it may be used in a fixed state to measure the three-dimensional shape of a measurement object that moves on a conveyor line.

[0067] (4) The three-dimensional measuring device 10 may be configured such that the projection unit 20, the imaging unit 30, and the measurement unit 40 are separate entities, and the measurement unit 40 is configured as an information processing terminal capable of wireless or wired communication with the projection unit 20 and the imaging unit 30. [Explanation of symbols]

[0068] 10. Three-dimensional measurement device 11 Control section 20 Projection section 30 Imaging unit 40 Measurement section E1, E2a to E2c Measurement range NP, NP1~NP3 Stripe number identification pattern SP Stripe Pattern SN Stripe Number R, R1, R2a~R2c Measurement object θ phase value (first phase value) φ fringe phase value (second phase value) φi Ideal fringe phase value Δφ, Δφ1~Δφ4 fringe phase value range

Claims

1. a projection unit that projects a predetermined stripe pattern onto the measurement object; an imaging unit that images the measurement object onto which the predetermined stripe pattern is projected; a measurement unit that measures the three-dimensional shape of the measurement object imaged by the imaging unit by a phase shift method; A three-dimensional measuring device comprising: the predetermined stripe pattern is generated so that a plurality of stripe regions, each of which has a luminance that periodically changes in a first direction and a luminance that does not change in a second direction orthogonal to the first direction, are arranged along the first direction, and the predetermined stripe pattern is projected a plurality of times with a phase shift; the projection unit further projects a stripe number identifying pattern for identifying a stripe number that distinguishes the stripe region from other stripe regions; the stripe number specifying pattern is generated so that the luminance varies in a sinusoidal manner in one period or less along the first direction and does not vary in the luminance along the second direction, and is projected a plurality of times with a phase shift; a measurement unit that measures the three-dimensional shape of the object based on a first phase value calculated from an image captured when the predetermined stripe pattern is projected and a second phase value calculated from an image captured when the stripe number identification pattern is projected, and identifies the stripe number for each pixel based on the first phase value calculated from an image captured when the predetermined stripe pattern is projected and the three-dimensional shape of the object

2. the projection unit is configured to control the gradation of light in accordance with a light emission time within a unit time for each pixel; 2. The three-dimensional measuring device according to claim 1, wherein the imaging unit is an event camera that includes an imaging element that outputs event data including two-dimensional point data that identifies the position of a pixel corresponding to a pixel that has experienced a change in luminance when receiving light, and that is capable of generating the captured image from the event data output from the imaging element.

3. 2. The three-dimensional measuring apparatus according to claim 1, wherein the stripe number specifying pattern is generated so that the luminance varies in a sinusoidal manner in less than one period along the first direction.

4. 2. The three-dimensional measuring device according to claim 1, wherein the measurement unit specifies a measurement target range from a measurement result of the three-dimensional shape based on the second phase value, and measures the three-dimensional shape of the measurement target range based on the fringe number and the first phase value.

Images