Three-dimensional measurement device

Abstract

To provide a configuration capable of suppressing degradation in three-dimensional shape measurement accuracy due to an error in a phase value near a stripe boundary.SOLUTION: A measurement unit 40 obtains a phase value θ corresponding to a three-dimensional shape of a measurement target from a photographed image during projection of a stripe pattern SP, acquires a first stripe number SN1 obtained as a stripe number from the photographed image during projection of a first stripe number acquisition pattern NP1, and a second stripe number SN2 obtained as a stripe number from the photographed image during projection of a second stripe number acquisition pattern NP2, measures the three-dimensional shape of the measurement target R by using the phase value θ in a second single-luminance region M2 and the second stripe number SN2 when the phase value θ falls within an error tolerance range centered around a stripe switching phase value θs, and measures the three-dimensional shape of the measurement target R by using the phase value θ in a first single-luminance region M1 and the first stripe number SN1 when the phase value θ falls outside the error tolerance range.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a three-dimensional measuring apparatus that measures the three-dimensional shape of a measurement object. [Background technology]

[0002] Conventionally, as a three-dimensional measuring device for measuring the three-dimensional shape of a measurement object, for example, a device using a phase shift method is known. The phase shift method is a technique for projecting a plurality of stripe pattern images with shifted phases, and performing three-dimensional measurement of the measurement object onto which the stripe pattern images are projected.

[0003] In relation to the technology of performing three-dimensional measurement using the phase shift method, a three-dimensional measuring device disclosed in the following Patent Document 1 is known for generating an image of a measurement object at a higher speed. In this three-dimensional measuring device, a sine wave pattern is adopted as a predetermined stripe pattern for the phase shift method, and an event camera is adopted that outputs event data including two-dimensional point data that identifies the position of a pixel corresponding to a pixel that has changed in luminance when receiving light, and is configured to generate a captured image of the measurement object on which a stripe pattern is projected from the event data. The event camera has a feature that it does not output pixel information with no change in luminance, that is, redundant data (event data) like conventional cameras, and therefore, by realizing a reduction in the amount of data communication and a reduction in the amount of image processing, it is possible to obtain information on the shape of the measurement object at a higher speed. On the other hand, since the event data does not contain the luminance information used in the phase shift method, luminance information (stripe pattern information) is obtained based on the time difference between the occurrence time of event data of a positive luminance change (positive polarity event data) that is output on a pixel-by-pixel basis when light is projected and the occurrence time of event data of a negative luminance change (negative polarity event data) that is output when the light is turned off, making it possible to measure the three-dimensional shape of the object to be measured using event data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-067644 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the phase shift method, the three-dimensional shape of the object is measured based on a phase value calculated according to the three-dimensional shape of the object from the captured image when the stripe pattern is projected and a stripe number specified from the captured image when the stripe number specifying pattern is projected. As the stripe number specifying pattern, for example, a pattern in which the luminance changes stepwise according to the stripe width can be adopted, but there is a possibility that an erroneous measurement result is obtained for pixels located near the boundary of the stripe region (the boundary between the light color region and the dark color region: stripe boundary).

[0006] For example, if stripe number "k+1" is identified at an imaging pixel near the boundary between stripe numbers "k" and "k+1," but the phase value is calculated to be slightly shifted in the increasing direction from the value that is actually required due to quantization error, etc., a phase value shifted by approximately one stripe will be erroneously identified. If an incorrect distance value is calculated due to such erroneous identification, there is a problem that the measurement accuracy of the 3D measurement will decrease.

[0007] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a configuration that can suppress a decrease in measurement accuracy of three-dimensional shapes caused by errors in phase values ​​near fringe boundaries. [Means for solving the problem]

[0008] In order to achieve the above object, one aspect of the present invention is a projection unit (20) that projects a predetermined pattern onto a measurement object (R); an imaging unit (30) that captures an image of the measurement object onto which the predetermined pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change upon receiving light; a measurement unit (40) that measures a three-dimensional shape of the measurement object imaged by the imaging unit; A three-dimensional measuring device (10) comprising: the predetermined pattern includes a predetermined stripe pattern (SP) that is generated so that a plurality of stripe regions whose luminance changes in a first direction and whose luminance does not change in a second direction perpendicular to the first direction are arranged along the first direction, and a first stripe number acquisition pattern (NP1) and a second stripe number acquisition pattern (NP1) that are projected toward the same projection range as the predetermined stripe pattern and are used to identify a stripe number (SN) that distinguishes the stripe region from other stripe regions; the first stripe number acquisition pattern is generated such that a plurality of first single luminance regions (M1) having the same shape as the stripe region are arranged along the first direction, and each of the first single luminance regions has a different luminance; the second stripe number acquisition pattern is generated such that a plurality of second single luminance areas (M2) having a width value in the first direction equal to or shorter than that of the first single luminance area are arranged along the first direction so as to be centered on a stripe boundary between the stripe areas, and the luminance of each of the second single luminance areas is different; The measurement unit is A phase value (θ) corresponding to the three-dimensional shape of the measurement object is obtained from the captured image when the predetermined stripe pattern is projected, and a first stripe number (SN1) obtained as the stripe number from the captured image when the first stripe number acquisition pattern is projected and a second stripe number (SN2) obtained as the stripe number from the captured image when the second stripe number acquisition pattern is projected are obtained; When the ideal value of the phase value at the fringe boundary is the fringe switching phase value (θs), if the phase value is within a predetermined range centered on the fringe switching phase value, the three-dimensional shape of the measurement object is measured using the phase value and the second fringe number, and if the phase value is outside the predetermined range, the three-dimensional shape of the measurement object is measured using the phase value and the first fringe number.

[0009] Another aspect of the present invention is a projection unit (20) that projects a predetermined pattern onto a measurement object (R); an imaging unit (30) that captures an image of the measurement object onto which the predetermined pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change when light is received; a measurement unit (40) that measures a three-dimensional shape of the measurement object imaged by the imaging unit; A three-dimensional measuring device (10) comprising: the predetermined pattern includes a predetermined stripe pattern (SP) that is generated so that a plurality of stripe regions, whose luminance changes in a first direction and whose luminance does not change in a second direction perpendicular to the first direction, are arranged along the first direction, and a stripe number identifying pattern (NP3) that is projected toward the same projection range as the predetermined stripe pattern and that identifies a stripe number (SN) that distinguishes the stripe region from other stripe regions; the stripe number specifying pattern is generated such that a plurality of regions each including a light-off control region (MS2) in which the timing of turning off the light differs from region to region and a light-on control region (MS1) in which the timing of turning on the light differs from region to region are arranged in the first direction, the centers of either the light-off control region or the light-on control region being aligned with a stripe boundary between the stripe regions, and the plurality of regions are arranged in the first direction, The measurement unit determines a phase value (θ) corresponding to the three-dimensional shape of the object to be measured from the captured image when the specified stripe pattern is projected, and obtains a stripe number from the captured image when the stripe number identifying pattern is projected, and measures the three-dimensional shape of the object to be measured using the phase value and the stripe number. The symbols in parentheses above indicate the corresponding relationship with the specific means described in the embodiments described later. Effect of the Invention

[0010] In the present invention, the first stripe number acquisition pattern is generated by arranging a plurality of first single luminance areas of the same shape as the stripe areas along a first direction, with each first single luminance area having a different luminance, and the second stripe number acquisition pattern is generated by arranging a plurality of second single luminance areas along the first direction, with each second single luminance area having the same or shorter width value in the first direction than the first single luminance areas, with each second single luminance area centered on a stripe boundary between the stripe areas, with each second single luminance area having a different luminance. The measurement unit determines a phase value corresponding to the three-dimensional shape of the object to be measured from the captured image when a predetermined stripe pattern is projected, and obtains a first stripe number obtained as a stripe number from the captured image when a first stripe number acquisition pattern is projected, and a second stripe number obtained as a stripe number from the captured image when a second stripe number acquisition pattern is projected.When the ideal value of the phase value at the stripe boundary is the stripe switching phase value, if the phase value is within a predetermined range centered on the stripe switching phase value, the measurement unit measures the three-dimensional shape of the object to be measured using the phase value and the second stripe number, and if the phase value is outside the predetermined range, the measurement unit measures the three-dimensional shape of the object to be measured using the phase value and the first stripe number.

[0011] As a result, when the phase value is within a predetermined range centered on the stripe switching phase value, i.e., near the stripe boundary, the three-dimensional shape of the measurement object is measured using the phase value and the second stripe number. Since the second stripe number acquisition pattern for acquiring the second stripe number is arranged so that each second single luminance region is centered on the stripe boundary, even if an error occurs in the phase value near the stripe boundary, the above-mentioned shift of approximately one stripe does not occur. Therefore, it is possible to suppress a decrease in measurement accuracy of the three-dimensional shape caused by an error in the phase value near the stripe boundary.

[0012] The above-mentioned predetermined range may be set according to an expected error range of the phase value, and the second single-luminance region may be set so that its width value in the first direction is shorter than that of the first single-luminance region, in accordance with pixels whose phase values ​​may be within the above-mentioned predetermined range.

[0013] As a result, when the second stripe number is acquired, the three-dimensional shape is measured using the phase value and the second stripe number, and when the second stripe number is not acquired, the three-dimensional shape is measured using the phase value and the first stripe number. This not only eliminates the need to determine whether to use the second stripe number or the first stripe number, but also prevents the second stripe number from being acquired unnecessarily, thereby reducing the processing load related to the measurement process.

[0014] In addition, in the present invention, the stripe number specifying pattern is generated so that a plurality of regions, each of which is an off control region in which the off timing differs from region to region and an on control region in which the on timing differs from region to region, are arranged in a first direction with the center of either the off control region or the on control region aligned with a stripe boundary between the stripe regions. The measurement unit obtains a phase value corresponding to the three-dimensional shape of the measurement object from the captured image when a predetermined stripe pattern is projected, and obtains a stripe number from the captured image when the stripe number specifying pattern is projected, and measures the three-dimensional shape of the measurement object using the phase value and the stripe number.

[0015] As a result, for example, when imaging a stripe number specifying pattern that is generated and projected so that the center of the light-on control area is aligned with the stripe boundary, positive polarity (positive luminance change) event data is output in the light-on control area near the stripe boundary, and negative polarity (negative luminance change) event data is output in the light-off control area. In this case, since each light-on control area is arranged so that the center is the stripe boundary, even if an error occurs in the phase value near the stripe boundary, the phase value in the light-on control area is used, so the above-mentioned deviation of approximately one stripe does not occur. The same is true for a stripe number specifying pattern that is generated and projected so that the center of the light-off control area is aligned with the stripe boundary. Therefore, it is possible to suppress a decrease in measurement accuracy of a three-dimensional shape caused by an error in the phase value near the stripe boundary. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a schematic configuration of a three-dimensional measuring apparatus according to a first embodiment. [Diagram 2]FIG. 1 is an explanatory diagram illustrating a state in which a stripe pattern for a general phase shift method is projected onto a measurement object. [Diagram 3] 1A and 1B are diagrams illustrating three-dimensional measurement by a phase shift method. [Figure 4] 11 is an explanatory diagram illustrating the relationship between the time difference between the output timing of positive polarity event data and negative polarity event data output during a unit time and the luminance value. FIG. [Diagram 5] FIG. 5(A) is an explanatory diagram illustrating three types of sine wave patterns whose phases are shifted by 2π / 3 based on the initial phase, and FIG. 5(B) is an explanatory diagram illustrating the relationship between the phase value of the initial phase in FIG. 5(A) and the fringe number. [Figure 6] FIG. 6(A) is an explanatory diagram illustrating the relationship between the phase value in FIG. 5(B) and the ON time of the stripe number identification pattern and the stripe number, and FIG. 6(B) is an explanatory diagram illustrating a problem caused by an error in the phase value. [Figure 7] Figure 7(A) is an explanatory diagram explaining the relationship between the phase value, the ON time of the pattern for obtaining the first stripe number, and the first stripe number, and Figure 7(B) is an explanatory diagram explaining the relationship between the phase value, the ON time of the pattern for obtaining the second stripe number, and the second stripe number. [Figure 8] An explanatory diagram explaining the relationship between the stripe number acquisition pattern and each stripe area in the second embodiment, where FIG. 8(A) shows an example of a first stripe number acquisition pattern, and FIG. 8(B) shows an example of a second stripe number acquisition pattern. [Figure 9] 13 is an explanatory diagram illustrating the relationship between a stripe number specifying pattern and each stripe area in the third embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] [First embodiment] Hereinafter, a first embodiment of a three-dimensional measuring device according to the present invention will be described with reference to the drawings. The three-dimensional measuring device 10 according to the present embodiment is a device for measuring the three-dimensional shape of a measurement object R, and is configured to include a control unit 11 for overall control, a projection unit 20 for projecting a predetermined pattern for three-dimensional measurement onto the measurement object R, an imaging unit 30 for capturing an image of the measurement object R onto which the predetermined pattern is projected, and a measurement unit 40 for measuring the three-dimensional shape of the measurement object R from the captured image, as shown in Figs. 1 and 2. The three-dimensional measuring device 10 configured in this manner measures the three-dimensional shape of the measurement object R, such as a workpiece, which is attached to the hand of a robot, for example, and moves relatively to the hand at high speed. Here, the relative movement refers to the relative movement between the movement of the three-dimensional measuring device 10 attached to the hand of the robot and the high-speed movement of the measurement object R. When the position of the three-dimensional measuring device 10 is fixed, the relative movement is the movement of the measurement object R.

[0018] For convenience, a typical stripe pattern having up to 13 stripes is shown in a simplified form in Fig. 2. More specifically, a typical stripe pattern is represented as a sine wave pattern, so the light and dark parts of the stripe pattern have the same width, but for convenience, the dark parts are shown as lines with smaller widths in Fig. 2. Also, the number of stripes is 13 or more in the embodiment, but is abbreviated to 13.

[0019] The control unit 11 is mainly composed of a microcomputer and has a CPU, a system bus, an input / output interface, etc., and constitutes an information processing device together with a storage unit consisting of a ROM, a RAM, a non-volatile memory, etc. In addition to programs related to robot control, the storage unit stores in advance programs related to control of the projection unit 20 and programs for executing control processing using the three-dimensional measurement results by the measurement unit 40, etc., which are executable by the control unit 11.

[0020] The projection unit 20 is a so-called DLP projector, and is controlled by the control unit 11 to project a predetermined pattern by reflecting light from a light source by a DMD element. The DMD element is an array of fine mirrors corresponding to each projection pixel of an image projected on a screen, and is configured to change the angle of each mirror to turn on / off the light emitted to the screen in microseconds. Therefore, the state is switched to a light-projecting state by changing reflection OFF to reflection ON for each mirror, and the state is switched to an off state by changing reflection ON to reflection OFF for each mirror. That is, the projection unit 20 functions to project a predetermined pattern by controlling the ON / OFF of the reflection of incident light by the DMD, which has multiple mirrors arranged in an array, for each mirror by the control unit 11. Therefore, the gradation (brightness) of the reflected light is changed depending on the ratio of the time each mirror is turned on and off, making it possible to display gradations based on the image data of the image to be projected. In this embodiment, the projection unit 20 includes mirrors that correspond to k×l projection pixels (for example, 1140×912), with the upper left being (1, 1) and the lower right being (k, l).

[0021] In such a configuration, the longer the light emission time (the time from reflection ON to reflection OFF) of the single pulse light emitted once within the unit time secured for each light emission state, the brighter the light emission state becomes, so that the light emission state can be specified according to the light emission time. For example, when R (red), G (green), and B (blue) colors are prepared as light incident on the DMD element, an R light emission state in which R is reflected by a mirror and light is emitted, a G light emission state in which G is reflected by a mirror and light is emitted, and a B light emission state in which B is reflected by a mirror and light is emitted are repeated in a short predetermined cycle, and each light emission time is individually adjusted, so that a color image can be projected. For this reason, the control unit 11 functions to set the reflection ON / OFF timing within the unit time for each mirror according to a predetermined pattern described later.

[0022] The imaging unit 30 is a so-called event camera, and is equipped with an imaging element that outputs event data (specifically, data including two-dimensional point data, time, and polarity of brightness change) including two-dimensional point data that identifies the position of an imaging pixel corresponding to an imaging pixel that has undergone a brightness change when light is received, and is configured to be able to generate an imaging image from the event data output from the imaging element. For this reason, the imaging unit 30 outputs event data of positive polarity (positive brightness change) when a brightness change occurs that makes the imaging pixel brighter by receiving light, and outputs event data of negative polarity (negative brightness change) when a brightness change occurs that makes the imaging pixel darker by extinguishing the light. Image data of the measurement object R can be generated by plotting the two-dimensional point data of a plurality of event data output within a certain period as points on a predetermined plane, and the imaging unit 30 is configured to output the image data or event data generated in this way to the measurement unit 40.

[0023] The measurement unit 40 is controlled by the control unit 11 to perform a three-dimensional measurement process for measuring the three-dimensional shape of the measurement object R based on an image captured by the imaging unit 30 of the measurement object R in a state where a predetermined pattern is projected from the projection unit 20. In the three-dimensional measurement process in this embodiment, a phase shift method is adopted.

[0024] Generally, as a stripe pattern for the phase shift method (hereinafter also referred to as stripe pattern SP), a sine wave pattern specified by the brightness value I(x, y, n) of the following formula (1) is adopted in order to obtain a phase value θ corresponding to a value distorted according to the surface shape of the measurement object R based on a grating image (stripe image) captured of the measurement object R in a projected state. The stripe pattern SP is projected so that the brightness periodically changes in a first direction (left-right direction in the example of FIG. 2) and does not change in a second direction (up-down direction in the example of FIG. 2) perpendicular to the first direction. When the number of phase shifts is N, the brightness value I(x, y, n) of N phase-shifted grating images (stripe images) is expressed by formula (1). I(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N}+b(x,y) (1) Here, point (x, y) is a point in the lattice image, a(x, y) indicates the luminance amplitude, b(x, y) indicates the background luminance, and θ(x, y) indicates the phase value of the lattice where n=0. The distance z to point (x, y) is measured according to the phase value θ(x, y) calculated from the luminance values ​​I(x, y, n) of N lattice images and the separately determined stripe number SN.

[0025] Specifically, for example, when three grating images are obtained in one period of the above-mentioned R, G, and B light emission states, the luminance value I(x, y, 0) in the R light emission state, the luminance value I(x, y, 1) in the G light emission state, and the luminance value I(x, y, 2) in the B light emission state are obtained from the captured image with N=3. In this case, the stripe pattern SP projected from the projection unit 20 is configured such that the phases of the sine wave pattern consisting of only R, the sine wave pattern consisting of only G, and the sine wave pattern consisting of only B are shifted by 2π / 3.

[0026] By obtaining the luminance values ​​I(x,y,0), I(x,y,1), and I(x,y,2) at the point (x,y) in the captured image, the phase value θ(x,y) is calculated using the above formula (1), and the distance z to the point (x,y) is measured according to the phase value θ(x,y) thus calculated. By measuring the distance z to each point (x,y) of the captured measurement object R in this manner, the three-dimensional shape of the measurement object R can be measured.

[0027] For example, when the distance z of point Pr1 in FIG. 3 is obtained, the phase value θ of point Pr1 and information on which stripe the point Pr1 is (stripe number SN) are obtained from N captured images of the imaging unit 30 in a state where the stripe pattern SP is shifted and projected from the projection unit 20 N times. When the angle θp1 at the projection unit 20 and the angle θc1 at the imaging unit 30 are obtained from the phase value θ and stripe number SN obtained in this way, the distance z of point Pr1 can be obtained by triangulation since the distance (parallax Lpc) between the projection unit 20 and the imaging unit 30 is known. Similarly, the distance z of point Pr2 in FIG. 3 can be obtained by triangulation based on the angle θp2 at the projection unit 20 and the angle θc2 at the imaging unit 30 obtained from the phase value θ of point Pr2 obtained from the N captured images and the stripe number SN. By performing this calculation over the entire measurement area, three-dimensional measurement can be performed.

[0028] In this embodiment, an event camera is used as an imaging unit for accurately imaging the measurement target R that moves relatively at high speed. In such a configuration, event data corresponding to imaging pixels where a luminance change has occurred is output, but the event data does not include a luminance value, so that it is not possible to directly obtain the luminance values ​​(for example, the above-mentioned I(x,y,0), I(x,y,1), and I(x,y,2)) required for the phase shift method.

[0029] For this reason, in this embodiment, stripe pattern information Is(x,y,n) corresponding to luminance value I(x,y,n) is calculated as the time difference between the output timing of positive polarity event data that is output first during unit time T (see the upward arrow in FIG. 4) and the output timing of negative polarity event data that is output next (see the downward arrow in FIG. 4), as shown in FIG. 4. This is because, after positive polarity event data is output at emission start timing t1, negative polarity event data is output at emission end timing t2, and therefore, the longer the time difference, the higher the luminance value at that imaging pixel.

[0030] The stripe pattern information Is(x,y,n) obtained in this manner is affected by the three-dimensional shape of the measurement object R, so the phase value θ(x,y) can be obtained using the following equation (2). Is(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N} +b(x,y) (2)

[0031] Next, a method for identifying the stripe number SN in this embodiment will be described below. In the phase shift method, it is necessary to determine which stripe region the phase value θ measured as described above belongs to, and a stripe number SN is specified to distinguish the stripe region to which it belongs from other stripe regions. In the conventional phase shift method, for example, the phase of each captured image of three types of sine wave patterns projected so that the phase is shifted by 2π / 3 based on the initial phase as shown in FIG. 5(A) is analyzed in terms of captured pixel units. In this case, the phase value θ of the initial phase at each projected pixel periodically changes so as to decrease at the boundary (striped boundary) of the stripe region as shown in FIG. 5(B). Note that FIG. 5 and FIG. 6 (described later) and the like show only a part of the projected pixel range for convenience.

[0032] The stripe number identifying pattern for identifying the stripe number SN is projected so that its brightness changes according to the stripe width. For example, in the phase shift method using an event camera, as shown in FIG. 6(A), a stripe number identifying pattern is projected such that the area projected onto stripe number SN=k is turned off for 10 ms×k. This makes it possible to identify the stripe number SN based on the output timing of negative polarity event data from the imaging unit 30, i.e., the ON time during which the lighted state is maintained. In this example, the stripe number SN of an imaging pixel with an ON time of 20 ms (an imaging pixel from which negative polarity event data is output in 20 ms) is identified as "2."

[0033] In such a conventional identification method, for pixels located near a fringe boundary, a slight detection error in the phase value θ may result in erroneous measurement results.

[0034] In the case of FIG. 6(A), at an imaging pixel near the boundary between stripe numbers "1" and "2," stripe number "2" is identified. However, if the phase value θ is calculated to be slightly shifted in the increasing direction from the originally calculated value due to a quantization error or the like, a phase value θ that is shifted by approximately one stripe is erroneously identified.

[0035] Specifically, for example, assume that the phase value θ near the stripe boundary of stripe number "2" should be calculated as 1.57 [rad] (see symbol θ1 in FIG. 6(B)), but is calculated as 1.6 [rad], shifting slightly in the increasing direction due to quantization error, etc. In this case, as shown in the example of FIG. 6(B), a projection pixel (see symbol θ2 in FIG. 6(B)) with a phase value θ of 1.6 [rad] is identified within the projection pixel range identified by stripe number "2", and the phase value θ shifted by approximately one stripe is erroneously identified.

[0036] To prevent such erroneous identification, in this embodiment, a first stripe number acquisition pattern NP1 and a second stripe number acquisition pattern NP2 are adopted as patterns projected from the projection unit 20 toward the same projection range as the stripe pattern SP in order to identify the stripe number. Then, a first stripe number SN1 obtained as the stripe number from the captured image when the first stripe number acquisition pattern NP1 is projected, and a second stripe number SN2 obtained as the stripe number from the captured image when the second stripe number acquisition pattern NP2 is projected are each obtained for each pixel.

[0037] The first stripe number acquisition pattern NP1 is generated by arranging a plurality of single luminance areas (hereinafter, also referred to as first single luminance areas M1) having the same shape as the stripe areas of the stripe pattern SP along the first direction, and the luminance of each first single luminance area M1 is different. Specifically, as shown in FIG. 7(A), the first stripe number acquisition pattern NP1 is generated as a pattern similar to a conventional stripe number identification pattern, and the luminance changes stepwise to become brighter according to the stripe width. In the example of FIG. 7(A), the first stripe number acquisition pattern NP1 is generated so that the area projected on the first stripe number SN1=k is turned off for 10ms×k. Also, the first first single luminance area M1(1) has the first stripe number SN1=1, the second first single luminance area M1(2) has the first stripe number SN1=2, and the third first single luminance area M1(3) has the first stripe number SN1=3.

[0038] The second stripe number acquisition pattern NP2 is generated such that a plurality of single luminance regions (hereinafter also referred to as second single luminance regions M2) having the same width value in the first direction as the first single luminance region M1 are arranged along the first direction so as to be centered on the stripe boundary between the stripe regions, and the luminance of each second single luminance region M2 is different. Specifically, as shown in FIG. 7B, the second stripe number acquisition pattern NP2 is generated such that the first stripe number acquisition pattern NP1 is shifted in the width direction (to the right in FIG. 7) by half the stripe width. The 0th second single luminance region M2(0) has the second stripe number SN2=0, the 1st second single luminance region M2(1) has the second stripe number SN2=1, the 2nd second single luminance region M2(2) has the second stripe number SN2=2, and the 3rd second single luminance region M2(3) has the second stripe number SN2=3.

[0039] In the example of Fig. 7(B), the area projected onto the second stripe number SN2=k is set to be turned off in 10 ms x k. Therefore, on the side where the phase value θ increases relative to the stripe boundary (left side of Fig. 7), the stripe number SN matches the second stripe number SN2, and on the side where the phase value θ decreases relative to the stripe boundary (right side of Fig. 7), the stripe number SN matches the second stripe number SN2+1.

[0040] Then, by the stripe number acquisition process performed by the measurement unit 40 when calculating the phase value θ, the first stripe number SN1 is acquired for each pixel from the image captured when the first stripe number acquisition pattern NP1 is projected, and the second stripe number SN2 is acquired for each pixel from the image captured when the second stripe number acquisition pattern NP2 is projected.

[0041] In the measurement unit 40, when the first stripe number SN1 and the second stripe number SN2 are acquired for each pixel as described above, if the phase value θ is within the error tolerance, a process is performed to measure the three-dimensional shape of the measurement object R using the phase value θ in the second single luminance region M2 and the second stripe number SN2, and if the phase value θ is outside the error tolerance, a process is performed to measure the three-dimensional shape of the measurement object R using the phase value θ in the first single luminance region M1 and the first stripe number SN1. In this embodiment, the error tolerance is an example of a "predetermined range", and is set according to an expected error range E of the phase value θ centered on the stripe switching phase value θs, where the ideal value of the phase value θ at the stripe boundary is the stripe switching phase value θs (see FIG. 7(B)).

[0042] Specifically, when the first fringe number acquisition pattern NP1 illustrated in Fig. 7(A) and the second fringe number acquisition pattern NP2 illustrated in Fig. 7(B) are employed, if the calculated phase value θ satisfies the relationship in the following formula (3), the second fringe number SN2 is used for the three-dimensional measurement. However, if θs+E>2π, the second fringe number SN2 is used for the three-dimensional measurement even if the relationship in the following formula (4) is satisfied. θs+E≧θ≧θs (3) θs+E-2π≧θ (4)

[0043] Furthermore, if the calculated phase value θ satisfies the relationship of the following formula (5), the fringe number (SN2+1) obtained by adding "1" to the second fringe number SN2 is used for the three-dimensional measurement. However, if θs-E<0, the fringe number (SN2+1) is used for the three-dimensional measurement even if the relationship of the following formula (6) is satisfied. θs-E≦θ<θs (5) θs-E+2π≦θ (6)

[0044] On the other hand, if the calculated phase value θ does not satisfy any of the above expressions (3) to (6), the first fringe number SN1 is used for the three-dimensional measurement.

[0045] As described above, in the three-dimensional measuring device 10 of this embodiment, the first stripe number acquisition pattern NP1 is generated by arranging a plurality of first single-luminance regions M1 of the same shape as the stripe regions of the stripe pattern SP along a first direction, and each first single-luminance region M1 has a different luminance, and the second stripe number acquisition pattern NP2 is generated by arranging a plurality of second single-luminance regions M2 having the same width value in the first direction as the first single-luminance region M1 along the first direction, each centered on a stripe boundary between the stripe regions, and each second single-luminance region M2 has a different luminance. The measurement unit 40 determines a phase value θ corresponding to the three-dimensional shape of the object to be measured from the image captured when the stripe pattern SP is projected, and also obtains a first stripe number SN1 obtained as a stripe number from the image captured when the first stripe number acquisition pattern NP1 is projected, and a second stripe number SN2 obtained as a stripe number from the image captured when the second stripe number acquisition pattern NP2 is projected.If the phase value θ is within an acceptable error range centered on the stripe switching phase value θs, the measurement unit 40 measures the three-dimensional shape of the object to be measured R using the phase value θ in the second single-luminance region M2 and the second stripe number SN2, and if the phase value θ is outside the above-mentioned acceptable error range, the measurement unit 40 measures the three-dimensional shape of the object to be measured R using the phase value θ in the first single-luminance region M1 and the first stripe number SN1.

[0046] As a result, when the phase value θ is within the error tolerance centered on the stripe switching phase value θs, that is, near the stripe boundary, the three-dimensional shape of the measurement object R is measured using the phase value θ in the second single luminance region M2 and the second stripe number SN2. Since the second stripe number acquisition pattern NP2 for acquiring the second stripe number SN2 is arranged so that each second single luminance region M2 is centered on the stripe boundary, even if an error occurs in the phase value θ near the stripe boundary, the phase value θ in the second single luminance region M2 is used, so that the above-mentioned shift of approximately one stripe does not occur. Therefore, it is possible to suppress the decrease in measurement accuracy of the three-dimensional shape caused by the error in the phase value θ near the stripe boundary. In particular, since the second stripe number acquisition pattern NP2 is generated so as to shift the first stripe number acquisition pattern NP1 in the width direction by half the stripe width, not only can the pattern design be simplified, but it can also be applied to areas where the phase is discontinuous.

[0047] [Second embodiment] Next, a three-dimensional measuring apparatus according to a second embodiment of the present invention will be described with reference to the drawings. The second embodiment is different from the first embodiment mainly in that the second fringe number SN2 is not acquired when the phase value θ is outside the error tolerance range. Therefore, the same reference numerals are used for the components substantially the same as those in the first embodiment, and the description thereof will be omitted.

[0048] The first stripe number acquisition pattern NP1 in this embodiment is generated in the same manner as in the first embodiment described above, as shown in Figure 8(A), and the second stripe number acquisition pattern NP2 is generated so that the width value of the second single luminance area M2 is shorter than the first single luminance area M1, as shown in Figure 8(B).

[0049] Specifically, the second single luminance region M2 is set so that its width in the first direction is shorter than that of the first single luminance region M1, in accordance with the projected pixels whose phase value θ can be within the above-mentioned error tolerance. Therefore, the second stripe number acquisition pattern NP2 is generated so that the second single luminance region M2 and the unlit region M3 from which the second stripe number SN2 is not acquired are alternately arranged along the first direction.

[0050] Therefore, in the measurement unit 40 of this embodiment, when the second stripe number SN2 is acquired, a process is performed to measure the three-dimensional shape of the measurement object R using the phase value θ in the second single-luminance region M2 and the second stripe number SN2, and when the second stripe number SN2 is not acquired, a process is performed to measure the three-dimensional shape of the measurement object R using the phase value θ in the first single-luminance region M1 and the first stripe number SN1.

[0051] Specifically, when the second fringe number SN2 is acquired, if the calculated phase value θ satisfies the relationship in the following formula (7), the second fringe number SN2 is used for the three-dimensional measurement. However, if θs+E>2π, the second fringe number SN2 is used for the three-dimensional measurement even if the relationship in the above formula (4) is satisfied. θ ≥ θs (7)

[0052] Furthermore, when the second fringe number SN2 is acquired, if the calculated phase value θ satisfies the relationship in the following formula (8), the fringe number (SN2+1) obtained by adding "1" to the second fringe number SN2 is used for the three-dimensional measurement. However, if θs-E<0, the fringe number (SN2+1) is used for the three-dimensional measurement even if the relationship in the above formula (6) is satisfied. θ<θs (8)

[0053] On the other hand, if the calculated phase value θ does not satisfy any of the above expressions (4), (6), (7), or (8), the first fringe number SN1 is used for the three-dimensional measurement.

[0054] As described above, in the three-dimensional measuring device 10 according to this embodiment, the above-mentioned error tolerance is set in accordance with the expected error range E of the phase value θ, and the second single-luminance region M2 is set so that its width value in the first direction is shorter than that of the first single-luminance region M1, in accordance with the projected pixels whose phase value θ can fall within the above-mentioned error tolerance.

[0055] As a result, when the second stripe number SN2 is acquired, the three-dimensional shape is measured using the phase value θ in the second single luminance region M2 and the second stripe number SN2, and when the second stripe number SN2 is not acquired, the three-dimensional shape is measured using the phase value θ in the first single luminance region M1 and the first stripe number SN1. This not only eliminates the need to determine whether to use the second stripe number SN2 or the first stripe number SN1, but also reduces the processing load for the measurement process because the second stripe number SN2 is not acquired unnecessarily. In particular, in each unlit region M3 of the second stripe number acquisition pattern NP2, it is not necessary to acquire event data, so the amount of events can be reduced (event saturation suppression, data transfer amount reduction, etc.).

[0056] [Third embodiment] Next, a three-dimensional measuring apparatus according to a third embodiment of the present invention will be described with reference to the drawings. The third embodiment differs from the second embodiment mainly in that, in order to identify the stripe number SN, one stripe number identification pattern is projected without projecting the first stripe number acquisition pattern NP1 and the second stripe number acquisition pattern NP2, respectively. Therefore, the same reference numerals are used for the components that are substantially the same as those in the second embodiment, and the description thereof will be omitted.

[0057] In this embodiment, the characteristics of the event camera are utilized to combine the output timing of positive polarity event data and the output timing of negative polarity event data, thereby identifying the stripe number SN by projecting one stripe number identifying pattern NP3.

[0058] The stripe number identifying pattern NP3 in this embodiment is generated so that the area corresponding to the second single luminance area M2 in the second embodiment is set as a light-on control area MS1 in which the stripe number SN is obtained based on the output timing of positive polarity event data, and the other areas are set as a light-off control area MS2 in which the stripe number SN is obtained based on the output timing of negative polarity event data.

[0059] Specifically, the stripe number identification pattern NP3 is generated such that a plurality of regions, each of which is formed by arranging the off control region MS2 and the on control region MS1 ​​in the first direction, are arranged along the first direction with the center of the on control region MS1 ​​aligned with the stripe boundary, as illustrated in Fig. 9. For convenience, in Fig. 9, positive polarity event data is shown in white and negative polarity event data is shown in black.

[0060] The lighting control region MS1 ​​is set so that the lighting timing differs from region to region, more specifically, the lighting timing is set so that the lighting timing is delayed stepwise from region to region. The extinguishing control region MS2 is set so that the extinguishing timing differs from region to region, more specifically, the extinguishing timing is set so that the extinguishing timing is delayed stepwise from region to region. As a result, when imaging the stripe number identification pattern NP3, the imaging pixels imaging the lighting control region MS1 ​​near the stripe boundary output positive polarity event data, and the imaging pixels imaging the extinguishing control region MS2 away from the stripe boundary output negative polarity event data.

[0061] In the example of Fig. 9, the light-on control area MS1 centered on the stripe boundary between the stripe area with stripe number SN=k and the stripe area with stripe number SN=k+1 is set to be lighted for 10 ms x k, and the light-off control area MS2 projected onto the stripe area with stripe number SN=k is set to be turned off for 10 ms x k. Therefore, on the side where the phase value θ increases with respect to the stripe boundary (left side of Fig. 9), the stripe number SN to be found matches the stripe number SN obtained from the output timing of the positive polarity event data, and on the side where the phase value θ decreases with respect to the stripe boundary (right side of Fig. 9), the stripe number SN to be found matches the stripe number SN+1 obtained from the output timing of the positive polarity event data.

[0062] Then, by the stripe number acquisition process performed by the measurement unit 40 when calculating the phase value θ, the stripe number SN is acquired for each pixel from the captured image when the stripe number identifying pattern NP3 is projected.

[0063] Therefore, in the measurement unit 40 according to this embodiment, when positive polarity event data is output, if the calculated phase value θ satisfies the relationship of the above formula (7), the fringe number SN acquired from the output timing of the positive polarity event data is used for three-dimensional measurement. However, if θs+E>2π, the fringe number SN acquired as described above is used for three-dimensional measurement even if the relationship of the above formula (4) is satisfied.

[0064] Furthermore, when positive polarity event data is output, if the calculated phase value θ satisfies the relationship in formula (8) above, the fringe number (SN+1) obtained by adding "1" to the fringe number SN obtained from the output timing of the positive polarity event data is used for three-dimensional measurement. However, if θs-E<0, the fringe number (SN+1) is used for three-dimensional measurement even if the relationship in formula (6) above is satisfied.

[0065] On the other hand, when negative polarity event data is output, the fringe number SN obtained from the output timing of the negative polarity event data is used for three-dimensional measurement.

[0066] As described above, in the three-dimensional measuring device 10 according to this embodiment, the stripe number identifying pattern NP3 is generated so that a plurality of regions, each of which is an arrangement of an off-control region MS2, in which the timing of turning off the light differs from region to region, and an on-control region MS1, in which the timing of turning on the light differs from region to region, are arranged in the first direction with the center of the on-control region MS1 ​​aligned with the stripe boundary. The measuring unit 40 obtains a phase value θ corresponding to the three-dimensional shape of the measurement object from the captured image when the stripe pattern SP is projected, and obtains a stripe number SN from the captured image when the stripe number identifying pattern NP3 is projected, taking into consideration the polarity of the event data, and measures the three-dimensional shape of the measurement object using the phase value θ and the stripe number SN.

[0067] In this way, since each lighting control area MS1 is arranged so as to be centered on a stripe boundary, even if an error occurs in the phase value θ near the stripe boundary, the phase value θ in the lighting control area MS1 is used, so the above-mentioned deviation of about one stripe does not occur. Therefore, it is possible to suppress a decrease in measurement accuracy of the three-dimensional shape caused by an error in the phase value near the stripe boundary. In particular, since there is no need to project two patterns, the first stripe number acquisition pattern NP1 and the second stripe number acquisition pattern NP2, as in the other embodiments described above, it is possible to shorten the measurement time.

[0068] In addition, the stripe number specifying pattern NP3 is not limited to being generated so that the center of each light-on control region MS1 ​​is aligned with the stripe boundary, but may be generated so that the center of each light-off control region MS2 is aligned with the stripe boundary. Even in this case, by using the phase value θ in the light-off control region MS2, the above-mentioned shift of approximately one stripe does not occur, so that the measurement accuracy of the three-dimensional shape can be suppressed from decreasing.

[0069] The present invention is not limited to the above-described embodiments, and may be embodied as follows, for example. (1) The first stripe number acquisition pattern NP1 and the second stripe number acquisition pattern NP2 may not only be generated so that the stripe number SN can be identified based on the output timing of negative polarity event data (the ON time during which the lit state is maintained), but may also be generated so that the stripe number SN can be identified based on the output timing of positive polarity event data (the OFF time during which the unlit state is maintained).

[0070] (2) The first stripe number acquisition pattern NP1 is not limited to being generated as a pattern in which the luminance gradually changes to become brighter for each first single luminance region M1, but may be generated so that the luminance differs for each first single luminance region M1 so that the first single luminance region M1 can be distinguished. Similarly, the second stripe number acquisition pattern NP2 is not limited to being generated as a pattern in which the luminance gradually changes to become brighter for each second single luminance region M2, but may be generated so that the luminance differs for each second single luminance region M2 so that the second single luminance region M2 can be distinguished.

[0071] (3) The three-dimensional measuring device 10 is not limited to being attached to the hand of a robot and moving to measure the three-dimensional shape of a measurement object that moves relative to the robot. For example, it may be used in a fixed state to measure the three-dimensional shape of a measurement object that moves on a conveyor line.

[0072] (4) The three-dimensional measuring device 10 may be configured such that the projection unit 20, the imaging unit 30 and the measurement unit 40 are separate entities, and the measurement unit 40 is configured as an information processing terminal capable of wireless or wired communication with the projection unit 20 and the imaging unit 30. [Explanation of symbols]

[0073] 10. Three-dimensional measuring device 11 Control section 20 Projection section 30 Imaging unit 40 Measurement section M1 First single luminance region M2 2nd single brightness area MS1 Lights-out control area MS2 Lighting control area NP1 1st stripe number acquisition pattern NP2 2nd stripe number acquisition pattern NP3 Stripe number identification pattern R Measurement target SN Stripe Number SN1 1st stripe number SN2 Second stripe number SP Stripe Pattern θ phase value θs fringe switching phase value

Claims

1. a projection unit that projects a predetermined pattern onto a measurement object; an imaging unit that captures an image of the measurement object onto which the predetermined pattern is projected, and outputs event data including two-dimensional point data that identifies a position of an imaging pixel that has experienced a luminance change upon receiving light; a measurement unit that measures a three-dimensional shape of the measurement object imaged by the imaging unit; A three-dimensional measuring apparatus comprising: the predetermined pattern includes a predetermined stripe pattern that is generated so that a plurality of stripe regions, whose luminance changes in a first direction and whose luminance does not change in a second direction perpendicular to the first direction, are arranged along the first direction, and a first stripe number acquisition pattern and a second stripe number acquisition pattern that are projected toward the same projection range as the predetermined stripe pattern and are used to identify stripe numbers that distinguish the stripe regions from other stripe regions; the first stripe number acquisition pattern is generated such that a plurality of first single luminance areas having the same shape as the stripe area are arranged along the first direction, and each of the first single luminance areas has a different luminance; the second stripe number acquisition pattern is generated such that a plurality of second single luminance areas, each having the same or shorter width value in the first direction as the first single luminance area, are arranged along the first direction so as to be centered on a stripe boundary between the first single luminance areas, and the second single luminance areas have different luminances; The measurement unit is determining a phase value corresponding to the three-dimensional shape of the measurement object from the captured image when the predetermined stripe pattern is projected, and obtaining a first stripe number obtained as the stripe number from the captured image when the first stripe number acquisition pattern is projected, and a second stripe number obtained as the stripe number from the captured image when the second stripe number acquisition pattern is projected; A three-dimensional measuring device characterized in that, when an ideal value of the phase value at the fringe boundary is defined as a fringe switching phase value, if the phase value is within a predetermined range centered on the fringe switching phase value, the three-dimensional shape of the measurement object is measured using the phase value and the second fringe number, and if the phase value is outside the predetermined range, the three-dimensional shape of the measurement object is measured using the phase value and the first fringe number.

2. the predetermined range is set according to an estimated error range of the phase value, 2. The three-dimensional measuring apparatus according to claim 1, wherein the second single-luminance region is set so that a width value in the first direction is shorter than that of the first single-luminance region, in accordance with pixels whose phase values ​​can be within the predetermined range.

3. a projection unit that projects a predetermined pattern onto a measurement object; an imaging unit that captures an image of the measurement object onto which the predetermined pattern is projected, and outputs event data including two-dimensional point data that identifies a position of an imaging pixel that has experienced a luminance change upon receiving light; a measurement unit that measures a three-dimensional shape of the measurement object imaged by the imaging unit; A three-dimensional measuring apparatus comprising: the predetermined pattern includes a predetermined stripe pattern that is generated so that a plurality of stripe regions, whose luminance changes in a first direction and whose luminance does not change in a second direction perpendicular to the first direction, are arranged along the first direction, and a stripe number identifying pattern that is projected toward the same projection range as the predetermined stripe pattern and is used to identify a stripe number that distinguishes the stripe region from other stripe regions; the stripe number specifying pattern is generated such that a plurality of regions, each of which is an extinction control region in which the timing of turning off the light differs from region to region and an illumination control region in which the timing of turning on the light differs from region to region, are arranged in the first direction, with the center of either the extinction control region or the illumination control region being aligned with a stripe boundary between the stripe regions; The measurement unit determines a phase value corresponding to the three-dimensional shape of the object to be measured from the captured image when the specified stripe pattern is projected, and obtains a stripe number from the captured image when the stripe number identifying pattern is projected, and measures the three-dimensional shape of the object to be measured using the phase value and the stripe number.

Images