Three-dimensional measuring device

Abstract

To provide a configuration that can suppress a decrease in measurement accuracy caused by a delay time related to the output timing of event data.SOLUTION: In the projection control process performed by a control unit 11, a projection unit 20 is controlled to project a first stripe pattern as a stripe pattern SP for three-dimensional measurement, and then to project a second stripe pattern in which the on and off timings of each projection pixel of the first stripe pattern are swapped. A measurement unit 40 measures the three-dimensional shape of a measurement object R by utilizing the average value of an event output time difference obtained when the first stripe pattern is projected and an event output time difference obtained when the second stripe pattern is projected.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a three-dimensional measuring apparatus that measures the three-dimensional shape of a measurement object. [Background technology]

[0002] Conventionally, as a three-dimensional measuring device for measuring the three-dimensional shape of a measurement object, for example, a device using a phase shift method is known. The phase shift method is a technique for projecting a plurality of stripe pattern images with shifted phases, and performing three-dimensional measurement of the measurement object onto which the stripe pattern images are projected.

[0003] In relation to the technology of performing three-dimensional measurement using the phase shift method, a three-dimensional measuring device disclosed in the following Patent Document 1 is known for generating an image of a measurement object at a higher speed. In this three-dimensional measuring device, a sine wave pattern is adopted as a predetermined stripe pattern for the phase shift method, and an event camera is adopted that outputs event data including two-dimensional point data that identifies the position of a pixel corresponding to a pixel that has changed in luminance when receiving light, and is configured to generate a captured image of the measurement object on which a stripe pattern is projected from the event data. The event camera has a feature that it does not output pixel information with no change in luminance, that is, redundant data (event data) like conventional cameras, and therefore, by realizing a reduction in the amount of data communication and a reduction in the amount of image processing, it is possible to obtain information on the shape of the measurement object at a higher speed. On the other hand, since the event data does not contain the luminance information used in the phase shift method, luminance information (stripe pattern information) is obtained based on the time difference (event output time difference) between the occurrence time of event data of a positive luminance change (positive polarity event data) that is output on a pixel-by-pixel basis when the light is turned on and the occurrence time of event data of a negative luminance change (negative polarity event data) that is output when the light is turned off, making it possible to measure the three-dimensional shape of the object to be measured using event data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-067644 A Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, the delay time from when the projection unit is turned on until positive polarity event data is output by the imaging unit (hereinafter also referred to as the on-time delay time) differs from the delay time from when the projection unit is turned off until negative polarity event data is output by the imaging unit (hereinafter also referred to as the off-time delay time) for each pixel. That is, a delay time difference, which is the time difference between the on-time delay time and the off-time delay time, occurs for each pixel. This is because the delay time varies depending on the threshold value that is the generation criterion for positive polarity event data and the threshold value that is the generation criterion for negative polarity event data, the amount of light (light source light amount and ambient light amount), the reflectance of the measurement target, the number of events that occur per unit time, etc.

[0006] In a configuration that utilizes an event output time difference (the time difference between the output timing of positive polarity event data and the output timing of negative polarity event data) like the three-dimensional measuring device disclosed in Patent Document 1, if the above-mentioned delay time difference does not occur, the time difference from turning on to turning off and the event output time difference match, so the delay time difference does not affect the measurement accuracy of the three-dimensional measurement. However, in reality, since the delay time difference occurs as described above, the time difference from turning on to turning off and the event output time difference do not match. For this reason, there is a problem that the measurement accuracy of the three-dimensional measurement decreases if the delay time difference differs for each pixel.

[0007] The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a configuration that can suppress a decrease in measurement accuracy caused by a delay time related to the output timing of event data. [Means for solving the problem]

[0008] In order to achieve the above object, one aspect of the present invention is a projection unit (20) that projects a predetermined stripe pattern onto a measurement object (R) by controlling the on / off timing of each projection pixel; an imaging unit (30) that captures an image of the measurement object onto which the predetermined stripe pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change when light is received; a measurement unit (40) that measures a three-dimensional shape of the measurement object by utilizing an event output time difference, which is a time difference between an output timing of positive polarity event data and an output timing of negative polarity event data that are output for each imaging pixel from the captured image; A projection control unit (11) that controls the projection unit; A three-dimensional measuring device (10) comprising: the projection control unit controls the projection unit so that, when projecting the predetermined stripe pattern, after projecting a first stripe pattern (SP1), the projection unit projects a second stripe pattern (SP2) in which a turn-on timing and a turn-off timing are interchanged for each projection pixel of the first stripe pattern; The measurement unit is characterized in that it measures the three-dimensional shape of the measurement object by utilizing an average value of the event output time difference obtained when projecting the first stripe pattern and the event output time difference obtained when projecting the second stripe pattern.

[0009] Another aspect of the present invention is a projection unit (20) that projects a predetermined stripe pattern onto a measurement object (R) by controlling the on / off timing of each projection pixel; an imaging unit (30) that captures an image of the measurement object onto which the predetermined stripe pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change when light is received; a measurement unit (40) that measures a three-dimensional shape of the measurement object by utilizing an event output time difference, which is a time difference between an output timing of positive polarity event data and an output timing of negative polarity event data that are output for each imaging pixel from the captured image; A projection control unit (11) that controls the projection unit; A three-dimensional measuring device (10) comprising: When the time from the light-on timing to the light-off timing is defined as the light-on time, and the time from the light-off timing to the light-on timing is defined as the light-off time, the projection control unit controls the projection unit when projecting the predetermined stripe pattern, so that, for each projection pixel, the on time and the off time are the same set time (Ts), and the off timing is shared so that the on time and the off time are continuous as a first measurement time (Ts1) and a second measurement time (Ts2), or the on timing is shared so that the off time and the on time are continuous as a first measurement time (Ts1) and a second measurement time (Ts2), The measurement unit is characterized in that it measures the three-dimensional shape of the measurement object by utilizing an average value of the event output time difference obtained at the first measurement time and the event output time difference obtained at the second measurement time. The symbols in parentheses above indicate the corresponding relationship with the specific means described in the embodiments described later. Effect of the Invention

[0010] In one embodiment of the present invention, the projection control unit controls the projection unit to project a first stripe pattern when projecting a predetermined stripe pattern, and then to project a second stripe pattern in which the on and off timings of each projection pixel of the first stripe pattern are swapped. The measurement unit measures the three-dimensional shape of the measurement object by using an average value of the event output time difference obtained when the first stripe pattern is projected and the event output time difference obtained when the second stripe pattern is projected.

[0011] The event output time difference obtained when the first stripe pattern is projected corresponds to the conventional event output time difference, and a delay time (light-on delay time) occurs in pixel units from when the projection unit is turned on until the positive polarity event data is output by the imaging unit, and a delay time (light-off delay time) occurs in pixel units from when the projection unit is turned off until the negative polarity event data is output by the imaging unit. Similarly, the event output time difference obtained when the second stripe pattern is projected also has a light-on delay time and a light-off delay time in pixel units. For the same pixel, the light-on delay time and the light-off delay time are the same, and since the light-on timing and the light-off timing are swapped between the first stripe pattern and the second stripe pattern, the average value of the event output time difference obtained when the first stripe pattern is projected and the event output time difference obtained when the second stripe pattern is projected is an event output time difference that is free of the influence of the delay time difference (the time difference between the light-on delay time and the light-off delay time). Therefore, by performing three-dimensional measurement using the average value of the event output time difference when the first stripe pattern is projected and the event output time difference when the second stripe pattern is projected, it is possible to suppress the decrease in measurement accuracy caused by the delay time related to the output timing of the event data.

[0012] In another aspect of the present invention, the projection control unit controls the projection unit when projecting a predetermined stripe pattern so that the on time and the off time are the same set time for each projection pixel, and the off timing is shared so that the on time and the off time are continuous as the first measurement time and the second measurement time, or the on timing is shared so that the off time and the on time are continuous as the first measurement time and the second measurement time. The measurement unit measures the three-dimensional shape of the measurement object by utilizing an average value of the event output time difference obtained at the first measurement time and the event output time difference obtained at the second measurement time.

[0013] The event output time difference obtained in the first measurement time corresponds to the conventional event output time difference, and has a turn-on delay time and a turn-off delay time in pixel units. Similarly, the event output time difference obtained in the second measurement time also has a turn-on delay time and a turn-off delay time in pixel units. For the same pixel, the turn-on delay time and the turn-off delay time are the same, and the first measurement time and the second measurement time have a relationship in which the turn-on time and the turn-off time of the same set time are exchanged, so that the average value of the event output time difference obtained in the first measurement time and the event output time difference obtained in the second measurement time is an event output time difference that is free from the influence of the delay time difference (the time difference between the turn-on delay time and the turn-off delay time). Therefore, by performing three-dimensional measurement using the average value of the event output time difference obtained in the first measurement time and the event output time difference obtained in the second measurement time, it is possible to suppress the deterioration of measurement accuracy caused by the delay time related to the output timing of the event data. [Brief description of the drawings]

[0014] [Figure 1] 1 is a block diagram showing a schematic configuration of a three-dimensional measuring apparatus according to a first embodiment. [Diagram 2] FIG. 1 is an explanatory diagram illustrating a state in which a stripe pattern for a general phase shift method is projected onto a measurement object. [Diagram 3] 1A and 1B are diagrams illustrating three-dimensional measurement by a phase shift method. [Figure 4] 11 is an explanatory diagram illustrating the relationship between the time difference between the output timing of positive polarity event data and negative polarity event data output during a unit time and the luminance value. FIG. [Diagram 5]FIG. 5(A) is an explanatory diagram showing the positional relationship of two irradiation points Pa, Pb on a plane where the light-on delay time and light-off delay time are the same value for every pixel, FIG. 5(B) is an explanatory diagram explaining the light-on timing and light-off timing at irradiation point Pa and the set light-on time, FIG. 5(C) is an explanatory diagram explaining the actual light-on time and delay time measured when projecting in FIG. 5(B), FIG. 5(D) is an explanatory diagram explaining the light-on timing and light-off timing at irradiation point Pb and the set light-on time, and FIG. 5(E) is an explanatory diagram explaining the actual light-on time and delay time measured when projecting in FIG. 5(D). [Figure 6] FIG. 6(A) is an explanatory diagram illustrating the waveform pattern of the actual lighting time measured under the assumption that no lighting delay time or extinguishing delay time occurs, and FIG. 6(B) is an explanatory diagram illustrating the waveform pattern of the actual lighting time measured when the same lighting delay time and extinguishing delay time occur in each pixel. [Figure 7] Figure 7(A) is an explanatory diagram showing the positional relationship of two irradiation points Pc, Pd on a surface where the light-on delay time and light-off delay time differ for each pixel, Figure 7(B) is an explanatory diagram explaining the light-on timing and light-off timing at irradiation point Pc and the set light-on time, Figure 7(C) is an explanatory diagram explaining the actual light-on time and delay time measured when projecting in Figure 7(B), Figure 7(D) is an explanatory diagram explaining the light-on timing and light-off timing at irradiation point Pd and the set light-on time, and Figure 7(E) is an explanatory diagram explaining the actual light-on time and delay time measured when projecting in Figure 7(D). [Figure 8] FIG. 8(A) is an explanatory diagram illustrating the waveform pattern of the actual lighting time measured under the assumption that no lighting delay time or extinguishing delay time occurs, and FIG. 8(B) is an explanatory diagram illustrating the waveform pattern of the actual lighting time measured when different lighting delay times and extinguishing delay times occur in each pixel. [Figure 9]Figure 9(A) is an explanatory diagram explaining the actual light-on time and the actual light-off time measured when the first stripe pattern and the second stripe pattern are projected onto the irradiation point Pc, and Figure 9(B) is an explanatory diagram explaining the actual light-on time and the actual light-off time measured when the first stripe pattern and the second stripe pattern are projected onto the irradiation point Pd. [Figure 10] 10A and 10B are explanatory diagrams illustrating the effect of the present invention in terms of reducing measurement time. FIG. 10A shows the set lighting time and actual measured lighting time in the conventional case, and FIG. 10B shows the set lighting time and actual measured lighting time in the present invention. [Figure 11] FIG. 11(A) is an explanatory diagram explaining the first on timing, the off timing, and the second on timing when a stripe pattern is projected in the second embodiment so that the set on time and the set off time are successive with the same set time, and FIG. 11(B) is an explanatory diagram explaining the actual measured on time and actual measured off time measured during projection in FIG. 11(A). [Figure 12] FIG. 12(A) is an explanatory diagram explaining the first light-off timing, the light-on timing, and the second light-off timing in a modified example of the second embodiment in which a stripe pattern is projected so that the set light-off time and the set light-on time are successive at the same set time, and FIG. 12(B) is an explanatory diagram explaining the actual light-off time and the actual light-on time measured during projection in FIG. 12(A). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] [First embodiment] Hereinafter, an embodiment of a three-dimensional measuring apparatus according to the present invention will be described with reference to the drawings. The three-dimensional measuring device 10 according to the present embodiment is a device for measuring the three-dimensional shape of a measurement object R, and is configured to include a control unit 11 for overall control, a projection unit 20 for projecting a predetermined stripe pattern or the like for three-dimensional measurement onto the measurement object R, an imaging unit 30 for capturing an image of the measurement object R onto which the predetermined stripe pattern or the like is projected, and a measurement unit 40 for measuring the three-dimensional shape of the measurement object R from the captured image, as shown in FIG. The three-dimensional measuring device 10 configured in this manner measures the three-dimensional shape of the measurement object R, such as a workpiece, which is to move relatively to the hand at high speed by being attached to, for example, a robot hand. Here, the relative movement refers to the relative movement between the movement of the three-dimensional measuring device 10 attached to the robot hand and the high-speed movement of the measurement object R. When the position of the three-dimensional measuring device 10 is fixed, the relative movement is the movement of the measurement object R.

[0016] For convenience, a typical stripe pattern having up to 13 stripes is shown in a simplified form in Fig. 2. More specifically, a typical stripe pattern is represented as a sine wave pattern, so the light and dark parts of the stripe pattern have the same width, but for convenience, the dark parts are shown as lines with smaller widths in Fig. 2. Also, the number of stripes is 13 or more in the embodiment, but is abbreviated to 13.

[0017] The control unit 11 is mainly composed of a microcomputer and has a CPU, a system bus, an input / output interface, etc., and constitutes an information processing device together with a storage unit consisting of a ROM, a RAM, a non-volatile memory, etc. In addition to programs related to robot control, the storage unit stores in advance programs related to control of the projection unit 20 and programs for executing control processing using the three-dimensional measurement results by the measurement unit 40, etc., which are executable by the control unit 11.

[0018] The projection unit 20 is a so-called DLP projector, and is controlled by the control unit 11 to project a predetermined stripe pattern or the like by reflecting light from a light source by a DMD element. The DMD element is an array of fine mirrors corresponding to each projection pixel of an image projected on a screen, and is configured to change the angle of each mirror to turn on / off the light emitted to the screen in microsecond units. Therefore, the mirror is switched from reflection OFF to reflection ON to a light-on state (light-projecting state), and switched from reflection ON to reflection OFF to a light-off state. That is, the projection unit 20 functions to project a predetermined stripe pattern or the like by controlling the ON / OFF of the reflection of incident light by the DMD, which is an array of multiple mirrors, for each mirror as a light-on timing and a light-off timing by the control unit 11. Therefore, the gradation (brightness) of the reflected light is changed depending on the ratio of the time each mirror is turned on and off, thereby making it possible to display gradations based on the image data of the image to be projected. In this embodiment, the projection unit 20 includes mirrors that correspond to k×l projection pixels (for example, 1140×912), with the upper left being (1, 1) and the lower right being (k, l).

[0019] In such a configuration, the longer the light emission time (the time from the lighting timing (reflection ON) to the lighting off timing (reflection OFF)) of the single pulse light emitted once within the unit time secured for each light emission state, the brighter the light emission state becomes, so that the light emission state can be specified according to the light emission time. For example, when R (red), G (green), and B (blue) colors are prepared as light incident on the DMD element, an R light emission state in which R is reflected by a mirror and light is emitted, a G light emission state in which G is reflected by a mirror and light is emitted, and a B light emission state in which B is reflected by a mirror and light is emitted are repeated in a short, predetermined cycle, and each light emission time is individually adjusted, so that a color image can be projected. For this reason, the control unit 11 functions to set the lighting timing and the lighting off timing (reflection ON / OFF timing) within the unit time for each mirror according to a predetermined pattern described later.

[0020] The imaging unit 30 is a so-called event camera, and is equipped with an imaging element that outputs event data (specifically, data including two-dimensional point data, time, and polarity of brightness change) including two-dimensional point data that identifies the position of an imaging pixel corresponding to an imaging pixel that has undergone a brightness change when light is received, and is configured to be able to generate an imaging image from the event data output from the imaging element. For this reason, the imaging unit 30 outputs event data of positive polarity (positive brightness change) when a brightness change occurs that makes the imaging pixel brighter by receiving light, and outputs event data of negative polarity (negative brightness change) when a brightness change occurs that makes the imaging pixel darker by extinguishing the light. Image data of the measurement object R can be generated by plotting the two-dimensional point data of a plurality of event data output within a certain period as points on a predetermined plane, and the imaging unit 30 is configured to output the image data or event data generated in this way to the measurement unit 40.

[0021] The measurement unit 40 is controlled by the control unit 11 and performs a three-dimensional measurement process to measure the three-dimensional shape of the measurement object R based on an image captured by the imaging unit 30 of the measurement object R in a state in which a predetermined stripe pattern or the like is projected from the projection unit 20. The three-dimensional measurement process in this embodiment employs a phase shift method, and the three-dimensional shape of the measurement object R is measured using an event output time difference, which is the time difference between the output timing of positive polarity event data and the output timing of negative polarity event data output for each imaging pixel.

[0022] Generally, as a stripe pattern for the phase shift method (hereinafter also referred to as stripe pattern SPo), a sine wave pattern specified by the brightness value I(x, y, n) of the following formula (1) is adopted in order to obtain a phase value θ corresponding to a value distorted according to the surface shape of the measurement object R based on a grating image (stripe image) obtained by capturing an image of the measurement object R in a projected state. The stripe pattern SPo is projected so that the brightness periodically changes in a first direction (left-right direction in the example of FIG. 2) and does not change in a second direction (up-down direction in the example of FIG. 2) perpendicular to the first direction. When the number of phase shifts is N, the brightness value I(x, y, n) of N phase-shifted grating images (stripe images) is expressed by formula (1). I(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N}+b(x,y) (1) Here, point (x, y) is a point in the lattice image, a(x, y) indicates the luminance amplitude, b(x, y) indicates the background luminance, and θ(x, y) indicates the phase value of the lattice where n=0. The distance z to point (x, y) is measured according to the phase value θ(x, y) calculated from the luminance values ​​I(x, y, n) of N lattice images and the separately determined stripe number SN.

[0023] Specifically, for example, when three grating images are obtained in one period of the above-mentioned R, G, and B light emission states, the luminance value I(x, y, 0) in the R light emission state, the luminance value I(x, y, 1) in the G light emission state, and the luminance value I(x, y, 2) in the B light emission state are obtained from the captured image with N=3. In this case, the stripe pattern SPo projected from the projection unit 20 is configured such that the phases of the sine wave pattern consisting of only R, the sine wave pattern consisting of only G, and the sine wave pattern consisting of only B are shifted by 2π / 3.

[0024] By obtaining the luminance values ​​I(x,y,0), I(x,y,1), and I(x,y,2) at the point (x,y) in the captured image, the phase value θ(x,y) is calculated using the above formula (1), and the distance z to the point (x,y) is measured according to the phase value θ(x,y) thus calculated. By measuring the distance z to each point (x,y) of the captured measurement object R in this manner, the three-dimensional shape of the measurement object R can be measured.

[0025] For example, when the distance z of point Pr1 in FIG. 3 is obtained, the phase value θ of point Pr1 and information on which stripe number point Pr1 is (stripe number SN) are obtained from N captured images of the imaging unit 30 in a state where the stripe pattern SPo is shifted and projected from the projection unit 20 N times. When the angle θp1 at the projection unit 20 and the angle θc1 at the imaging unit 30 are obtained from the phase value θ and stripe number SN obtained in this way, the distance z of point Pr1 can be obtained by triangulation since the distance (parallax Lpc) between the projection unit 20 and the imaging unit 30 is known. Similarly, the distance z of point Pr2 in FIG. 3 can be obtained by triangulation based on the angle θp2 at the projection unit 20 and the angle θc2 at the imaging unit 30 obtained from the phase value θ of point Pr2 obtained from the N captured images and the stripe number SN. By performing this calculation over the entire measurement area, three-dimensional measurement can be performed.

[0026] In this embodiment, an event camera is used as an imaging unit for accurately imaging the measurement target R that moves relatively at high speed. In such a configuration, event data corresponding to imaging pixels where a luminance change has occurred is output, but the event data does not include a luminance value, so that it is not possible to directly obtain the luminance values ​​(for example, the above-mentioned I(x,y,0), I(x,y,1), and I(x,y,2)) required for the phase shift method.

[0027] For this reason, in this embodiment, stripe pattern information Is(x,y,n) corresponding to luminance value I(x,y,n) is calculated as the time difference between the output timing of positive polarity event data that is output first during unit time T (see the upward arrow in FIG. 4) and the output timing of negative polarity event data that is output next (see the downward arrow in FIG. 4), as shown in FIG. 4. This is because, after positive polarity event data is output at emission start timing t1, negative polarity event data is output at emission end timing t2, and therefore, the longer the time difference, the higher the luminance value at that imaging pixel.

[0028] The stripe pattern information Is(x,y,n) obtained in this manner is affected by the three-dimensional shape of the measurement object R, so the phase value θ(x,y) can be obtained using the following equation (2). Is(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N} +b(x,y) (2)

[0029] Moreover, the stripe pattern information Is(x, y, n) can be obtained as the time difference between the output timing of the negative polarity event data that is output first and the positive polarity event data that is output next during the unit time T, and used to calculate the phase value θ(x, y). That is, the three-dimensional shape of the measurement object R can be measured by using the event output time difference, which is the time difference between the output timing of the positive polarity event data that is output for each imaging pixel and the output timing of the negative polarity event data.

[0030] Next, assuming that the delay time from when the projection unit 20 is turned on until positive polarity event data is output by the imaging unit 30 is defined as the on-time delay time, and the delay time from when the projection unit 20 is turned off until negative polarity event data is output by the imaging unit 30 is defined as the off-time delay time, the delay time difference between the on-time delay time and the off-time delay time will be described below, and the stripe pattern SP etc. projected from the projection unit 20 to eliminate the influence of this delay time difference will be described below.

[0031] First, a plane where the light-on delay time and the light-off delay time are the same value for each pixel (a plane where the reflectance is uniform) will be described by taking as an example a case where the light-on point Pa and the light-off point Pb on the line Lab with the same Y coordinate are the measurement targets as shown in FIG. 5(A). Note that the time from the light-on timing to the light-off timing is the set light-on time, the time from the output timing of the positive polarity event data to the output timing of the negative polarity event data is the actual light-on time, the time from the light-off timing to the light-on timing is the set light-off time, and the time from the output timing of the negative polarity event data to the output timing of the positive polarity event data is the actual light-off time. Also, for convenience, in FIG. 5 and the like, the light-on timing is illustrated by an upward solid arrow, the positive polarity event data is illustrated by an upward dashed arrow, the light-off timing is illustrated by a downward solid arrow, and the negative polarity event data is illustrated by a downward dashed arrow.

[0032] For example, by irradiating the irradiation point Pa with a set lighting time of 180 μs, which results in lighting and extinguishing timings of 20 μs and 200 μs (see FIG. 5B), if positive and negative event data are output at 30 μs and 205 μs, respectively, the actual lighting time of 175 μs is measured from the time difference between these event outputs (see FIG. 5C). During this measurement, a lighting delay time D1a of 10 μs and an extinguishing delay time D2a of 5 μs are generated.

[0033] Also, for example, by irradiating the irradiation point Pb with a set lighting time of 100 μs, with the lighting timing and the lighting-out timing being 20 μs and 120 μs (see FIG. 5(D)), when the positive polarity event data and the negative polarity event data are output at 30 μs and 125 μs, the actual lighting time of 95 μs is measured from the event output time difference (see FIG. 5(E)). During this measurement, a lighting delay time D1b of 10 μs and a lighting-out delay time D2b of 5 μs are generated, which are the same values ​​as the lighting delay time D1a and the lighting-out delay time D2a at the irradiation point Pa as described above.

[0034] Then, when each pixel on the line Lab in FIG. 5(A) is considered, on the premise that a sine wave pattern is projected as a predetermined stripe pattern for the phase shift method, and assuming that no on-time delay time and no off-time delay time occur, the waveform pattern of the actual on-time restored from the event data is restored to the same pattern as the projected sine wave pattern (waveform pattern of the set on-time), as illustrated in FIG. 6(A). In reality, since the on-time delay time and the off-time delay time occur and the values ​​of the two are different, a difference Δt occurs between the waveform pattern of the actual on-time restored from the event data and the projected sine wave pattern (waveform pattern of the set on-time), as illustrated in FIG. 6(B). On the other hand, if the on-time delay time and the off-time delay time are the same value for each pixel, the difference Δt is simply a shift of the same amount in the Y direction, and does not affect the measurement process. In FIG. 6(B), the waveform pattern of the actual on-time restored from the event data is illustrated by a solid line, and the projected sine wave pattern (waveform pattern of the set on-time) is illustrated by a dashed line.

[0035] However, in reality, the on-time delay time and the off-time delay time change for each pixel depending on the reflectance of the measurement target R. Below, a surface with different on-time delay time and off-time delay time will be described using an example in which the measurement targets are irradiation points Pc and Pd on the line Lcd with the same Y coordinate, as shown in Fig. 7(A). Note that the reflectance of the irradiation point Pc and its surroundings is relatively high, and the reflectance of the irradiation point Pd and its surroundings is relatively low.

[0036] For example, by irradiating the irradiation point Pc with a set lighting time of 180 μs, which results in lighting and extinguishing timings of 20 μs and 200 μs (see FIG. 7B), if positive and negative event data are output at 22 μs and 201 μs, respectively, the actual lighting time of 179 μs is measured from the time difference between these event outputs (see FIG. 7C). During this measurement, a lighting delay time D1c of 2 μs and an extinguishing delay time D2c of 1 μs are generated.

[0037] Also, for example, by irradiating the irradiation point Pd with a set lighting time of 100 μs with lighting timing and lighting off timing of 20 μs and 120 μs (see FIG. 7(D)), when positive polarity event data and negative polarity event data are output at 40 μs and 132 μs, the actual lighting time of 92 μs is measured from the event output time difference (see FIG. 7(E)). During this measurement, a lighting delay time D1d of 20 μs and a lighting off delay time D2d of 12 μs are generated, which are different values ​​from the lighting delay time D1c and the lighting off delay time D2c at the irradiation point Pc as described above.

[0038] Then, when each pixel on the line Lcd in FIG. 7(A) is extracted and it is assumed that there is no on-time delay time and no off-time delay time as described above, as illustrated in FIG. 8(A), the waveform pattern of the actual on-time restored from the event data is restored in the same pattern as the sine wave pattern (waveform pattern of the set on-time) to be projected. In reality, a delay time difference occurs which is the time difference between the on-time delay time and the off-time delay time, and this delay time difference differs for each pixel, so that, as illustrated in FIG. 8(B), a difference in value differs on a pixel-by-pixel basis between the waveform pattern of the actual on-time restored from the event data and the sine wave pattern (waveform pattern of the set on-time) (see symbols Δt1 and Δt2 in FIG. 8). In particular, since the reflectance is lower around the irradiation point Pd than around the irradiation point Pc, the difference Δt2 around the irradiation point Pd is larger than the difference Δt1 around the irradiation point Pc. The difference caused by the delay time difference which differs for each pixel reduces the measurement accuracy of the three-dimensional measurement. In FIG. 8B, the waveform pattern of the actually measured lighting time restored from the event data is shown by a solid line, and the projected sine wave pattern (waveform pattern of the set lighting time) is shown by a dashed line.

[0039] For this reason, in the projection control process performed by the control unit 11 to control the projection unit 20 in this embodiment, a predetermined stripe pattern (hereinafter also referred to as stripe pattern SP) set to eliminate the influence of the delay time difference is projected from the projection unit 20. In the projection control process, the projection unit 20 is controlled to project a stripe pattern (hereinafter also referred to as first stripe pattern SP1) corresponding to the above-mentioned stripe pattern SPo as the stripe pattern SP, and then project a stripe pattern (hereinafter also referred to as second stripe pattern SP2) in which the on and off timings of the first stripe pattern SP1 are swapped for each projection pixel. In the three-dimensional measurement process performed by the measurement unit 40, the three-dimensional shape of the measurement target R is measured using the average value of the event output time difference obtained when the first stripe pattern SP1 is projected and the event output time difference obtained when the second stripe pattern SP2 is projected. The control unit 11 performing the projection control process may correspond to an example of a "projection control unit".

[0040] The following describes why the influence of the delay time difference can be eliminated by averaging the event output time difference obtained when the first stripe pattern SP1 is projected and the event output time difference obtained when the second stripe pattern SP2 is projected.

[0041] Assume that, as shown in FIG. 9(A), a pattern with a set lighting time of 180 μs, in which the lighting timing and the turning-off timing are 20 μs and 200 μs during a unit time T, is projected as a first stripe pattern SP1 to the above-mentioned irradiation point Pc, and then a second stripe pattern SP2 with the lighting timing and the turning-off timing swapped is projected during the next unit time T. In the example of FIG. 9(A), the second stripe pattern SP2 is a pattern with a set turning-off time of 180 μs, in which the turning-off timing and the lighting timing are 20 μs and 200 μs. Note that FIG. 9 illustrates the timing of the start of the unit time T as 0 μs. Also, although not illustrated in FIG. 9, the second stripe pattern SP2 needs to control the turning-off timing first, so it is in a lighting state immediately after the start of projection of the second stripe pattern SP2.

[0042] In this case, the actual light-on time of 179 μs is measured from the event output time difference obtained when the first stripe pattern SP1 is projected, and the actual light-off time of 181 μs is measured from the event output time difference obtained when the second stripe pattern SP2 is projected. At that time, since the measurements are taken at the same irradiation point Pc, the light-on delay time D1c and the light-off delay time D2c occurring when the first stripe pattern SP1 is projected and the light-on delay time D1c and the light-off delay time D2c occurring when the second stripe pattern SP2 is projected are the same values ​​(D1c=2 μs, D2c=1 μs).

[0043] Then, by averaging the event output time difference obtained when the first stripe pattern SP1 is projected and the event output time difference obtained when the second stripe pattern SP2 is projected, the delay times cancel each other out, and an event output time difference that eliminates the influence of the delay time difference can be obtained. In the example of Fig. 9(A), the average value of the actual measured light-on time of 179μs and the actual measured light-off time of 181μs, 180μs, matches the set light-on time of 180μs and the set light-off time of 180μs.

[0044] Similarly, as shown in Fig. 9(B), a first stripe pattern SP1 with a set light-on time of 100 μs, in which the light-on timing and the light-off timing are 20 μs and 120 μs during unit time T, is projected onto the above-mentioned irradiation point Pd, and then a second stripe pattern SP2 with the light-on timing and the light-off timing swapped is projected onto the next unit time T. In the example of Fig. 9(B), the second stripe pattern SP2 is a pattern with a set light-off time of 100 μs, in which the light-off timing and the light-on timing are 20 μs and 120 μs.

[0045] In this case, an actual light-on time of 92 μs is measured from the event output time difference obtained when the first stripe pattern SP1 is projected, and an actual light-off time of 108 μs is measured from the event output time difference obtained when the second stripe pattern SP2 is projected. At that time, since the measurements are taken at the same irradiation point Pd, the light-on delay time D1d and the light-off delay time D2d occurring when the first stripe pattern SP1 is projected and the light-on delay time D1d and the light-off delay time D2d occurring when the second stripe pattern SP2 is projected are the same values ​​(D1d=20 μs, D2d=12 μs), respectively.

[0046] Even in this case, by averaging the event output time difference obtained when the first stripe pattern SP1 is projected and the event output time difference obtained when the second stripe pattern SP2 is projected, the delay times cancel each other out, so that an event output time difference that eliminates the influence of the delay time difference can be obtained. In the example of Fig. 9(B), the average value of the actual measured light-on time of 92μs and the actual measured light-off time of 108μs, 100μs, matches the set light-on time of 100μs and the set light-off time of 100μs.

[0047] In addition, assuming that the off timing and on timing are controlled by the off time set for the first stripe pattern SP1, the above effect can be achieved even if the second stripe pattern SP2 is controlled to swap the off timing and on timing.

[0048] As described above, in the three-dimensional measuring apparatus 10 according to this embodiment, the projection control process performed by the control unit 11 controls the projection unit 20 to project a first stripe pattern SP1 as a stripe pattern SP for three-dimensional measurement, and then to project a second stripe pattern SP2 in which the on and off timings of each projection pixel of the first stripe pattern SP1 are swapped. The measurement unit 40 measures the three-dimensional shape of the measurement object R by utilizing the average value of the event output time difference obtained when projecting the first stripe pattern SP1 and the event output time difference obtained when projecting the second stripe pattern SP2.

[0049] For the same pixel, the on-time delay time D1 and the off-time delay time D2 are the same, and since the on-time and off-time are swapped between the first stripe pattern SP1 and the second stripe pattern SP2, the average value of the event output time difference obtained when the first stripe pattern SP1 is projected and the event output time difference obtained when the second stripe pattern SP2 is projected is an event output time difference that is free of the influence of the delay time difference (the time difference between the on-time delay time D1 and the off-time delay time D2). Therefore, by performing three-dimensional measurement using the average value of the event output time difference when the first stripe pattern SP1 is projected and the event output time difference when the second stripe pattern SP2 is projected, it is possible to suppress a decrease in measurement accuracy caused by the delay time related to the output timing of the event data.

[0050] In particular, since the delay time difference can be eliminated, even in a configuration in which the second stripe pattern SP2 is projected immediately after the first stripe pattern SP1, it is possible to set the set time per piece of information (set on time / set off time) to a short time, and depending on the set time, the measurement time can be shortened.

[0051] The above-mentioned reduction in measurement time will be explained using FIGS. 10(A) and 10(B) as examples. In the conventional configuration where the delay time may be large, in order to absorb the effect of the delay time, the set time per piece of information must be set long, as shown in Fig. 10(A) for example. In the example of Fig. 10(A), the conditions are set so that the time interval is [1] when the time interval is 0-15 μs, the time interval is [2] when the time interval is 16-25 μs, and the time interval is [3] when the time interval is 26-35 μs, in order to allow a set lighting time of 20 μs and an actual lighting time of 16 μs.

[0052] On the other hand, in this embodiment, in order to eliminate the effect of the delay time (delay time difference), the conditions are as follows: when the time interval is 0-7 μs, the time interval is [1], when the time interval is 8-12 μs, the time interval is [2], and when the time interval is 13-17 μs, the time interval is [3], so as to allow a set lighting time of 10 μs and an actual lighting time of 10.5 μs, as exemplified in FIG. 10(B). In other words, it becomes possible to design the time resolution itself to be smaller, and the set time per piece of information can be set shorter, so that even when two patterns are projected consecutively, the measurement time can be shortened depending on the set time (set lighting time / set lighting off time).

[0053] [Second embodiment] Next, a three-dimensional measuring apparatus according to a second embodiment of the present invention will be described with reference to the drawings. The second embodiment is different from the first embodiment in that the first stripe pattern SP1 and the second stripe pattern SP2 are projected so as to be continuous with each other. Therefore, the same components as those in the first embodiment are denoted by the same reference numerals and the description thereof is omitted.

[0054] In the projection control process of this embodiment, the stripe pattern SP is projected so that the first stripe pattern SP1 and the second stripe pattern SP2 are continuous with each other by sharing the same turn-off timing.

[0055] Specifically, on the premise that the set on time and the set off time are the same set time Ts for each projection pixel, the stripe pattern SP is projected so that the set on time and the set off time are continuous as the first measurement time Ts1 and the second measurement time Ts2 by sharing the off timing. For example, as illustrated in Fig. 11(A) for the above-mentioned irradiation point Pc, a stripe pattern SP with a set on time of 180 μs and a set off time of 180 μs is projected, in which the first on timing, the off timing, and the second on timing are 20 μs, 200 μs, and 380 μs.

[0056] Assume that, when projecting the stripe pattern SP of Fig. 11(A), positive polarity event data, negative polarity event data, and positive polarity event data are output at 22 μs, 201 μs, and 382 μs, as illustrated in Fig. 11(B). In this case, an actual light-on time of 179 μs is measured from the event output time difference obtained at the first measurement time Ts1, and an actual light-off time of 181 μs is measured from the event output time difference obtained at the second measurement time Ts2. The average value 180 μs of the actual light-on time 179 μs and the actual light-off time 181 μs matches the set light-on time 180 μs and the set light-off time 180 μs, so that an event output time difference that is free from the influence of the delay time difference can be obtained.

[0057] As described above, in the three-dimensional measuring apparatus 10 according to this embodiment, the projection control process performed by the control unit 11 controls the projection unit 20 so that, when projecting the stripe pattern SP for three-dimensional measurement, the on time and off time are the same set time Ts for each projection pixel, and the off timing is shared, so that the on time and off time are continuous as the first measurement time Ts1 and the second measurement time Ts2. The measurement unit 40 measures the three-dimensional shape of the measurement object R by utilizing the average value of the event output time difference obtained at the first measurement time Ts1 and the event output time difference obtained at the second measurement time Ts2.

[0058] For the same pixel, the on-time delay time D1 and the off-time delay time D2 are the same, and the first measurement time Ts1 and the second measurement time Ts2 have a relationship in which the on-time and off-time of the same set time Ts are swapped, so that the average value of the event output time difference obtained in the first measurement time Ts1 and the event output time difference obtained in the second measurement time Ts2 is an event output time difference that is free from the influence of the delay time difference (the time difference between the on-time delay time and the off-time delay time). Therefore, by performing three-dimensional measurement using the average value of the event output time difference obtained in the first measurement time Ts1 and the event output time difference obtained in the second measurement time Ts2, it is possible to suppress a decrease in measurement accuracy caused by the delay time related to the output timing of the event data.

[0059] In addition, in the projection control process performed by the control unit 11, the projection unit 20 may be controlled so that the lighting timing is shared, and the lighting-off time and the lighting-on time are continuous as the first measurement time Ts1 and the second measurement time Ts2.

[0060] Specifically, for example, a stripe pattern SP with a set off time of 180 μs and a set on time of 180 μs may be projected onto the above-mentioned irradiation point Pc, as illustrated in FIG. 12(A), in which the first off timing, on timing, and second off timing are 20 μs, 200 μs, and 380 μs.

[0061] Assume that when the stripe pattern SP of Fig. 12(A) is projected, negative polarity event data, positive polarity event data, and negative polarity event data are output at 21 μs, 202 μs, and 381 μs, as illustrated in Fig. 12(B). In this case, an actual light-off time of 181 μs is measured from the event output time difference obtained at the first measurement time Ts1, and an actual light-on time of 179 μs is measured from the event output time difference obtained at the second measurement time Ts2. The average value 180 μs of the actual light-off time 181 μs and the actual light-on time 179 μs matches the set light-off time 180 μs and the set light-on time 180 μs, so that an event output time difference that is free from the influence of the delay time difference can be obtained.

[0062] The present invention is not limited to the above-described embodiments, and may be embodied as follows, for example. (1) The stripe pattern that switches the on and off timing for each projection pixel is not limited to a specific stripe pattern for three-dimensional measurement using the phase shift method, but may also be, for example, a stripe pattern for identifying a stripe number.

[0063] (2) The three-dimensional measuring device 10 is not limited to being attached to the hand of a robot and moving to measure the three-dimensional shape of a measurement object that moves relative to the robot. For example, it may be used in a fixed state to measure the three-dimensional shape of a measurement object that moves on a conveyor line.

[0064] (3) The three-dimensional measuring device 10 may be configured such that the projection unit 20, the imaging unit 30 and the measurement unit 40 are separate entities, and the measurement unit 40 is configured as an information processing terminal capable of wireless or wired communication with the projection unit 20 and the imaging unit 30. [Explanation of symbols]

[0065] 10. Three-dimensional measuring device 11 Control unit (projection control unit) 20 Projection section 30 Imaging unit 40 Measurement section D1, D1a~D1d Lighting delay time D2, D2a~D2d Light-off delay time R Measurement target SP Stripe Pattern SP1 First stripe pattern SP2 Second stripe pattern Ts Setting time Ts1 First measurement time Ts2 Second measurement time

Claims

1. a projection unit that projects a predetermined stripe pattern onto a measurement object by controlling the on / off timing of each projection pixel; an imaging unit that captures an image of the measurement object onto which the predetermined stripe pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change when light is received; a measurement unit that measures a three-dimensional shape of the measurement object by using an event output time difference that is a time difference between an output timing of positive polarity event data and an output timing of negative polarity event data that are output for each imaging pixel from the captured image; A projection control unit that controls the projection unit; A three-dimensional measuring apparatus comprising: the projection control unit controls the projection unit so that, when projecting the predetermined stripe pattern, after projecting a first stripe pattern, the projection unit projects a second stripe pattern in which a turn-on timing and a turn-off timing are interchanged for each projection pixel of the first stripe pattern; a measurement unit that measures a three-dimensional shape of the object to be measured by using an average value of the event output time difference obtained when projecting the first stripe pattern and the event output time difference obtained when projecting the second stripe pattern.

2. a projection unit that projects a predetermined stripe pattern onto a measurement object by controlling the on / off timing of each projection pixel; an imaging unit that captures an image of the measurement object onto which the predetermined stripe pattern is projected, and outputs event data including two-dimensional point data that identifies the position of an imaging pixel that has experienced a luminance change when light is received; a measurement unit that measures a three-dimensional shape of the measurement object by using an event output time difference that is a time difference between an output timing of positive polarity event data and an output timing of negative polarity event data that are output for each imaging pixel from the captured image; A projection control unit that controls the projection unit; A three-dimensional measuring apparatus comprising: When the time from the light-on timing to the light-off timing is defined as the light-on time, and the time from the light-off timing to the light-on timing is defined as the light-off time, the projection control unit controls the projection unit when projecting the predetermined stripe pattern, so that, for each projection pixel, the on time and the off time are set to the same set time, and the off timing is shared so that the on time and the off time are continuous as a first measurement time and a second measurement time, or the on timing is shared so that the off time and the on time are continuous as a first measurement time and a second measurement time, the measurement unit measures a three-dimensional shape of the measurement object by utilizing an average value of the event output time difference obtained at the first measurement time and the event output time difference obtained at the second measurement time.

Images