Three-dimensional measuring device

Generate images and calculate brightness information through event data, which solves the problem that event cameras cannot directly measure brightness values, and realizes efficient measurement of three-dimensional shapes, which is suitable for high-speed moving measurement objects.

CN114424020BActive Publication Date: 2024-11-01DENSO WAVE INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080064477.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-10-28
Filing Date
2020-10-27
Publication Date
2024-11-01
Estimated Expiration
2040-10-27

AI Technical Summary

Technical Problem

When using the data output from the event camera, the existing three-dimensional measurement device cannot directly measure the brightness value, resulting in the inability to perform three-dimensional shape measurement based on the brightness value.

Method used

A three-dimensional measurement device is designed to generate images using event data, and to calculate brightness information through the time difference or output number of positive brightness changes and negative brightness changes, thereby realizing the measurement of three-dimensional shape. The device adopts a phase shift method, a short pulse light emitting or a light-cutting method in the projection unit. The shooting unit uses an event camera to output two-dimensional point data, and the measuring unit performs three-dimensional shape measurement based on these data.

Benefits of technology

The accurate measurement of three-dimensional shapes through event data is realized, and the data traffic and image processing burden can be reduced while generating images at high speed, and is suitable for high-speed moving measurement objects.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114424020B_ABST
    Figure CN114424020B_ABST
Patent Text Reader

Abstract

The present invention provides a three-dimensional measuring device (10) that measures the three-dimensional shape of a measuring object or information representing the shape based on event data. In the device (10), a predetermined stripe pattern is optically projected from a projection unit (20). In the projection state, the measuring object (R) is optically photographed by a photographing unit (30) to obtain a photographed image based on the event data. That is, in the photographing unit (30), event data including two-dimensional point data that determines the position of the pixel is output from a photographing element corresponding to a pixel having a brightness change when receiving the stripe pattern, and a photographed image of the measuring object (R) is obtained based on the event data. The photographing element is configured to output event data of a positive brightness change in the case of a brightening brightness change, and output event data of a negative brightness change in the case of a darkening brightness change. Furthermore, the measuring unit (40) obtains brightness information based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the photographed image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a three-dimensional measuring device for optically measuring the three-dimensional shape of a measurement object or information representing the shape, and in particular to a three-dimensional measuring device equipped with an event camera. Background Art

[0002] Conventionally, as a three-dimensional measuring device that optically measures the three-dimensional shape of a measurement target object or information indicating the shape, for example, a device using a phase shift method is known.

[0003] The phase shift method is a method of optically measuring the shape of a measurement object projecting a stripe pattern image by projecting multiple stripe pattern images (optical images) with phase shifts. As a technology for performing three-dimensional measurement using the phase shift method, a three-dimensional measuring device disclosed in the following patent document 1 is known. The three-dimensional measuring device assigns stripes of each phase to light of different wavelengths, projects the stripe pattern image obtained by synthesizing them onto the measurement object, and photographs the measurement object projecting the stripe pattern image with a color camera. In addition, by extracting each color component from the photographed image and performing phase calculation in one shot, the time required for measuring the three-dimensional shape is shortened.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 3723057

[0007] Patent Document 2: U.S. Patent Application Publication No. 2016 / 0227135 Summary of the invention

[0008] Technical Problems to be Solved by the Invention

[0009] However, as a technology for generating an image of a measurement object at a higher speed, an event camera disclosed in Patent Document 2 mentioned above is known.

[0010] The event camera is a brightness value differential output camera developed based on the structure of the biological retina. It is configured to sense brightness changes in pixel units and output its coordinates, time, and polarity of brightness changes. According to this structure, the event camera has the characteristic of not outputting pixel information without brightness changes, that is, redundant data (event data) like previous cameras. Therefore, it is possible to generate images of the measured object at a higher speed by reducing the amount of data communication and lightweighting image processing.

[0011] However, in the captured image of the measurement object generated using the event data output from the event camera, even if the presence or absence of brightness changes of the pixel unit can be grasped from the captured image, the brightness value cannot be directly measured. Therefore, in the three-dimensional measurement method such as the phase shift method using the brightness value, there is a problem that the three-dimensional shape of the measurement object cannot be measured.

[0012] The present invention is proposed to solve the above-mentioned technical problems, and its object is to provide a device for measuring the three-dimensional shape of a measurement object or information representing the shape using event data.

[0013] Technical solutions for solving technical problems

[0014] In order to achieve the above-mentioned object, a three-dimensional measuring device (10) described in a first exemplary embodiment includes:

[0015] A projection unit (20) projects a predetermined stripe pattern onto a measurement object (R);

[0016] an imaging unit (30) for imaging the object to be measured on which the predetermined stripe pattern is projected; and

[0017] A measuring unit (40) measures the three-dimensional shape of the object to be measured by a phase shift method using brightness information obtained from the image captured by the imaging unit.

[0018] It is characterized in that

[0019] The imaging unit includes an imaging element that outputs event data including two-dimensional point data specifying the position of a pixel corresponding to a pixel having a brightness change when receiving light, and generates the captured image based on the event data output from the imaging element.

[0020] The imaging element is configured to output event data of a positive brightness change in the case of a brightening brightness change, and to output event data of a negative brightness change in the case of a darkening brightness change.

[0021] The measuring unit obtains the brightness information based on a time difference between output of the event data of the positive brightness change and output of the event data of the negative brightness change of the pixel unit in the captured image.

[0022] In addition, the three-dimensional measuring device (10) described in the second exemplary embodiment includes:

[0023] A projection unit (20) projects a predetermined stripe pattern onto a measurement object (R);

[0024] an imaging unit (30) for imaging the object to be measured on which the predetermined stripe pattern is projected; and

[0025] A measuring unit (40) measures the three-dimensional shape of the object to be measured by a phase shift method using brightness information obtained from the image captured by the imaging unit.

[0026] It is characterized in that

[0027] The imaging unit includes an imaging element that outputs event data including two-dimensional point data specifying the position of a pixel corresponding to a pixel having a brightness change when receiving light, and generates the captured image based on the event data output from the imaging element.

[0028] The projection unit projects the specified stripe pattern by using short pulse light emission that emits light multiple times per unit time.

[0029] The measuring unit obtains the brightness information based on the number of times the event data is output per unit time by the pixel unit in the captured image.

[0030] Furthermore, the three-dimensional measuring device (10) described in the third exemplary embodiment includes:

[0031] A projection unit (20) projects a pattern for a light section method onto a measurement object (R);

[0032] an imaging unit (30) for imaging the object to be measured on which the predetermined pattern is projected; and

[0033] a measuring unit (40) for measuring a three-dimensional shape of the object to be measured photographed by the photographing unit,

[0034] It is characterized in that

[0035] The pattern for the light sectioning method is projected in such a way that the brightness changes at a certain ratio in a first direction and the brightness does not change in a second direction orthogonal to the first direction.

[0036] The imaging unit includes an imaging element that outputs event data including two-dimensional point data specifying the position of a pixel corresponding to a pixel having a brightness change when receiving light, and generates the captured image based on the event data output from the imaging element.

[0037] The measuring unit measures a three-dimensional shape of the object to be measured by a light section method based on positions of the event data outputted in the same time period in a captured image capturing the object to be measured.

[0038] In addition, the reference numerals in the above-mentioned parentheses indicate the corresponding relationship with the specific means described in the embodiment to be described later.

[0039] Effects of the Invention

[0040] In the three-dimensional measuring device described in the first exemplary embodiment, the imaging unit that captures the object to be measured projecting a predetermined stripe pattern from the projection unit includes an imaging element that outputs event data including two-dimensional point data that specifies the position of the pixel corresponding to the pixel having a brightness change when receiving light, and generates a captured image based on the event data output from the imaging element. The imaging element is configured to output event data of a positive brightness change in the case of a brightening brightness change, and output event data of a negative brightness change in the case of a darkening brightness change. Furthermore, the measuring unit obtains brightness information based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.

[0041] In the projection unit, when projecting a predetermined stripe pattern for the phase shift method, for example, the R color light emitting state, the G color light emitting state, and the B color light emitting state are repeated in a predetermined period according to the pixel unit, and the respective light emitting time is adjusted separately. On the other hand, in the shooting unit, for each pixel unit in the captured image, when a brightness change of brightening is generated by receiving light, event data of positive brightness change is output, and when a brightness change of darkening is generated by eliminating the light, event data of negative brightness change is output. In this way, at a certain pixel level, for example, from the R color light emitting state to the G color light emitting state, event data of positive brightness change is output at the moment when the R color light emitting starts, event data of negative brightness change is output at the moment when the R color light emitting ends, and event data of positive brightness change is output at the moment when the G color light emitting starts. Here, the longer the time from the moment when the R color light emitting starts to the moment when the R color light emitting ends, the brighter the R color is, so the brightness information (brightness value) of the R color can be obtained based on the time from the moment when the R color light emitting starts to the moment when the R color light emitting ends. Therefore, the brightness information can be obtained based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image, and the three-dimensional shape of the object to be measured can be measured by the phase shift method using the obtained brightness information. In other words, the three-dimensional shape of the object to be measured can be measured using the event data.

[0042] In the three-dimensional measuring device described in the second exemplary embodiment, the imaging unit that captures the object to be measured on which the predetermined stripe pattern is projected from the projection unit includes an imaging element that outputs event data including two-dimensional point data that specifies the position of the pixel corresponding to the pixel having a brightness change when receiving light, and generates a captured image based on the event data output from the imaging element. Furthermore, the projection unit projects the predetermined stripe pattern using short pulse light emission that emits light multiple times per unit time, and the measuring unit obtains brightness information based on the number of times per unit time of output of the event data of the pixel unit in the captured image.

[0043] In the projection unit, for the purpose of reducing power consumption, the above-mentioned R color light emitting state, G color light emitting state, and B color light emitting state can be realized respectively by short pulse light emission. In this case, the more the number of short pulse light emission per unit time ensured for each light emitting state, the brighter the light emission color. In this way, in the pixel unit of the imaging element, since the event data of positive brightness change and the event data of negative brightness change are output corresponding to one short pulse light emission, the brightness information (brightness value) of the light emission color can be obtained based on the number of short pulse light emission per unit time, that is, the number of outputs of event data per unit time. Therefore, the three-dimensional shape of the measured object can be measured by the phase shift method using the brightness information obtained based on the number of outputs of event data per unit time. That is, the three-dimensional shape of the measured object can be measured using event data.

[0044] In the three-dimensional measuring device described in the second exemplary embodiment, the imaging unit that captures the object to be measured on which the pattern for the optical section method is projected from the projection unit includes an imaging element that outputs event data including two-dimensional point data that specifies the position of the pixel corresponding to a pixel whose brightness changes when receiving light, and generates a captured image based on the event data output from the imaging element. The pattern for the optical section method is projected from the projection unit in such a manner that the brightness changes at a certain ratio in a first direction and does not change in a second direction orthogonal to the first direction. Furthermore, the measuring unit measures the three-dimensional shape of the object to be measured based on the position of the event data output in the same time period in the captured image of the object to be measured.

[0045] In the projection unit, in a state where the R color light emitting state, the G color light emitting state, and the B color light emitting state are repeated in a prescribed cycle, for example, when the R color light is emitted, the longer the time from the moment when the R color light emission starts to the moment when the R color light emission ends, the brighter the R color. In the shooting element that receives light according to the projection state, for each unit time ensured for the light emitting state, event data of positive brightness change is output at the moment when the R color light emission starts, and event data of negative brightness change is output at the moment when the R color light emission ends. Therefore, for example, in the case where the shooting unit only shoots a pattern for the light cutting method projected in a manner that the brightness of the R color changes from the left to the right in the left-right direction (first direction) at a certain ratio and the brightness of the R color does not change in the up-down direction (second direction), the event data group of negative brightness change output in the same time period is shot as a line in the up-down direction, and moves from the left to the right in the above-mentioned unit time. The event data group moving in a linear shape can utilize the linear laser used in the light sectioning method, so the three-dimensional shape of the object to be measured can be measured by the light sectioning method based on the position of the event data output in the same time period within the unit time ensured for each light emission state. In other words, the three-dimensional shape of the object to be measured can be measured using the event data.

[0046] More specifically, the measuring unit functions as a judging unit that judges whether the relative distance change of the measured object relative to the imaging unit is greater than or equal to a specified distance based on the difference between the last measurement result and the previous measurement result. Furthermore, when the measuring unit judges that the relative distance change is less than the specified distance, the projection unit projects a specified stripe pattern using short pulse light emission, and the measuring unit obtains brightness information based on the number of outputs per unit time of event data of the pixel unit in the captured image. On the other hand, when the measuring unit judges that the relative distance change is greater than or equal to the specified distance, the projection unit projects a specified stripe pattern using single pulse light emission, and the measuring unit obtains brightness information based on the time difference between the output of event data of positive brightness change and the output of event data of negative brightness change of the pixel unit in the captured image.

[0047] Thus, it is possible to switch between a structure for calculating brightness information based on the number of outputs of event data per unit time and a structure for calculating brightness information based on the time difference between the output of event data of positive brightness change and the output of event data of negative brightness change of pixel units in the captured image, in accordance with a change in the relative distance of the measured object with respect to the imaging unit.

[0048] In another example, the measuring unit functions as a judging unit, and the judging unit judges whether the relative distance change of the measured object relative to the imaging unit is greater than or equal to the specified distance based on the difference between the last measurement result and the previous measurement result. Moreover, when the measuring unit judges that the relative distance change is greater than or equal to the specified distance, the projection unit projects a pattern for the optical section method, and the measuring unit measures the three-dimensional shape of the measured object by the optical section method based on the position of the event data output at the same time in the captured image of the measured object. On the other hand, when the measuring unit judges that the relative distance change is less than the specified distance, the projection unit projects a specified stripe pattern for the phase shift method, and the measuring unit measures the three-dimensional shape of the measured object by the phase shift method using the brightness information obtained from the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.

[0049] Thus, in accordance with the change in the relative distance of the object to be measured with respect to the imaging unit, it is possible to switch between a structure for measuring the three-dimensional shape of the object to be measured by a light section method based on the position of event data output in the same time period in the captured image, and a structure for measuring the three-dimensional shape of the object to be measured by a phase shift method using brightness information obtained by using a time difference between the output of event data of a positive brightness change and the output of event data of a negative brightness change of a pixel unit in the captured image. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a block diagram showing a schematic configuration of a three-dimensional measuring device according to the first embodiment.

[0051] Figure 2 (A) is an explanatory diagram for explaining the R color luminous state, G color luminous state, and B color luminous state at a certain pixel level when a predetermined stripe pattern is projected. Figure 2 (B) in the figure is to explain Figure 2 (A) is an illustration of the R color light emitting state, the G color light emitting state, and the B color light emitting state at different pixel levels.

[0052] Figure 3 It is an explanatory diagram for explaining the R color light emission state, the G color light emission state, and the B color light emission state when a prescribed stripe pattern is projected by short pulse light emission according to the second embodiment.

[0053] Figure 4 It is an explanatory diagram illustrating a pattern for a light cutting method according to the third embodiment.

[0054] Figure 5 This is to illustrate the use of only projected Figure 4 An explanatory diagram of a captured image generated by event data of a state output of a pattern used in a light sectioning method, Figure 5 (A) shows the state 10 μs after the projection of the pattern for the light sectioning method is started. Figure 5 (B) shows the state after 50 μs. Figure 5 (C) in the figure shows the state after 100 μs.

[0055] Figure 6 This is to explain the use of projecting a circular object for measurement. Figure 4 An explanatory diagram of a captured image generated by event data of a state output of a pattern used in a light sectioning method, Figure 6 (A) shows the state 10 μs after the projection of the pattern for the light sectioning method is started. Figure 6 (B) shows the state after 50 μs. Figure 6 (C) in the figure shows the state after 100 μs. DETAILED DESCRIPTION

[0056] [First embodiment]

[0057] Hereinafter, a first embodiment of a three-dimensional measuring device for measuring the three-dimensional shape of a measurement object or information indicating the shape according to the present invention will be described with reference to the drawings.

[0058] The three-dimensional measuring device 10 according to the present embodiment is a device for optically measuring the three-dimensional shape of the measuring object R. Figure 1 As shown in FIG. 1 , the three-dimensional measuring device 10 is configured as independent parts, each of which includes: a projection unit 20 for projecting a predetermined stripe pattern (optical pattern) for the phase shift method onto the measuring object R; a photographing unit 30 for photographing the measuring object on which the predetermined stripe pattern is projected and obtaining an optical image (photographed image); and a measuring unit 40 for measuring or restoring the three-dimensional shape of the measuring object based on the photographed image. The three-dimensional measuring device 10 configured in this way can be assembled into a robot hand, for example, to measure the three-dimensional shape of the measuring object R such as a workpiece that moves relatively at high speed relative to the hand.

[0059] The projection unit 20 is a so-called DLP (Digital Light Processing) projector, which projects the above-mentioned predetermined stripe pattern by reflecting light from a light source through a DMD (Digital Mirror Device) element. The DMD element is a device that arranges tiny mirrors corresponding to each pixel of the image projected on the screen in an array shape, and is configured to switch the light emitted to the screen by changing the angle of each mirror in microseconds. Therefore, the color scale of the reflected light can be changed according to the ratio of the time when each mirror is turned on and the time when each mirror is turned off, thereby performing a color scale display based on the image data of the projected image. In particular, the light incident on the DMD element is prepared in R color (red), G color (green), and B color (blue), and the R color luminous state of luminous light by reflecting the R color by the reflector, the G color luminous state of luminous light by reflecting the G color by the reflector, and the B color luminous state of luminous light by reflecting the B color by the reflector are repeated in a short predetermined cycle, and the respective luminous time is adjusted separately, thereby projecting a color image. In such a structure, the longer the emission time of single pulse emission that emits light once within a unit time ensured for each emission state, the brighter the emission color.

[0060] In this embodiment, as the predetermined stripe pattern (optical pattern) for the above-mentioned phase shift method, a sine wave pattern determined by the brightness value I (x, y, n) according to the following formula (1) is used. That is, when the number of phase shifts is N, the brightness value I (x, y, n) of the N phase-shifted grid images (striped images) is expressed by formula (1).

[0061] I(x,y,n)=a(x,y)cos{θ(x,y)+2πn / N}+b(x,y)…(1)

[0062] Here, point (x, y) is a point in the grid image, a(x, y) represents the brightness amplitude, b(x, y) represents the background brightness, θ(x, y) represents the phase of the grid with n=0, and the distance z to the point (x, y) is measured corresponding to θ(x, y) calculated based on the brightness values ​​I(x, y, n) of N grid images.

[0063] Specifically, in this embodiment, based on the premise that three grid images are obtained corresponding to one cycle based on the above-mentioned R color light emitting state, G color light emitting state, and B color light emitting state, as N=3, the brightness value I(x, y, 0) of the R color light emitting state, the brightness value I(x, y, 1) of the G color light emitting state, and the brightness value I(x, y, 2) of the B color light emitting state are obtained from the captured image. Therefore, the predetermined stripe pattern (optical pattern) used for the phase shift method in this embodiment is composed of a sine wave pattern composed of only R color, a sine wave pattern composed of only G color, and a sine wave pattern composed of only B color so that the phases are shifted by 2π / 3.

[0064] The photographing unit 30 is a photographing device equipped with a photographing element 30 that functions as a so-called event camera (Event-Based Camera) and a circuit, a data output circuit, a threshold setting circuit, etc. (none of which are shown in the figure) that drives the photographing element 30. Therefore, the photographing unit 30 includes a photographing element 30 that outputs event data (specifically, two-dimensional point data, time, and polarity of brightness change) including two-dimensional point data that determines the position of each pixel corresponding to the pixel that has brightness change when receiving light through the event camera, and is configured to generate a captured image based on the event data output from the photographing element. The photographing device 30 constituting the event camera is known, for example, from U.S. Patent Application Publication No. 2016 / 0227135.

[0065] Therefore, in the imaging unit 30, when each pixel unit of the captured image generates a brightness change that becomes brighter by receiving light, event data of a positive brightness change is output, and when the brightness change that becomes darker is generated by eliminating the light, event data of a negative brightness change is output. Image data of the captured measurement object can be generated by plotting two-dimensional point data of a plurality of event data outputted within a certain period as points on a predetermined plane, and the imaging unit 30 is configured to output the image data or event data (two-dimensional point data, time, polarity of brightness change) generated in this way to the measuring unit 40.

[0066] The measuring unit 40 measures the three-dimensional shape of the measuring object R based on the phase shift method, based on the captured image of the measuring object R captured by the imaging unit 30 in a state where the above-mentioned predetermined stripe pattern is projected from the projection unit 20. Specifically, the measuring unit 40 obtains θ(x, y) by obtaining the brightness value I(x, y, 0), the brightness value I(x, y, 1), and the brightness value I(x, y, 2) of the point (x, y) in the captured image, and measures the distance z to the point (x, y) based on the thus-obtained θ(x, y).

[0067] The measuring unit 40 is typically configured as a microcomputer including a CPU 40A and various memories 40B. The CPU 40A sequentially executes each step of a program for three-dimensional measurement stored in advance in the memory 40B, and obtains information representing the three-dimensional shape of the object R to be measured by the three-dimensional measurement.

[0068] On the other hand, in order to perform such three-dimensional measurement, the measuring unit 40, i.e., the CPU 40A, needs to make the projection unit 20 and the imaging unit 30 cooperate with each other, and the program for controlling the cooperation is obviously stored in advance in another area of ​​the memory 40B. Therefore, the CPU 40A can be responsible for the drive control of the projection unit 30 and the control of the light projection timing, the drive control of the imaging unit and the control of the reading timing of the event data output from the imaging unit 30, i.e., the imaging element 31, by executing the control program (see Figure 1 dashed arrow).

[0069] Therefore, the measuring unit 40 functions as a measuring means for three-dimensional measurement and a control means for performing the above-mentioned various controls.

[0070] Thus, in this embodiment, the control function (control means, control unit) of the three-dimensional measuring device 10 is concurrently performed by the computer structure of the measuring unit 40. Of course, the control function (control means, control unit) may also be provided in a dedicated control unit provided independently of the measuring unit 40.

[0071] Here, the three-dimensional measurement process performed by the measuring unit 40 when the three-dimensional shape of the measurement target R is measured using the phase shift method will be described in detail below.

[0072] In this embodiment, an event camera is used as a main part of the imaging unit 30 for accurately imaging the high-speed relatively moving measuring object R. In this structure, event data corresponding to pixels with brightness changes are output, but since the event data does not include brightness values, it is not possible to directly obtain the brightness values ​​(I(x, y, 0), I(x, y, 1), I(x, y, 2)) required for the phase shift method.

[0073] Therefore, in this embodiment, the brightness value (brightness information) is calculated as information representing the three-dimensional shape of the measured object R based on the time difference between the output of event data of a positive brightness change (positive change in brightness polarity) and the output of event data of a negative brightness change (negative change in brightness polarity) of a pixel unit in the captured image.

[0074] To explain in more detail, for example, at a certain pixel level, such as Figure 2 As shown in (A) of FIG. 1 , it is assumed that the R color luminous state, the G color luminous state, and the B color luminous state are repeated at a predetermined period T. Figure 2(A) and Figure 2 In (B) of FIG. 1 , the output of event data of a positive brightness change is illustrated by an upward arrow, and the output of event data of a negative brightness change is illustrated by a downward arrow.

[0075] In this light-emitting state, when the R color light emission starts, the event data of positive brightness change is output (refer to Figure 2 (A) t11), outputs event data of negative brightness change at the moment when the R color light emission ends (refer to Figure 2 Then, at the moment when the G color light emission starts, the event data of positive brightness change is output (refer to Figure 2 t13 in (A)), outputs event data of negative brightness change at the moment when the G color light emission ends (refer to Figure 2 Then, at the moment when the B color light emission starts, the event data of positive brightness change is output (refer to Figure 2 (A) t15), outputs event data of negative brightness change at the moment when B color light emission ends (refer to Figure 2 t16 in (A).

[0076] In addition, for example, Figure 2 As shown in (B) in the example, Figure 2 In the case of different pixel levels of the pixels in (A), event data of positive brightness change is output at the time when R color light emission starts (refer to Figure 2 t21 in (B)), outputs event data of negative brightness change at the moment when the R color light emission ends (refer to Figure 2 Then, at the moment when the G color light emission starts, the event data of positive brightness change is output (refer to Figure 2 t23 in (B)), outputs event data of negative brightness change at the moment when the G color light emission ends (refer to Figure 2 After that, the event data of positive brightness change is output at the moment when the B color light emission starts (refer to Figure 2 t25 in (B)), outputs event data of negative brightness change at the moment when B color light emission ends (refer to Figure 2 t26 in (B).

[0077] Here, the longer the time from the moment when the R color starts to emit light to the moment when the R color ends to be emitted light, the brighter the R color becomes, so the brightness value of the R color can be obtained from the time from the moment when the R color starts to emit light to the moment when the R color ends to be emitted light. Similarly, the brightness value of the G color can be obtained from the time from the moment when the G color starts to emit light to the moment when the G color ends to be emitted light, and the brightness value of the B color can be obtained from the time from the moment when the B color starts to emit light to the moment when the B color ends to be emitted light.

[0078] Therefore, the measuring unit 40 can obtain the brightness value (brightness information) which is information indicating the three-dimensional shape of the object R to be measured based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.

[0079] exist Figure 2 In the example (A), the measuring unit 40 can obtain the brightness value I (x, y, 0) of the R color light emitting state based on t12-t11, which is the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the R color light emitting state. Similarly, the brightness value I (x, y, 1) of the G color light emitting state and the brightness value I (x, y, 2) of the B color light emitting state can be obtained based on the time difference t14-t13 and the time difference t16-t15.

[0080] That is, the measuring unit 40 can obtain brightness information as information indicating the three-dimensional shape of the measurement object based on the time difference between the output of event data of positive brightness change and the output of event data of negative brightness change of the pixel unit in the captured image.

[0081] The three-dimensional shape of the object to be measured can be measured by the phase shift method using each brightness value thus obtained. That is, the three-dimensional shape of the object to be measured can be measured using event data. In addition, as will be described later, when the above-mentioned striped pattern is projected using short pulse light emission that emits light multiple times per unit time, the brightness value can be obtained based on the sum of the time differences between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change that are repeatedly generated per unit time.

[0082] In particular, event data using three-dimensional measurement is output in response to brightness changes, so even measurement objects with a large brightness difference with the background (for example, a black object on a white table), which is not good at conventional phase shift methods that do not use event data, can be accurately measured.

[0083] Furthermore, even when color mixing occurs due to the overlap of color bands of the measurement object and the background, which is not an advantage of the RGB phase shift method using a color camera, accurate measurement can be performed.

[0084] [Second Embodiment]

[0085] Next, a three-dimensional measuring device according to a second embodiment will be described with reference to the drawings.

[0086] In the second embodiment, the main difference from the first embodiment is that the projection unit 20 utilizes a short pulse light emission projection of the above-mentioned prescribed stripe pattern with a short pulse width (time width) that emits light multiple times per unit time. Therefore, the same reference numerals are given to the structural parts substantially the same as those in the first embodiment, and their description is omitted.

[0087] In this embodiment, the R color light emitting state, the G color light emitting state, and the B color light emitting state are realized by short pulse light emission for the purpose of reducing power consumption in the projection unit 20. In this structure, the more the number of short pulse light emission per unit time (T / 3) ensured for each light emitting state, the brighter the light emission color.

[0088] Then, in the pixel unit of the imaging element 31, as shown in FIG. Figure 3 As shown, event data of positive brightness change and event data of negative brightness change are output corresponding to a short pulse light emission. Therefore, the brightness value of the light emission color can be calculated based on the number of short pulse light emission per the above unit time, that is, the number of outputs of event data per the above unit time.

[0089] exist Figure 3 In the example, the brightness value I(x, y, 0) in the R color light emitting state can be obtained based on the number of outputs of the event data related to the R color light emitting state per the above-mentioned unit time (the number of outputs between t31 and t32). Similarly, the brightness value I(x, y, 1) in the G color light emitting state can be obtained based on the number of outputs of the event data related to the G color light emitting state per the above-mentioned unit time (the number of outputs between t33 and t34). In addition, the brightness value I(x, y, 2) in the B color light emitting state can be obtained based on the number of outputs of the event data related to the B color light emitting state per the above-mentioned unit time (the number of outputs between t35 and t36).

[0090] That is, the measuring unit 40 can obtain brightness information as information indicating the three-dimensional shape of the measurement target object from the number of outputs per unit time of event data of the pixel unit in the captured image.

[0091] The three-dimensional shape of the object can be measured by the phase shift method using the brightness values ​​thus obtained. That is, even if the projection unit 20 is configured to project the predetermined stripe pattern using short pulse light emission, the three-dimensional shape of the object can be measured using event data.

[0092] In addition, according to the change in the relative distance of the measured object R with respect to the imaging unit 30, it is also possible to switch between a structure for calculating brightness information based on the number of outputs of event data per unit time (the characteristic structure of the second embodiment) and a structure for calculating brightness information based on the time difference between the output of event data of positive brightness change and the output of event data of negative brightness change of pixel units in the captured image (the characteristic structure of the above-mentioned first embodiment).

[0093] Specifically, the projection unit 20 is configured to be able to switch between a state in which a predetermined stripe pattern is projected using short pulse light emission that emits light multiple times per unit time and a state in which a stripe pattern is projected using a single pulse light emission that emits light once per unit time. In addition, the measuring unit 40 functions as a judgment unit that judges whether the relative distance change of the measurement object R relative to the imaging unit 30 is greater than or equal to the first predetermined distance based on the difference between the last measurement result and the previous measurement result (for example, the measurement result of the last measurement). Moreover, when the measurement unit 40 judges that the relative distance change is less than the first predetermined distance, the projection unit 20 projects a predetermined stripe pattern using short pulse light emission, and the measurement unit 40 obtains brightness information based on the number of outputs per unit time of event data of a pixel unit in the captured image. On the other hand, when the measurement unit 40 judges that the relative distance change is greater than or equal to the first predetermined distance, the projection unit 20 projects a predetermined stripe pattern using single pulse light emission, and the measurement unit 40 obtains brightness information based on the time difference between the output of event data of a positive brightness change and the output of event data of a negative brightness change of a pixel unit in the captured image. This brightness information is information indicating the three-dimensional shape of the measurement object R.

[0094] Thus, when measuring the three-dimensional shape of the measurement object R whose relative distance change is relatively small, that is, the measurement object R whose movement is relatively small, the brightness information can be obtained based on the number of outputs of the event data per unit time, thereby improving the robustness against the influence of the color and ambient light of the measurement object R. In contrast, when measuring the three-dimensional shape of the measurement object R whose relative distance change is relatively large, that is, the measurement object R whose movement is relatively large, the brightness information can be obtained based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change, thereby more quickly obtaining the brightness information than the case where the brightness information is obtained based on the number of outputs of the event data as described above.

[0095] [Third Embodiment]

[0096] Next, a three-dimensional measuring device according to a third embodiment will be described with reference to the drawings.

[0097] The third embodiment is different from the first embodiment in that the three-dimensional shape of the object to be measured is measured using the light section method. Therefore, the same reference numerals are given to the components substantially the same as those in the first embodiment, and their description is omitted.

[0098] In the present embodiment, the three-dimensional shape of the object to be measured is measured by the light section method, and therefore the projection unit 20 is configured to project a pattern for the light section method onto the object to be measured R. The pattern for the light section method in the present embodiment is projected in such a manner that the brightness of any of the R, G, and B colors increases at a certain ratio from the left to the right in a first direction and does not change in a second direction orthogonal to the first direction.

[0099] Furthermore, the measuring unit 40 is configured to measure the three-dimensional shape of the measurement object by a light section method based on the positions of event data outputted in the same time period in the captured image.

[0100] Hereinafter, the light section method will be described using the pattern for the light section method projected as described above.

[0101] In the projection unit 20, in the state where the R color light emitting state, the G color light emitting state, and the B color light emitting state are repeated in a predetermined cycle, for example, when the R color light is emitted, the longer the time from the moment when the R color light emission starts to the moment when the R color light emission ends, the brighter the R color. In the imaging element 31 of the imaging unit 30 that receives light corresponding to the projection state, event data of positive brightness change is output at the moment when the R color light emission starts, and event data of negative brightness change is output at the moment when the R color light emission ends within the unit time ensured for each light emitting state.

[0102] Here, if Figure 4 As illustrated, it is assumed that the imaging unit 30 only captures a pattern for the optical sectioning method (captured image), wherein the pattern for the optical sectioning method is projected in a manner such that the brightness of the R color increases from the left side to the right side in the left-right direction (first direction) at a certain ratio and the brightness of the R color does not change in the up-down direction (second direction).

[0103] In this case, in the shooting unit 30, the event data group of negative brightness change output in the same time period is captured in the form of a dynamic image in the vertical direction as a line, and moves from the left to the right in the above-mentioned unit time. This is because the closer to the left end of the above-mentioned pattern, the darker it is, so the moment when the R color light emission ends is earlier as it approaches the left end, and the closer to the right end of the above-mentioned pattern, the brighter it is, so the moment when the R color light emission ends is later as it approaches the right end, and the moment when the R color light emission ends in the vertical direction is almost the same.

[0104] Specifically, for example, in a state where the image is captured by the image capturing unit 30 10 μs after the projection unit 20 starts projecting the pattern for the light section method, Figure 5 As shown in (A) in FIG. 1 , the event data group S is captured as a line in the vertical direction near the left end of the above pattern. Then, for example, after 50 μs, as shown in FIG. Figure 5 As shown in (B) in FIG. 1 , the event data group S captured as a line moves to the right. After 100 μs, as shown in FIG. Figure 5 As shown in (C) in FIG. 1 , the event data group S captured in a linear form moves further to the right.

[0105] The event data group captured in such a linear motion can also be used as a linear laser for the optical sectioning method. Specifically, for example, for a circular object R to be measured, 10 μs after the projection unit 20 starts projecting the pattern for the optical sectioning method, Figure 6 In the state where the event data group S shown in (A) in FIG. 1 has not reached the measurement object R, the event data group S is captured as a line in the vertical direction. Then, for example, after 50 μs, as shown in FIG. Figure 6 As shown in (B) in FIG. 1 , when the event data set S reaches the measurement object R, the arrangement of the event data set S in the measurement object R is changed according to the shape. Figure 6 As shown in (C) in FIG. 1 , the arrangement of the event data set S in the object R to be measured is photographed to change according to the shape.

[0106] Therefore, within the unit time ensured for each light emitting state, the three-dimensional shape of the object can be measured by the light section method based on the position of the event data output in the same time period by the measuring unit 40. That is, the three-dimensional shape of the object can be measured using the event data.

[0107] In addition, the pattern used in the optical cutting method is not limited to being projected in a manner that the brightness of the R color changes in a manner that increases from the left side to the right side in the left-right direction at a certain ratio and the brightness of the R color does not change in the up-down direction. For example, it is also possible to project in a manner that the brightness of the G color (B color) changes in a manner that increases from the left side to the right side in the left-right direction at a certain ratio and the brightness of the G color (B color) does not change in the up-down direction. In addition, the pattern used in the optical cutting method is not limited to being projected in a manner that the brightness of the R color (G color, B color) changes in a manner that increases from the left side to the right side in the left-right direction at a certain ratio and the brightness of the R color (G color, B color) does not change in the up-down direction. It is also possible to project in a manner that the brightness of any of the R color, G color, and B color changes in a first direction at a certain ratio and the brightness does not change in a second direction orthogonal to the first direction.

[0108] In addition, it is also possible to switch between a structure for measuring the three-dimensional shape of the measured object using the light section method (the characteristic structure of the third embodiment) and a structure for measuring the three-dimensional shape of the measured object using the phase shift method (the characteristic structure of the above-mentioned first embodiment) according to the change in the relative distance of the measured object R relative to the imaging unit 30.

[0109] Specifically, the projection unit 20 is configured to be able to switch between a state of projecting a pattern for the light section method and a state of projecting a predetermined stripe pattern for the phase shift method. In addition, the measuring unit 40 functions as a judgment unit that judges whether the relative distance change of the measurement object R relative to the imaging unit 30 is greater than or equal to the second predetermined distance based on the difference between the last measurement result and the previous measurement result (for example, the last measurement result). Moreover, when the measuring unit 40 judges that the relative distance change is greater than or equal to the second predetermined distance, the projection unit 20 projects the pattern for the light section method, and the measuring unit 40 measures the three-dimensional shape of the measurement object by the light section method based on the position of the event data output in the same time period in the captured image of the measurement object R. On the other hand, when the measuring unit 40 judges that the relative distance change is less than the second predetermined distance, the projection unit 20 projects the predetermined stripe pattern using single pulse light emission, and the measuring unit 40 measures the three-dimensional shape of the measurement object by the phase shift method based on the brightness information obtained based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.

[0110] Thus, when measuring the three-dimensional shape of the object R to be measured whose relative distance change is relatively large, that is, the object R to be measured whose movement is relatively large, the three-dimensional shape of the object R to be measured is measured by the light section method based on the position of the event data output in the same time period, and the three-dimensional shape can be measured more quickly than when the phase shift method is used. On the other hand, when measuring the three-dimensional shape of the object R to be measured whose relative distance change is relatively small, that is, the object R to be measured whose movement is relatively small, the three-dimensional shape of the object R to be measured is measured by the phase shift method using the brightness information obtained from the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change, and the three-dimensional shape can be measured with high accuracy compared with the case of using the light section method.

[0111] In addition, the present invention is not limited to the above-mentioned embodiments, and may be embodied as follows, for example.

[0112] (1) The three-dimensional measuring device 10 is not limited to being moved in a state assembled on a robot hand and measuring the three-dimensional shape of a relatively moving object to be measured. For example, the three-dimensional measuring device 10 may be used in a fixed state and measure the three-dimensional shape of an object to be measured that moves on a conveyor line.

[0113] (2) The three-dimensional measuring device 10 may be configured as an information processing terminal in which the projection unit 20 , the imaging unit 30 , and the measuring unit 40 are independent components and the measuring unit 40 can communicate with the projection unit 20 and the imaging unit 30 wirelessly or by wire.

[0114] (3) The present invention can also be applied to a three-dimensional measuring device that measures the three-dimensional shape of a measurement object using a dot pattern projection method. In this three-dimensional measuring device, by providing brightness information in the same manner as the light cutting method described in the third embodiment, the number of points that can be identified depends on the resolution of the brightness information, but each point can be identified individually. In this dot pattern projection method, the amount of data transmission can be reduced compared to the light cutting method, so that the function can be accelerated.

[0115] Description of Reference Numerals

[0116] 10: Three-dimensional measuring device

[0117] 20: Projection Department

[0118] 30: Photography Department

[0119] 31: Image capturing element that outputs event data (functions as an event camera)

[0120] 40: Measurement unit (functions as a measurement means and a control means)

[0121] R: Measurement object

Claims

1. A three-dimensional measuring device comprising: a projection unit, the projection unit being a DLP projector, and projecting a plurality of stripe-shaped patterns sequentially during a predetermined period onto the measurement object by causing the DMD element to reflect light from the light source, the plurality of stripe-shaped patterns being formed as a plurality of optical patterns that change according to a predetermined waveform intensity in only one direction of two mutually orthogonal directions set in the measurement object and are phase-shifted with each other; an imaging unit that optically captures the object to be measured on which the stripe pattern is projected and obtains a two-dimensional image formed of a plurality of pixels at the predetermined period; as well as a measuring unit that measures a three-dimensional shape of the measurement object by applying a phase shift method to the plurality of captured images acquired by the imaging unit at each predetermined period, The three-dimensional measuring device is characterized in that The imaging unit is configured as an event camera, which includes an imaging element having pixels arranged two-dimensionally, and outputs event data including two-dimensional point data indicating the position of the pixel, time, and polarity of the brightness change for each pixel having a brightness change when the imaging element sequentially receives the plurality of stripe patterns, The imaging element is configured to output event data of positive brightness change in each pixel of the plurality of pixels when the event data changes with time series as a brightening brightness change, and conversely output event data of negative brightness change in the case of a darkening brightness change, The measuring unit is configured to obtain, in each of the plurality of pixels, information corresponding to brightness corresponding to a time difference between output of the event data of the positive brightness change and output of the event data of the negative brightness change as information representing a three-dimensional shape of the object to be measured.

2. A three-dimensional measuring device comprising: a projection unit, the projection unit being a DLP projector, and projecting a plurality of stripe-shaped patterns sequentially during a predetermined period onto the measurement object by causing the DMD element to reflect light from the light source, the plurality of stripe-shaped patterns being formed as a plurality of optical patterns that change according to a predetermined waveform intensity in only one direction of two mutually orthogonal directions set in the measurement object and are phase-shifted with each other; an imaging unit that optically captures the object to be measured on which the stripe pattern is projected and obtains a two-dimensional image formed of a plurality of pixels at the predetermined period; as well as a measuring unit that measures a three-dimensional shape of the measurement object by applying a phase shift method to the plurality of captured images acquired by the imaging unit at each predetermined period, The three-dimensional measuring device is characterized in that The imaging unit is configured as an event camera, which includes an imaging element having pixels arranged two-dimensionally, and outputs event data including two-dimensional point data indicating the position of the pixel, time, and polarity of the brightness change for each pixel having a brightness change when the imaging element sequentially receives the plurality of stripe patterns, The projection unit is configured to project each of the plurality of stripe patterns by short pulse light emission that emits light a plurality of times per unit time in each of the cycles. The measuring unit is configured to obtain, in each of the plurality of pixels, information corresponding to brightness corresponding to the number of outputs of the event data per the unit time as information indicating the three-dimensional shape of the object to be measured.

3. The three-dimensional measuring device according to claim 2, characterized in that: The projection unit is configured to switch between a state in which the predetermined stripe pattern is projected using short pulse light emission that emits light multiple times within the unit time and a state in which the predetermined stripe pattern is projected using a single pulse light emission that emits light once within the unit time. The imaging element is configured to output the event data of a positive brightness change in the case of a brightening brightness change, and to output the event data of a negative brightness change in the case of a darkening brightness change, A determination unit is provided for determining whether a change in the relative distance of the object to be measured with respect to the imaging unit is greater than or equal to a predetermined distance based on a difference between a previous measurement result and a previous measurement result by the measuring unit, When the determination unit determines that the relative distance change is less than the specified distance, the projection unit projects the specified striped pattern based on the short pulse light emission, and the measurement unit obtains information corresponding to the brightness based on the number of outputs per unit time of the event data of the pixel unit in the captured image, When the judgment unit determines that the relative distance change is greater than or equal to the specified distance, the projection unit projects the specified striped pattern based on the single pulse light emission, and the measuring unit obtains the information corresponding to the brightness based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.

4. A three-dimensional measuring device comprising: A projection unit, wherein the projection unit is a DLP projector and projects a predetermined optical pattern for the light section method onto the object to be measured by causing the DMD element to reflect light from the light source; an imaging unit that optically images the measurement object on which the predetermined optical pattern is projected, and obtains a captured image; as well as a measuring unit that measures a three-dimensional shape of the object to be measured based on the optical image captured by the imaging unit, The three-dimensional measuring device is characterized in that The optical pattern for the light sectioning method is projected in such a manner that the brightness of light based on the optical pattern changes in a first direction relative to the measurement object at a certain ratio and the brightness does not change in a second direction orthogonal to the first direction. The photographing unit is configured as an event camera, and The imaging unit includes an imaging element having a two-dimensional pixel group that responds to changes in brightness of light when receiving the optical pattern, and The imaging element changes the positions of a plurality of pixels arranged in a line along the second direction in accordance with an increase in the brightness of the light of the optical pattern in the first direction, and sequentially outputs event data, the event data including information indicating the position of the pixel where the brightness change occurs, the time, and the polarity of the brightness change, and is event data accompanying the brightness change in the plurality of pixels arranged in a line, and generating the captured image of the event data, The measuring unit obtains information corresponding to the brightness based on a time difference between output of event data of a positive brightness change and output of event data of a negative brightness change of a pixel unit in the captured image, The three-dimensional shape of the object to be measured is measured based on the information corresponding to the position of the pixel of the event data output per unit time in the captured image and the brightness.

5. The three-dimensional measuring device according to claim 4, characterized in that: The projection unit is configured to switch between a state of projecting the optical pattern for the light section method and a state of projecting a predetermined stripe pattern for the phase shift method. The imaging element is configured to output the event data of a positive brightness change in the case of a brightening brightness change, and to output the event data of a negative brightness change in the case of a darkening brightness change, A determination unit is provided for determining whether a change in the relative distance of the object to be measured with respect to the imaging unit is greater than or equal to a predetermined distance based on a difference between a previous measurement result and a previous measurement result by the measuring unit, When the determination unit determines that the relative distance change is greater than or equal to the predetermined distance, the projection unit projects the optical pattern for the light section method, and the measurement unit measures the three-dimensional shape of the object to be measured by the light section method based on the position of the event data output in the same time period in the captured image of the object to be measured, When the judgment unit determines that the relative distance change is less than the specified distance, the projection unit projects the specified striped pattern used for the phase shift method, and the measuring unit measures the three-dimensional shape of the measured object by the phase shift method based on the information corresponding to the brightness calculated based on the time difference between the output of the event data of the positive brightness change and the output of the event data of the negative brightness change of the pixel unit in the captured image.