Three-dimensional measurement device and three-dimensional measurement program

Abstract

Through the present invention, high-speed and high-precision optical scanning three-dimensional measurement is performed using an event-based sensor.　A three-dimensional-measurement device 8 comprises: a projection unit 7 for projecting a reference light onto an object W while causing the reference light to scan; a light-receiving unit 3 for receiving the reference light reflected by the object W, monitoring a luminance change of the reference light for each pixel, and outputting the position of the pixel, the time at which a luminance change occurred, and a polarity representing the direction of the luminance change as an event, when a luminance change is equal to or greater than a pre-set threshold value; and a three-dimensional-information calculation unit 9 for selecting an event group that includes a designated number of events in which the events are changing from positive polarity to negative polarity, calculating a passage time at which the reference light passes a pixel on the basis of the time of an event included in the event group and the threshold value of the luminance change constituting occurrence of the event, and calculating three-dimensional information of the object W by triangulation on the basis of the passage time.

Description

[Technical field]

[0001] The present invention relates to a three-dimensional measurement technique, and more particularly to an optical scanning type three-dimensional measurement device and a three-dimensional measurement program. [Background technology]

[0002] Many of the three-dimensional measuring devices commercialized for industrial use are based on triangulation using structured illumination. As structured illumination, various light projection methods have been proposed, such as the light section method, phase shift method, and spatial code method. In the light section method, a strip-shaped slit light is projected onto an object while scanning, the object is imaged from an imaging position different from the light projection position, and the distance to the object is calculated by triangulation based on the projection angle of the slit light, the incidence angle of the slit light on the imaging plane, and the baseline length between the projection position and the imaging position. The projection angle of the slit light is obtained, for example, from a command value to a scanner or the passing time of the slit light appearing on the imaging plane, and the incidence angle of the slit light is obtained, for example, from the incidence position of the slit light on the imaging plane. The light section method is generally said to have good distance measurement accuracy, but compared to the phase shift method, spatial code method, etc., there is a problem that it takes a long measurement time because a large number of images are required for one measurement.

[0003] In recent years, event-based image sensors based on a different concept from general frame-based image sensors have been proposed. Frame-based image sensors output frame images at a predetermined cycle by opening and closing the shutter for a predetermined period of time for exposure, whereas event-based image sensors monitor each pixel independently and asynchronously from moment to moment, and when they detect a luminance change exceeding a predetermined threshold, they output the position of the pixel where the luminance change occurred, the time when the luminance change occurred, and the polarity indicating the direction of the luminance change (whether it became brighter or darker) as an event. Event-based image sensors have a wider dynamic range than frame-based image sensors, and are characterized by being high-speed because they only output information on pixels where the luminance change occurred. Therefore, it is believed that the use of event-based image sensors will enable the light-section method to be performed at high speed.

[0004] The slit light of the light-section method has a certain width and has a luminance distribution that is maximum near the center. When a slit light with such a luminance distribution passes through a pixel, the luminance increases as the center of the slit light approaches the pixel, and the luminance decreases when the slit light has passed through the pixel. The time when the slit light passes through the pixel is when the pixel has the maximum luminance. However, an event does not have luminance itself, and generally occurs when the luminance change from the luminance at the time when the previous event occurred is equal to or exceeds a predetermined threshold, so the timing of the event occurrence is not necessarily the time of maximum luminance. In practical use of three-dimensional measurement, it is not necessarily the time of maximum luminance, and it is sufficient to capture the same timing with respect to the luminance distribution of the slit light, but depending on the brightness of the background, it is not known at what point in the luminance distribution of the slit light the event occurs, and the time interval at which the event occurs is not constant. Therefore, since the event-based image sensor cannot accurately detect the time when the slit light passes through the pixel, there is a problem that sufficient distance measurement accuracy cannot be obtained even if the event-based image sensor is directly applied to the light-section method. As techniques related to the present application, for example, the documents listed below are known.

[0005] Patent Document 1 discloses a three-dimensional input device that captures images synchronously using two image sensors while scanning an object with a slit light, stores the frame number at which the maximum brightness is detected for each pixel of each image sensor, matches pixels based on the frame numbers, and calculates distance based on the matched pixels.

[0006] Patent document 2 describes a method for three-dimensionally reconstructing a scene, which includes receiving a first successive event for each pixel from a first sensor, receiving a second successive event for each pixel from a second sensor, and matching a first event in the first successive events to a second event in the second successive events according to minimization of a cost function.

[0007] Patent Document 3 describes a method of projecting a multi-slit pattern light, calculating the shift amount of each slit image from the signal of the multi-slit image detected by a detection optical system with a predetermined magnification, and calculating the height of the sample surface by weighting the calculated shift amount of the multi-slit image according to the shift amount and taking a weighted average. It also describes calculating the height of the sample surface by interpolating two heights.

[0008] Patent document 4 describes that when reflected light from an object is incident on a pixel of an area sensor at a certain time, triangulation is applied to determine the distance to the object based on the baseline length from the projection starting point of the slit light to the principal point position of the light receiving lens, the incident angle of the reflected light with respect to a reference direction, and the deflection angle at that time.

[0009] Patent Document 5 describes the use of a frame number (image capture time) as irradiation angle information of a slit light when performing three-dimensional measurement using the light section method.

[0010] Patent Document 6 describes the use of rising or falling timing of luminance change in each pixel in a three-dimensional measurement device that scans with spot light. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] JP 2000-088539 A [Patent Document 2] Special Publication No. 2018-516395 [Patent Document 3] Patent No. 3542478 [Patent Document 4] JP 2002-090681 A [Patent Document 5] JP 2006-333493 A [Patent Document 6] JP 2003-329419 A Summary of the Invention [Problem to be solved by the invention]

[0012] In view of the problems in the related art, an object of the present invention is to provide a high-speed, high-precision optical scanning three-dimensional measurement technique using an event-based sensor. [Means for solving the problem]

[0013] One aspect of the present disclosure provides a three-dimensional measuring device including a light-projecting unit that projects a reference light onto an object while scanning it; a light-receiving unit that receives the reference light reflected by the object, monitors a change in luminance of the reference light for each pixel, and, if the change in luminance is equal to or greater than a predetermined threshold, outputs the position of the pixel, the time when the luminance change occurred, and a polarity indicating the direction of the luminance change as an event; and a three-dimensional information calculation unit that selects an event group including a specified number of events in which the event changes from positive to negative polarity, calculates a passing time at which the reference light passed through the pixel based on the time of the event included in the event group and a threshold for the luminance change at which the event occurs, and calculates three-dimensional information of the object by triangulation based on the passing time. Another aspect of the present disclosure provides a three-dimensional measurement program that causes a computer to execute the steps of projecting a reference light onto an object while scanning it; receiving the reference light reflected by the object, monitoring a change in luminance of the reference light for each pixel, and outputting, if the change in luminance is equal to or greater than a predetermined threshold, the position of the pixel, the time when the luminance change occurred, and a polarity indicating the direction of the luminance change as an event; selecting an event group in which the event changes from positive to negative polarity and which includes a specified number of events; calculating the passing time when the reference light passed through the pixel based on the time of the event included in the event group and the threshold value of the luminance change at which the event occurs; and calculating the distance to the object by triangulation based on the passing time. Effect of the Invention

[0014] According to one aspect of the present disclosure, even with an event-based sensor in which the timing of occurrence of an event is not constant, the passing time when the reference light passes through a pixel can be calculated more accurately. By selecting a specified number of events, the influence of errors in the time of each event can be mitigated. Meanwhile, by using an event-based sensor, three-dimensional measurement can be performed faster than with a frame-based sensor. As a result, a high-speed and high-precision optical scanning three-dimensional measurement technique can be provided by using an event-based sensor. [Brief description of the drawings]

[0015] [Figure 1] FIG. 2 is a plan view of a stereo camera illustrating the measurement principle of the stereo method. [Diagram 2] FIG. 2 is a plan view of a light-section system illustrating the measurement principle of the light-section method. [Diagram 3] FIG. 1 is a block diagram of a three-dimensional measuring apparatus according to an embodiment. [Figure 4] 11 is a graph of luminance change showing an example of a group of events occurring upon passage of a reference beam. [Diagram 5] 11 is a graph of luminance change showing an example of calculation of the passage time of a reference light. [Figure 6] 11 is a graph of luminance change showing an example of the time it takes for a reference beam to pass through. [Figure 7] 11 is a graph of luminance changes showing an example of an event that does not exist within a specified time period. [Figure 8] 4 is a flowchart showing an example of the operation of the three-dimensional measuring apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In each drawing, the same or similar components are given the same or similar reference numerals. In addition, the embodiments described below do not limit the technical scope and meaning of the terms of the invention described in the claims.

[0017] The measurement principle of the three-dimensional measuring device of this embodiment will be described. For ease of understanding, the measurement principles of the stereo method and the light-section method will be described first. FIG. 1 is a plan view of a stereo camera 1 showing the measurement principle of the stereo method. The stereo camera 1 includes, for example, a left light receiving unit 2 and a right light receiving unit 3, which correspond to two cameras. The left light receiving unit 2 and the right light receiving unit 3 are arranged, for example, in an equal and parallel arrangement. That is, the two light receiving units are separated by a base length B, the optical axes of the two light receiving units are arranged in parallel, the left light receiving surface 4 and the right light receiving surface 5 are arranged in a plane perpendicular to the two optical axes, and the x direction and the y direction of each light receiving surface are oriented in the same direction. Each light receiving surface is, for example, an image sensor in which a plurality of pixels are two-dimensionally arranged, but may also be a line sensor in which a plurality of pixels are one-dimensionally arranged (for example, arranged only in the x direction).

[0018] Here, the position of the pixel on the left light receiving surface 4 that captures the image of point P on the object in the object space is x l Let the pixel position on the right photosensitive surface 5 be x r Then, the parallax between the left light receiving unit 2 and the right light receiving unit 3 is D=x l -x r If the origin of the XYZ coordinate system representing three-dimensional space is placed at the right focal point, the focal lengths of both light-receiving parts are f, and the pixel pitch of both light-receiving surfaces is 1, then the distance Z to point P on the object (depth to point P; the same applies below) can be calculated using the following formula.

[0019]

number

[0020] Since the baseline length B and focal length f are constants determined by the design of the stereo camera 1, if the image of point P on the right light-receiving surface 5 that corresponds to the image of point P on the left light-receiving surface 4 can be detected by image processing such as pattern matching, the disparity D can be found, and therefore the distance Z to the target point P can be found.

[0021] In the light section system, for example, the left light receiving unit 2 of the stereo camera 1 is replaced with a light projecting unit. FIG. 2 is a plan view of a light section system 6 illustrating the measurement principle of the light section method. The light section system 6 includes a light projecting unit 7, which corresponds to a projector, for example. The light projecting unit 7 projects a strip-shaped slit light onto an object while scanning, and the right light receiving unit 3 receives the slit light reflected from the object. Here, if the light projection origin (center of rotation) is at the left focal position of the stereo camera 1 and the projection angle from the left optical axis of the stereo camera 1 is θ, then the pixel position x on the virtual left light receiving surface 4 of the light projecting unit 7 is l is calculated from the following formula:

[0022]

number

[0023] In addition, if light-projection unit 7 rotates and projects a strip of slit light from the projection origin at a constant angular velocity ω around the Y axis perpendicular to the XZ plane, and the slit light passes through the left optical axis at time t0 and is projected onto point P on the target object at a projection angle θ at time t, the projection angle θ can be calculated from the following equation.

[0024]

number

[0025] Therefore, the reflected light from the slit light at point P is r If the light is received at this point, then the distance Z to the target point P can be calculated by substituting equations 2 and 3 into equation 1 as shown below.

[0026]

number

[0027] Since the base length B, the focal length f, the angular velocity ω, and the time t0 are constants determined by the design of the light-section system 6, the position x of the pixel on the right light-receiving surface 5 that projects the image of the slit light is rIt can be seen that by determining the distance Z to point P on the object and the time t at which the image of the slit light was detected, it is possible to find the distance Z to point P on the object.

[0028] The three-dimensional measuring device of this embodiment utilizes the configuration and measurement principle of the light-section method. However, it should be noted that the above configuration and measurement principle are merely an example, and that the design can be appropriately changed according to the design of the system configuration, layout, and the like. For example, the light projecting unit 7 and the right light receiving unit 3 may be laid out without being made uniformly parallel, and instead of replacing the left light receiving unit 2 with the light projecting unit 7, the light projecting unit 7 may be prepared in addition to the left light receiving unit 2 and the right light receiving unit 3, and a system configuration combining the stereo method and the light-section method may be adopted. Furthermore, the light projecting unit 7 may be used to project a beam-shaped spot light or a block check pattern light onto the target object, instead of a strip-shaped slit light. It should be noted that the calculation method of the three-dimensional information also changes according to such design changes.

[0029] The configuration of the three-dimensional measuring device of this embodiment will be described below. Fig. 3 is a block diagram of the three-dimensional measuring device 8. Although not shown, the three-dimensional measuring device 8 includes a computer device including, for example, a processor, a memory, an input / output unit, and the like. The processor includes, for example, a central processing unit (CPU), the memory includes, for example, a random access memory (RAM) and a read only memory (ROM), and the input / output unit inputs or outputs various data used or generated by the processor. The memory stores, for example, programs executed by the processor and various data used or generated by the processor.

[0030] The three-dimensional measuring device 8 includes a light projecting unit 7, a light receiving unit 3, and a three-dimensional information calculation unit 9. The light projecting unit 7 corresponds to, for example, a projector, the light receiving unit 3 corresponds to, for example, a camera, and the three-dimensional information calculation unit 9 corresponds to, for example, a processor. It is assumed that the light projecting unit 7 and the light receiving unit 3 are synchronized in time. The light projecting unit 7 projects a reference light onto the object W while scanning, the light receiving unit 3 receives the reference light reflected by the object W, and the three-dimensional information calculation unit 9 calculates the three-dimensional information of the object W by triangulation based on the information output from the light receiving unit 3. It should be noted that various types of light, such as slit light, spot light, and pattern light, can be used as the reference light.

[0031] The light receiving unit 3 is equipped with an event-based sensor. The sensor of the light receiving unit 3 is, for example, an image sensor in which a plurality of pixels are arranged two-dimensionally, but may also be a line sensor in which a plurality of pixels are arranged one-dimensionally. The light receiving unit 3 independently and asynchronously monitors each pixel from moment to moment, and when it detects a luminance change equal to or greater than a predetermined threshold, it outputs, for each pixel, the position of the pixel where the luminance change occurred, the time when the luminance change occurred, and the polarity representing the direction of the luminance change (whether it became brighter (positive polarity) or darker (negative polarity)) as an event. If the luminance of the background of the object W does not change and there is no noise in the light receiving unit 3, it is highly likely that the reference light passed over a certain pixel when the pixel output an event. Therefore, the three-dimensional information calculation unit 9 calculates the three-dimensional information of the object W based on the event output from the light receiving unit 3.

[0032] FIG. 4 is a graph of luminance change showing an example of an event group generated by the passage of a reference light. The reference light has a certain width and has a luminance distribution that is maximum near the center. When a pixel of a sensor is focused on, when a reference light having such a luminance distribution passes through the pixel, the luminance increases as the center of the reference light approaches the pixel, and decreases when the reference light has passed through the pixel. When a pixel of an event-based sensor detects such a luminance change, it is expected that a positive polarity event occurs once or multiple times as the luminance increases, and a negative polarity event occurs once or multiple times as the luminance decreases. The time when the reference light passes through the pixel is when the pixel has the maximum luminance, but an event does not have the luminance itself, and generally occurs when the luminance change is equal to or exceeds a predetermined threshold value based on the luminance at the time when the previous event occurred, so the timing of the event occurrence is not necessarily the time of maximum luminance. In practical use for 3D measurement, it is not necessary to have the time of maximum brightness, as long as the same timing is captured with respect to the brightness distribution of the reference light, but depending on the brightness of the background, it is not known at what point in the brightness distribution of the reference light the event will occur, and the time interval at which the events occur is not constant. Therefore, since an event-based sensor cannot accurately detect the time when the reference light passes through that pixel, sufficient distance measurement accuracy cannot be obtained even if the event-based sensor is directly applied to the light-section method.

[0033] 3 again, the three-dimensional information calculation unit 9 includes an event selection unit 10, a passage time calculation unit 11, and a distance calculation unit 12. It should be noted that the "unit" included in the three-dimensional information calculation unit 9 is a functional module realized by a program executed by a processor, for example. The event selection unit 10 selects an event group including a specified number of events in which the events change from positive to negative polarity, the passage time calculation unit 11 calculates the passage time at which the reference light passed through a pixel based on the time of the event included in the event group and the threshold value of the luminance change at which the event occurs, and the distance calculation unit 12 calculates the distance to the object W by triangulation based on the passage time.

[0034] The three-dimensional measuring device 8 may further include a designated number acquisition unit 13. The designated number acquisition unit 13 corresponds to, for example, a user interface (UI), a memory, or a communication unit capable of communicating with an external device. The designated number acquisition unit 13 acquires the number of events designated by the user (designated number). The designated number acquisition unit 13 sends the designated number to the event selection unit 10.

[0035] The event selection unit 10 acquires the designated number from the designated number acquisition unit 13. When the designated number is, for example, two, the event selection unit 10 selects an event group including at least two events of positive polarity and negative polarity. However, in order to more accurately calculate the passage time of the reference light, it is preferable that the event selection unit 10 selects an event group including at least three events. When the designated number is, for example, three, the event selection unit 10 selects an event group including two positive polarity events E as shown in FIG. P1 , E P2 and one negative polarity event E N1 Alternatively, an event group including one positive event and two negative events may be selected. When the specified number is three, the amount of calculation can be reduced compared to when more than three events are selected.

[0036] In addition, if the intensity of the reference light becomes too weak depending on the reflectance or distance of the object W, the width of the reference light becomes too narrow compared to the pixel size, or the scanning speed of the reference light becomes too fast, there is a possibility that the sensor cannot detect the luminance change and an event does not occur even though the reference light actually passes through the pixel. In such a case, the event selection unit 10 cannot select the specified number of events and the passage of the reference light is overlooked. Therefore, in order to obtain the specified number of events, the three-dimensional measurement device 8 is assumed to have the intensity of the reference light, the width of the reference light, the scanning speed of the reference light, the threshold value of the luminance change, etc. adjusted in advance depending on the sensitivity of the sensor, the pixel size, the event detection rate, etc.

[0037] For example, the passage time calculation unit 11 calculates an event E P1 , E P2 , and EN1 Time t1, t2, and t3 of the event E P1 , E P2 , and E N1 The threshold value of the luminance change at which an event occurs and the passing time at which the reference light passed through the pixel are calculated based on K1 (K1>1, for example, K1=1.2) and K2 (K2<1, for example, K2=0.8) are calculated. The threshold value of the luminance change at which an event occurs can be set to, for example, 20% of the luminance at the time when the previous event occurred. In this case, it is preferable to set the threshold value of the positive polarity luminance change as K1 (K1>1, for example, K1=1.2) and the threshold value of the negative polarity luminance change as K2 (K2<1, for example, K2=0.8).

[0038] 5 is a graph of luminance change showing an example of calculation of the passing time of the reference light. The passing time calculation unit 11 calculates the time when the reference light passes through a pixel, that is, the time t of the peak of the luminance distribution of the reference light, by using, for example, interpolation, weighted averaging, etc. When performing the interpolation or weighted averaging, for example, the luminance of the first event in the event group is set to 100, and then, when a positive polarity event occurs, the luminance of the previous event is multiplied by a threshold value K1, and when a negative polarity event occurs, the luminance of the previous event is multiplied by a threshold value K2. For example, in the example shown in FIG. 5, the luminance of the first event E P1 The brightness of the second event E P2 The brightness of the third event E is set to 100×1.2=120. N1 The luminance of is 120 x 0.8 = 96. Using this luminance information, it is advisable to perform interpolation or weighted averaging.

[0039] For example, polynomial interpolation (such as Lagrange interpolation), spline interpolation, linear interpolation, etc. can be used as the interpolation. In the example shown in FIG. 5, since the number of selected events is three, it is advisable to use linear interpolation, such as equiangular straight line fitting, which is said to have a relatively small amount of calculation and good accuracy. In the example shown in FIG. 5, the straight line L1 is a straight line that represents two positive polarity events E P1 , E P2 The line L2 has a slope that is the opposite of the slope of the line L1 and has one negative polarity event E N1The time of intersection of the lines L1 and L2 is the time t of the apex of the luminance distribution of the reference light, that is, the passing time of the reference light. Although not shown, polynomial interpolation such as parabolic fitting may be used.

[0040] On the other hand, when using a weighted average, the time t of the peak of the luminance distribution of the reference light, that is, the passing time of the reference light, may be calculated using the luminance of the event as a weight. In the example shown in FIG. 5, for example, the three selected events E P1 , E P2 , and E N1 The weights of these are preferably 100, 120, and 96, respectively, to perform the weighted average. The calculation formula for the passage time t of the reference light based on the weighted average is, for example, as follows:

[0041]

number

[0042] As described above, by calculating the passage time t of the reference light using interpolation, weighted averaging, etc., it is possible to calculate the passage time t of the reference light more accurately even in an event-based sensor in which the timing of events is not constant. In addition, by selecting a specified number N of events, the effect of errors in the times of individual events can be mitigated to 1 / √N.

[0043] 3 again, distance calculation unit 12 substitutes the passing time t of the reference light into, for example, Equation 4 to calculate distance Z to point P on object W captured in the pixel. Distance Z is the depth from the base line connecting the light projection origin of light projector 7 and the focal point of light receiver 3 as shown in FIG. 2, but may be converted into three-dimensional coordinates in the XYZ coordinate system.

[0044] The three-dimensional information calculated by the three-dimensional information calculation unit 9 is output to the outside of the three-dimensional measuring device 8. The three-dimensional information is used by an external device 20, such as a robot control device or a vehicle control device. Based on the three-dimensional information calculated at high speed and with high accuracy using an event-based sensor, the external device 20 can perform, for example, position control, speed control, acceleration control, and the like.

[0045] According to the above embodiment, the passage time t of the reference light can be calculated more accurately. On the other hand, by using an event-based sensor, three-dimensional measurement can be performed faster than with a frame-based sensor. In addition, a high-speed and highly accurate optical scanning three-dimensional measurement technique can be provided by using an event-based sensor. However, the selected event may not have been generated by the passage of the reference light in the first place, but may have been generated by other factors such as a change in external light, a movement of the object, a reference light that has been multiply reflected, or noise in the light receiving unit.

[0046] Therefore, it is preferable that the three-dimensional measuring device 8 further includes a designated time acquisition unit 14. The designated time acquisition unit 14 corresponds to, for example, a UI, a memory, or a communication unit capable of communicating with an external device. The designated time acquisition unit 14 acquires the passage time (designated time) of the reference light designated by the user. The designated time acquisition unit 14 sends the designated time to the event selection unit 10.

[0047] 6 is a graph of luminance change showing an example of the time taken for the reference light to pass through. The time taken for the reference light to pass through a pixel depends on the scanning speed of the reference light, the width of the reference light, the viewing angle of the light receiving section, the distance to the object reflected in the pixel, and the like, which are determined according to the specifications of the three-dimensional measuring device 8. Therefore, for example, it is advisable to scan the object with the reference light and measure the time taken for the reference light to pass through each pixel, and specify a designated time that is unlikely to take any longer based on the longest time taken for the reference light to pass through the pixel. In other words, it is advisable to specify a designated time that is longer than the longest time taken for the reference light to pass through a pixel.

[0048] Referring again to Fig. 3, the event selection unit 10 acquires the designated time from the designated time acquisition unit 14. The event selection unit 10 selects an event group including a designated number of events in which the event changes from positive to negative polarity within the acquired designated time. Fig. 7 is a graph of brightness change showing an example of an event that does not exist within the designated time. In the example shown in Fig. 7, an event E changes from positive to negative polarity and the designated number is three. P1 , E P2 , EN1 A series of events including:

[0049] However, event E P1 does not exist within the specified time, and other events E P2 , E N1 In this case, event E P1 is not caused by the passage of the reference light, but is likely to be a noise event caused by other factors such as changes in external light, movement of the target object, multiple reflections of the reference light, or noise in the light receiving section. P1 Therefore, according to the designated time, the event selector 10 does not select any noise events, and therefore functions as a filter for removing noise events.

[0050] In addition, the increase in luminance due to changes in external light, movement of the object, multiple reflections of the reference light, noise of the light receiving unit 3, etc. is generally smaller than the increase in luminance due to the reference light directly reflected by the object. Therefore, when there are multiple event groups to be selected, the event selection unit 10 may calculate a rank corresponding to the increase in luminance and select the event group with the largest rank. Since the rank represents the height of the peak of the luminance distribution of the reference light, it is preferable to calculate the rank at the timing of reversal from positive to negative polarity. In addition, when the threshold values ​​of the luminance change of the positive polarity and the negative polarity are the same, it is sufficient to simply increment (add points) the number of events of positive polarity and decrement (subtract points) the number of events of negative polarity. For example, if events occur in the order of positive → positive → positive → negative → negative → negative, and then events occur in the order of positive → negative due to a noise event, the rank of the first peak is 3 and the rank of the second peak is 3-3+1=1, so the event selection unit 10 selects the first event group with a rank of 3. Alternatively, instead of decrementing (subtracting points) by the number of negative events, the initial value may be reset to 0 when a negative event occurs.

[0051] Furthermore, if the threshold value of the positive luminance change is K1 (K1>1) and the threshold value of the negative luminance change is K2 (K2<1), and the threshold value of the positive luminance change is different from the threshold value of the negative luminance change (K1≠K2), the event selection unit 10 may multiply K1 by the number of positive events (addition of points) and multiply K2 by the number of negative events (subtraction of points). For example, if K1 is 1.2, K2=0.6, and events occur in the order of positive→positive→positive→negative→negative→negative, and then events occur in the order of positive→negative due to a noise event, the rank of the first apex is 1.2×1.2×1.2=1.728, and the rank of the second apex is 1.2×1.2×1.2×0.6×0.6×0.6×1.2≒0.447, so the event selection unit 10 selects the first event group with a rank of 1.728. Alternatively, instead of multiplying (subtracting points from) K2 (K2<1) by the number of negative events, the initial value 1 may be reset when a negative event occurs.

[0052] 8 is a flowchart showing an example of the operation of the three-dimensional measuring device 8. It should be noted that this flowchart is executed by, for example, a processor of the three-dimensional measuring device 8. In step S10, the number of events (specified number) specified by the user is acquired. In step S11, the passing time of the reference light (specified time) specified by the user is acquired. In step S12, light projection scanning is started.

[0053] In step S13, an event group including a specified number of events in which the event changes from positive to negative polarity is selected. At this time, by selecting an event group that exists within a specified time, noise events are not selected. In addition, when there are multiple event groups to be selected, a rank corresponding to the amount of increase in luminance is calculated, and the noise events are not selected by selecting the event group with the largest rank.

[0054] In step S14, the time when the reference light passes through the pixel is calculated based on the time of the event included in the selected event group and the threshold value of the luminance change. At this time, it is preferable to calculate the time t of the peak of the luminance distribution of the reference light by using interpolation, weighted averaging, or the like.

[0055] In step S15, the passing time of the reference light is substituted into, for example, Equation 4, and the distance to the object is calculated by triangulation. At this time, the distance may be converted into three-dimensional coordinates instead of depth. In step S16, the calculated three-dimensional information of the object is output to the outside.

[0056] Note that steps S10 to S11 are pre-processing for three-dimensional measurement and may be performed when installing or calibrating three-dimensional measuring device 8, and steps S12 to S16 may be performed when measuring with three-dimensional measuring device 8. Therefore, these steps may be implemented as separate programs and may not be processed sequentially. Also, note that if three-dimensional measuring device 8 and external device 20 shown in FIG. 2 are implemented in the same device, step S16 is unnecessary.

[0057] It should be noted that the above-mentioned configuration and operation of the three-dimensional measuring device 8 are one example, and other configurations may be adopted. For example, the three-dimensional measuring device 8 may have an integrated circuit that does not execute a program (e.g., an FPGA (field-programmable gate array), an ASIC (application specific integrated circuit), etc.) rather than a processor that executes a program. In this case, the "~ unit" of the three-dimensional information calculation unit 9 shown in FIG. 2 is implemented as a section that constitutes the integrated circuit.

[0058] According to the above embodiment, even if an event-based sensor in which the timing of occurrence of an event is not constant is used, the passing time when the reference light passes through a pixel can be calculated more accurately. In addition, by selecting a specified number N of events, the influence of errors in the time of each event can be mitigated to 1 / √N. On the other hand, by using an event-based sensor, three-dimensional measurement can be performed faster than with a frame-based sensor. In addition, a high-speed and high-precision optical scanning three-dimensional measurement technology can be provided by using an event-based sensor.

[0059] In addition, by using an event-based sensor, a dense distance image can be generated because the distance is calculated for each pixel of the light receiving unit 3. In addition, because event processing is independent for each pixel, parallelization is possible, and there are secondary effects such as no influence from individual differences in the pixels of the light receiving unit.

[0060] It should be noted that the program executed by the above-mentioned processor may be provided by being recorded on a computer-readable non-transitory recording medium, such as a CD-ROM, or may be provided by being distributed from a server device on a WAN (wide area network) or LAN (local area network) via wired or wireless connections.

[0061] Although various embodiments have been described herein, it should be appreciated that the present invention is not limited to the above-described embodiments, but may be modified in various ways within the scope of the appended claims. [Explanation of symbols]

[0062] 1 Stereo camera 2 Left receiver 3 Right light receiving section (light receiving section) 4 Left photosensitive area 5 Right light receiving surface (light receiving surface) 6 Light cutting system 7 Light projector 8. Three-dimensional measuring device 9 Three-dimensional information calculation section 10 Event Selection Section 11 Passage time calculation section 12 Distance calculation unit 13 Specified number acquisition section 14 Specified time acquisition section 20 External device B Baseline length D parallax E P1 , E P2 Positive Event E N1 Negative Event f focal length K1, K2 thresholds L1, L2 straight line P Point on the object t1, t2, t3 Event times W Object Z Distance to point P of the object θ Projection angle

Claims

1. a light projection unit that projects a reference light onto an object while scanning the object; a light receiving unit that receives the reference light reflected by the object, monitors a luminance change of the reference light for each pixel, and when the luminance change is equal to or greater than a predetermined threshold, outputs a position of the pixel, a time when the luminance change occurred, and a polarity representing a direction of the luminance change as an event; a three-dimensional information calculation unit that selects an event group including a specified number of events in which the event changes from positive polarity to negative polarity, calculates a passing time at which the reference light passed through the pixel based on the time of the event included in the event group and the threshold value of the luminance change at which the event occurs, and calculates three-dimensional information of the object by triangulation based on the passing time; A three-dimensional measuring device comprising:

2. The three-dimensional measuring apparatus according to claim 1 , wherein the three-dimensional information calculation unit selects the event group including at least three of the events.

3. The three-dimensional measuring apparatus according to claim 1 , wherein the three-dimensional information calculation unit selects the event group including the specified number of events in which the event changes from the positive polarity to the negative polarity within a specified time.

4. 4. The three-dimensional measuring device according to claim 1, wherein, when there are a plurality of the event groups to be selected, the three-dimensional information calculation unit calculates a rank corresponding to an amount of increase in luminance, and selects the event group with the largest rank.

5. The three-dimensional measuring apparatus according to claim 4 , wherein the three-dimensional information calculation section calculates the rank by adding points according to the number of the events of the positive polarity included in the event group.

6. projecting a reference beam onto an object while scanning the object; receiving the reference light reflected by the object, monitoring a luminance change of the reference light for each pixel, and when the luminance change is equal to or greater than a predetermined threshold, outputting a position of the pixel, a time when the luminance change occurred, and a polarity representing a direction of the luminance change as an event; selecting a group of events in which the event changes from positive to negative polarity and which includes a specified number of the events; calculating a passing time at which the reference light passes through the pixel based on the time of the event included in the event group and the threshold value of the luminance change at which the event occurs; calculating a distance to the object by triangulation based on the passing time; A three-dimensional measurement program that causes a computer to execute the above.

Images