Distance measuring device having a distance correction function

Abstract

A distance measuring device (10) comprising a light-emitting section (11) that emits reference light to a target measurement space at a predetermined light emission time, and a plurality of light-receiving elements (17) that are arranged two-dimensionally and that receive incident light from the target measurement space at a predetermined image acquisition time, wherein the distance measuring device outputs a distance image to an object in the target measurement space based on light reception amounts of the light-receiving elements and a two-dimensional image corresponding to the distance image, the distance measuring device further comprising: a reference object distance calculation section (26) that calculates a distance to the reference object based on the two-dimensional image in which a reference object including a plurality of feature points having obvious three-dimensional coordinate relationships is captured, and a correction amount calculating section (27) that calculates a correction amount for correcting the distance image by comparing the calculated distance to the reference object with a distance measurement value to the reference object in the distance image.

Description

Background of the invention 1. Field of the invention

[0001] The present invention relates to a distance measuring device that measures a distance to an object based on a travel time of light, and more particularly to a distance measuring device having a distance correction function. 2. Description of related prior art

[0002] TOF (time-of-flight) cameras, which output a distance based on the time of flight of light, are known as a distance measuring device for measuring the distance to an object. Many TOF cameras use a phase-difference method in which measurement light with intensity modulated over predetermined cycles is irradiated toward a target measurement space, and a phase difference between the irradiated measurement light and the light reflected from the object in the target space is detected.

[0003] TOF cameras, which are three-dimensional sensors, cause distance measurement errors due to individual characteristic variations in their electronic elements (e.g., light-receiving elements, A / D conversion elements, etc.), due to aging of the electronic elements, etc. Regarding individual characteristic variations, distance measurement errors may increase depending on differences in the usage environment (especially temperature), aging, etc., although a camera manufacturer adjusts the variations within target error ranges through calibration performed under specific conditions at the time of shipment. These distance measurement errors differ individually.

[0004] Japanese Unexamined Patent Publication (Kokai) JP 2015-175752 A discloses a distance image generation device using a time-of-flight type distance image sensor. The distance image generation device performs distance image smoothing by determining filter parameters (size, coefficient, frame number, etc.) used to smooth objects of a known size.

[0005] Japanese Unexamined Patent Publication (Kokai) JP 2015-56057 A discloses a method for estimating the pose of a marker using a TOF camera. In this pose estimation method, the marker is detected from a camera image, the marker area is obtained from a distance image, an optimal plane is estimated from the obtained distance image, and the position and attitude of the marker are estimated from the estimated plane.

[0006] Japanese Unexamined Patent Publication (Kokai) JP 2014-70936 A discloses a defective pixel detection device for detecting defective pixels of a TOF camera. The defective pixel detection device compares a distance-corrected TOF image, which is a TOF image (bright image) corrected based on a distance image, with a captured image acquired by an imaging camera, and detects defective pixels that have measurement errors in the distance image based on the comparison result. Summary of the invention

[0007] Real reference light produces time delays and waveform blunting relative to an ideal light emission time of the reference light due to individual properties of the electronic elements of TOF cameras, in particular semiconductor lasers (LD), transistors, resistors, etc. (see Fig. 5). Such time delays and waveform blunting are not random, but rather repetitive, and since such time delays and waveform blunting can be considered as a simple delay or an average delay for ideal emission pulses, such time delays and waveform blunting correspond to a condition in which an offset Δt relative to a phase of the reflected light has been added to distance measurements, and in which a correction of the offset relative to the distance measurements can be considered necessary.Therefore, the manufacturer of the TOF camera acquires, as a parameter at the time of calibration before shipment, an offset ΔL of the distance measurement values caused by the offset Δt, which is an individual difference for each TOF camera, from the results of distance measurement of an object at a given distance, and thereafter, the TOF camera outputs the distance measurement values corrected based on the offset ΔL.

[0008] However, during actual use of the TOF camera, the time delay and waveform truncation (i.e., simple delay) corresponding to the offset Δt fluctuate due to the ambient temperature, temperature changes due to heat generation of components, time-dependent changes in individual characteristics, etc., and the distance measurement values may also change. Specifically, the distance measurement values of each pixel in the TOF camera will vary. Although many attempts have been made to place a temperature sensor inside a TOF camera to detect such fluctuations and change a correction amount according to the detected temperature, such a solution does not completely solve problems due to aspects related to the position and accuracy of the temperature sensor.

[0009] Therefore, there is a need for a distance measuring device that easily enables accurate distance correction.

[0010] One aspect of the present invention provides a distance measuring device comprising a light-emitting section that emits reference light to a target measurement space at a predetermined light emission time, and a plurality of light-receiving elements that are arranged two-dimensionally and that receive incident light from the target measurement space at a predetermined image acquisition time, wherein the distance measuring device outputs a distance image to an object in the target measurement space based on light reception amounts of the light-receiving elements and a two-dimensional image corresponding to the distance image, wherein the distance measuring device further comprises a reference object distance calculation section that calculates a distance to the reference object based on the two-dimensional image in which a reference object including a plurality of feature points having obvious three-dimensional coordinate relationships is captured,and a correction amount calculation section that calculates a correction amount for correcting the distance image by comparing the calculated distance to the reference object with a distance measurement value to the reference object in the distance image. Short description of the drawings Fig. 1 is a block diagram of a distance measuring device according to an embodiment. Fig. 2A is an explanatory diagram showing a distance correction method according to an embodiment in which a reference mark is used as a reference object. Fig. 2B is an explanatory diagram showing a distance correction method according to an embodiment in which an arbitrary object is used as a reference object. Fig. 3 is a view showing distance measurement values at the time of construction of the distance measuring device according to an embodiment, after several years or when the ambient temperature changes significantly, and after correction. Fig. 4 is an explanatory diagram showing a distance correction method according to another embodiment. Fig. 5 is a view showing a light emission time delay and a waveform blunting of reference light according to the prior art. Description of preferred embodiments

[0011] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings. In the drawings, identical or similar individual elements have been designated with the same or similar reference numerals. Furthermore, the embodiments described below do not limit the technical scope of the inventions described in the claims or the definitions of terms.

[0012] Fig. 1 is a block diagram of a distance measuring device 10 according to the present embodiment. The distance measuring device 10 is a TOF camera that measures a distance to an object based on, for example, a phase difference method. The distance measuring device 10 includes a light-emitting section 11 that emits reference light L1 emitted toward a target measurement space, a light-receiving section 12 that receives incident light L2 from the target measurement space, and a distance image generation section 13 that generates a distance image to the object in the target measurement space.

[0013] The light-emitting section 11 is constituted by a light source such as a light-emitting diode (LED) or LD that emits, for example, near-infrared (NIR) light, and emits reference light L1 intensity-modulated at a predetermined cycle based on a light emission timing signal from an emission / image acquisition timing control section 14. The reference light L1 is scattered by a scattering plate 15 and emitted toward the target measurement space.

[0014] The light-receiving section 12 is formed by an image sensor such as a CCD or a CMOS, which includes, for example, an RGB filter, an NIR filter, etc., and receives incident light L2 via an optical system 16 including a converging lens or the like. The incident light L2 includes ambient light in addition to the reference light reflected from the object. The light-receiving section 12 includes four light-receiving elements in which each pixel receives red light, blue light, green light, and NIR light. Alternatively, the light-receiving section 12 may include a single light-receiving element in which each pixel receives only NIR light.

[0015] The light-receiving element 17 is formed, for example, by a photodiode or a capacitor. A light-receiving element 17 that receives NIR light receives light at multiple image acquisition times that are delayed by a predetermined phase relative to the light emission time of the reference light L1 based on an image acquisition timing signal from the emission / image acquisition timing control unit 14. For example, light reception amounts Q1 to Q4 are detected at image acquisition times Et1 to Et4 that are out of phase by 0°, 90°, 180°, and 270°, respectively, relative to the ideal light emission time of the reference light. A light-receiving element 17 that receives red light, blue light, and green light detects the light reception amounts over a predetermined time acquisition period. As shown in Fig. 1, the detected light reception amounts are amplified by an amplifying section 18, A / D converted by an A / D converting section 19, and the A / D converted values are stored in a buffer memory 20.

[0016] The distance image generation section 13 generates a distance image 30 to the object in the target measurement space based on the A / D converted values of the light reception amounts Q1 to Q4 of the NIR light. The distance measurement value L tof is calculated, for example, from the known formula described below. Td is a phase difference between the reference light and the reflected light, c is a speed of light, and f is a frequency. The generated distance image is stored in a buffer memory 21 and output to an application 23 via an output control section 22. Td=arctan(Q2−Q4Q1−Q3) Ltof=c4πfTd

[0017] A two-dimensional image generation section 24 generates a two-dimensional image 31 based on the A / D converted values of the light reception amounts of the RGB light or the NIR light. In other words, the two-dimensional image 31 may be an RGB image (color image) or it may be an NIR image (monochromatic image). The two-dimensional image 31 is stored in the buffer memory 21 and output to the application 23 via the output control section 22.

[0018] As above with reference to Fig. As described in Section 5, the actual light emission time of the reference light includes an offset Δt relative to the ideal light emission time of the reference light; an offset ΔL of the distance measurement values caused by the offset Δt is recorded during calibration at the time of delivery. The distance measurement values corrected based on the offset ΔL are output within the TOF camera. Therefore, the distance measurement value L tof can be calculated using the formula described below, in which the offset correction ΔL has been added. Ltof=c4πfTd+ΔL

[0019] Since the offset Δt changes in accordance with changes in temperature, aging, etc., there is a possibility that the final distance image may also contain distance measurement errors. To correct such variations in the offset Δt (and to correct the distance image), the distance measuring device 10 of the present embodiment has a distance correction function. The distance measuring device 10 uses the distance to a reference object 25 (see Fig. 1) geometrically calculated from the two-dimensional image 31 to calculate a correction amount Li for correcting the distance image 30. To geometrically calculate the distance to the reference object 25, it is necessary to include a plurality of feature points 32 that have obvious three-dimensional coordinate relationships on the reference object 25. Note that "obvious three-dimensional coordinate relationships" means that the relative positional relationships thereof can be known. In other words, it is not always necessary for the three-dimensional coordinate relationships to be known (specifically, it is not necessary for them to be stored in advance in a memory or the like of the distance measuring device 10).

[0020] Fig. 2A is an explanatory diagram showing a distance correction method in which a reference mark 25a is used as the reference object 25. As in Fig. As shown in Figure 4, the reference mark 25a is a quadrangular white plate-like member having a known size on which a perfect circle, a quadrilateral, and a rhombus are arranged, each having known sizes and positional relationships. It includes a large number of feature points having known three-dimensional coordinate relationships. For example, the feature points may be central parts of the perfect circle, the quadrilateral, and the rhombus (indicated as 32a, 32b, and 32c, respectively). Further, the center of the perfect circle of the reference mark 25a is a representative feature point 32b. The distance measuring device 10 detects the reference mark 25a from a two-dimensional image in which the reference mark 25a is captured using known image processing, and determines the position coordinates of the various feature points of the reference mark 25a in the image at the sub-pixel level.

[0021] The distance measuring device 10 geometrically calculates the distance L ref to the representative feature point 32b based on the combination of the position coordinates of the plurality of feature points (generally four or more) in the image. To determine the distance L ref To calculate more accurately, several values of L ref based on different combinations of the plurality of feature points, and the average thereof can be used. The distance measuring device 10 calculates the correction amount Li for correcting the distance image by comparing the distance L ref with the representative feature point 32b, which is determined on the basis of the two-dimensional image with the distance measurement value L tofof the representative feature point 32b in the distance image. In such a distance correction method, the two-dimensional image and the distance image correspond to each other at the pixel-to-pixel level, and accordingly, it is not necessary to perform a process in which the feature points of the two images are aligned or fused as in known stereo methods. Furthermore, an accurate correction amount Li can be calculated when the position coordinates of the representative feature point 32b in the image are determined at a sub-pixel level, since the distance measurement value of the representative feature point in the distance image can also be calculated with high precision by interpolation with distance measurement values of the surrounding pixels. Furthermore, by preparing the reference mark in advance, a user can easily perform correction when correction is desired.Alternatively, the distance measuring device may continuously image the reference mark 25a and change the correction amount at any time to maintain the accuracy.

[0022] Fig. 2B is an explanatory diagram showing the distance correction method using arbitrary objects 25b, 25c as reference objects. In the present example, the corrections to the three-dimensional coordinates of ten feature points of the two rectangular parallelepipeds 25b, 25c are represented by nine vectors. For example, an operator can directly input the three-dimensional coordinates of the nine vectors to the distance measuring device 10, or they can specify the positions of the eight corners 32d, 32e, 32f, 32g, 32h, 32i, 32j, 32k of the two rectangular parallelepipeds in the two-dimensional image acquired by the distance measuring device 10, and input the lengths of the sides of the two rectangular parallelepipeds and the distances between the two rectangular parallelepipeds.As understood from the foregoing, the distance measuring device 10 preferably includes means for specifying feature values of the object (e.g., three-dimensional coordinates of vectors, positions of feature points on the object, a size of the object itself, a specific shape, pattern, color, etc. of the object) or the positional relationship of multiple objects (e.g., a distance between objects) as the three-dimensional coordinate relationship. As a result, the arbitrary objects 25b, 25c serve as the reference mark 25a.

[0023] With further reference to Fig. 1, the distance measuring device 10 of the present embodiment includes a reference object distance calculation section 26 that calculates a distance to the reference object 25, and a correction amount calculation section 27 that uses the distance to the reference object 25 to calculate the correction amount Li for correcting the distance image 30. The reference object distance calculation section 26 and the correction amount calculation section 27 may be implemented as software that causes a processor such as a central processing unit (CPU) to function. Alternatively, for example, the processing of at least some of such software may be executed as hardware such as an executable processor.

[0024] The reference object distance calculation section 26 obtains the two-dimensional image 31 in which the reference object 25 is captured, which includes a plurality of feature points 32 (including the representative feature point 32b) having known three-dimensional coordinate relationships, from the buffer memory 21 and geometrically calculates the distances to the feature points 32 based on the two-dimensional image 31.

[0025] The correction amount calculating section 27 calculates the correction amount for correcting the distance image by comparing the distance L ref to the representative feature point 32b calculated by the reference object distance calculation section 26 with the distance measurement value L tof of the representative feature point 32b in the distance image 30 stored in the buffer memory 21. For example, the correction amount Li may be a value expressed as the difference between the distance L refand the distance measurement value L tof calculated as shown in the formula below, or it may be calculated from several groups of coefficient values in a functional expression obtained from separate verification testing to perform a more advanced correction relative to the different distance measurement values of all pixels. Li=Ltof−Lref

[0026] The correction amount calculation section 27 stores the correction amount Li in a non-volatile memory 28 and obtains the correction amount Li from the non-volatile memory 28 and reuses it when the distance measuring device 10 is turned on. Alternatively, in applications where the distance measuring device 10 continuously images the reference mark 24a, the correction amount Li can be changed as needed to maintain accuracy. Further, the correction amount calculation section 27 outputs the correction amount Li to the emission / image acquisition timing control section 14 or to the distance image generation section 13.

[0027] The emission / image capture timing control section 14 controls the light emission time or the image capture time based on the correction amount Li. For example, when the correction amount Li is a distance correction value as in the formula described above, the emission / image capture timing control section 14 calculates an offset Δt' corrected based on the formula described below, and shifts the image capture time or the light emission time such that the image capture time is delayed relative to the light emission time only by the offset Δt'. The reason why the calculation is multiplied by 2 is that the reflected light travels twice the distance of the distance measurement value. Δt'=2×Lic

[0028] Alternatively, the distance image generating section 13 may correct the distance image based on the correction amount Li. For example, in addition to correcting the offset ΔL, the distance image generating section 13 corrects the distance measurement value L tof by superimposing the correction amount Li as in the formula below, when the correction amount Li from the correction amount calculation section 27 is valid. Ltof=c4πfTd+ΔL+Li

[0029] Fig. 3 is a view showing the distance measurement values at the time of installation of the distance measuring device 10, after several years, or when the ambient temperature changes, and after correction. Although the distance measurement value at the time of installation of the distance measuring device 10 is within an acceptable range, the operator corrects the distance measurement value using the distance correction function of the distance measuring device 10 after several years or when the ambient temperature changes, because the distance measurement value may deviate from the acceptable range. Alternatively, the distance measuring device 10 can continuously monitor the reference object 25 and perform distance correction when needed to maintain accuracy.According to such a distance measuring device 10, distance measurement errors due to individual characteristic deviations of the electronic elements or due to aging of the electronic elements can be easily corrected.

[0030] As a distance correction method of other embodiments of the present application, a method based on Fig.4, in which multiple distance values on a plane of the reference object are used, which is different from the method in which the distances of the representative feature points are used as described above. To measure distances based on TOF, the distance measuring device 10 can measure the distance of a plane of an object without changes in intensity, unlike known stereo methods. Therefore, the reference distance calculation section 26 and the correction amount calculation section 27 can calculate the correction amount Li for correcting the distance image by determining the position and attitude of a plane 33 from multiple feature points in a two-dimensional image in which the plane 33 is captured, and complexly comparing the distance values of each pixel in which the plane 33 is captured with the distance measurement values of each pixel in the distance image corresponding to those pixels.As a result, the distance measuring device 10 can perform more accurate corrections. Note that the position of the plane 33 represents the distance values of the multiple feature points, and the attitude of the plane 33 represents the inclination of the plane 33 relative to the optical axis of the distance measuring device 10.

[0031] For example, for a reference mark 25a having, for example, four corners 321, 32m, 32n, 32o, the reference object distance calculation section 26 detects the reference mark 25a and the plurality of feature points (for example, four corners, a central portion of the perfect circle, the quadrilateral, and the parallelogram) from the two-dimensional image and determines the position and attitude of the plane 33. Next, the reference object distance calculation section 26 calculates the distance measurement values of each of the pixels determining the plane 33 from the position and attitude of the plane 33, and outputs the positions of the determined pixels in the image and the distance measurement values of the pixels to the correction amount calculation section 27.The correction amount calculating section 27 may determine the difference between the average value of the distance values of the pixels and an average value of the distance measurement values in the distance image corresponding to the positions of the specific pixels to calculate the correction amount Li.

[0032] In a distance image based on the TOF principle, pixels with high light intensity generally have higher accuracy than pixels with low light intensity. Therefore, weighted averaging can be performed by weighting the light intensities of the pixels when averaging the distance measurement values in the distance image corresponding to the positions of the pixels in the image, as described above. As a result, a more accurate correction amount Li can be acquired. The light intensity I is calculated, for example, using the following well-known formula. I=(Q1−Q3)2+(Q2−Q4)22

[0033] According to the above embodiments, since the two-dimensional image 31 and the distance image 30 correspond to each other on a pixel-to-pixel level, the correction amount for correcting the distance image 30 can be calculated using the distances to the reference object 25 geometrically calculated from the two-dimensional image 31. Therefore, correction of distance measurement errors generated due to individual characteristic changes of the electronic elements or due to aging of the electronic elements can be easily implemented.

[0034] Although various embodiments have been described in the present specification, the present invention is not limited to the above-described embodiments. It should be noted that various modifications may be made without departing from the scope described in the following claims.

Claims

[1] A distance measuring device (10) comprising a light-emitting portion (11) that emits reference light to a target measurement space at a predetermined light emission time, and a plurality of light-receiving elements (17) that are arranged two-dimensionally and that receive incident light from the target measurement space at a predetermined image acquisition time, wherein the distance measuring device outputs a distance image to an object in the target measurement space based on light reception amounts of the light-receiving elements and a two-dimensional image corresponding to the distance image, the distance measuring device further comprising: a reference object distance calculation section (26) which calculates a distance to the reference object on the basis of the two-dimensional image in which a reference object including a plurality of feature points having obvious three-dimensional coordinate relationships is captured, and a correction amount calculation section (27) that calculates a correction amount for correcting the distance image by comparing the calculated distance to the reference object with a distance measurement value to the reference object in the distance image. [2] The distance measuring device according to claim 1, wherein the reference object is a reference mark comprising a plurality of feature points (32) having known three-dimensional coordinate relationships. [3] The distance measuring device according to claim 1, wherein the reference object is any object, and wherein the distance measuring device further comprises means for indicating feature values of the object or the positional relationship between a plurality of the objects as three-dimensional coordinate relationships. [4] The distance measuring device according to any one of claims 1 to 3, further comprising an emission / image acquisition timing control section (14) which controls the light emission time or the image acquisition time based on the correction amount. [5] A distance measuring apparatus according to any one of claims 1 to 3, further comprising a distance image generating section (13) which generates the distance image based on the correction amount.

Images