3D measurement device, 3D measurement method, 3D measurement program

Abstract

To provide a three-dimensional measurement device, three-dimensional measurement method and three-dimensional measurement program that can properly remove an impact of environment light offset, and to provide a three-dimensional measurement device, three-dimensional measurement method and three-dimensional measurement program that can reduce an impact of a change in environment light offset.SOLUTION: A three-dimensional measurement device 1 comprises: a light reception control unit 31 that acquires data when projecting light, and acquires data when not projecting the light; an approximation computation unit 32 that, as to each of the data when projecting the light and data when not projecting the light generates approximation data when projecting the light and approximation data when not projecting the light approximating a distribution of an amount of received light to a function for each first specific pixel area preliminarily divided and set in a plurality of areas including a plurality of pixels; an environment light component removal computation unit 33 that removes or alleviates a component of environment light from the approximation data when projecting the light, using a computation result by the approximation computation unit 32; and a distance computation unit 34 that generates distance distribution data on a measurement object on the basis of a computation result by the environment light component removal computation unit 33.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a three-dimensional measurement device, a three-dimensional measurement method, and a three-dimensional measurement program. [Background technology]

[0002] There is a known technology in which a detection light is projected onto an object, an image is taken, and distance information to each position on the object is obtained from the obtained image data, thereby creating distance distribution data, also known as shape information or a three-dimensional map (see Patent Document 1). The method disclosed in Patent Document 1 uses a structured light method in which a predetermined pattern of detection light is projected from a light-projecting unit and distance is detected from changes in the position and shape of the pattern on the target object. Also, the time-of-flight method has been used for some time, which detects distance by utilizing the time difference between when a detection light is projected onto an object and when it returns to the light receiving section.

[0003] Regardless of which method is used, the output of the light projecting unit that projects the detection light is very small compared to sunlight. Therefore, particularly outdoors, the detection light is projected in a state where the surrounding sunlight and various illumination lights (hereinafter, these are referred to as ambient light) are also shining on the target object, and image data is acquired in a state where the detection light overlaps with the ambient light (hereinafter, ambient light offset). Therefore, there are cases where the detection light is buried in the stronger ambient light, and the detection light cannot be properly extracted. In addition, in order to eliminate the influence of ambient light, conventionally, image data is acquired both in a state where the detection light is projected and in a state where it is not projected, and these are compared to extract the detection light from which the influence of the ambient light offset is eliminated.

[0004] However, the image information obtained by the image sensor (imaging element) used as the light receiving section only has a resolution of 256 gradations if the A / D converter is configured with 8-bit gradations, for example, and contains a fair amount of quantization noise. If this were to be made 16-bit, this resolution would be sufficient for precise measurement indoors where there is almost no ambient light offset, but under sunlight, the numerical range would be lost by several tens of percent. In addition, the image information is generated while still containing various noises such as jitter noise, which is a measurement noise, and these also become an obstacle to accurate measurement.

[0005] Also, for example, the S / N ratio can be improved by increasing the light intensity of the detection light to match the intensity of the ambient light offset. However, if the output of the detection light is made too high, it is undesirable since the detection light may have a negative effect on the retina of the human eye. The light intensity standard for the detection light that does not damage the human eye must be limited to an output that does not damage the retina when the light output enters a pupil with a diameter of 7 mm at a distance of 10 cm from the light source. Therefore, when people are also targeted, there is a limit to how much the output of the detection light can be increased.

[0006] Furthermore, there are other issues under sunlight. For example, when measuring near a plant, the leaves may sway due to the wind, and the ambient light offset may change drastically due to fluctuations in reflected light and shadows cast by the leaves. In order to eliminate the influence of the ambient light offset by acquiring image data in both the state where the detection light is projected and the state where it is not projected, it is necessary that the ambient light offset at the time of shooting is the same. Therefore, there is no problem if multiple images can be acquired (photographed) almost simultaneously, but if the frame rate of the image sensor is about 1 / 30 seconds / frame, for example, there is a risk that it will not be possible to respond to cases where the ambient light offset changes drastically. In addition, assuming that the 3D measuring device is mounted on, for example, a car and used while driving, the ambient light offset changes significantly due to the influence of the reflected light intensity and shadows of roadside trees, surrounding buildings, and objects, so accurate detection could not be expected with conventional methods. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] JP 2010-504522 A Summary of the Invention [Problem to be solved by the invention]

[0008] An object of the present invention is to provide a three-dimensional measurement device, a three-dimensional measurement method, and a three-dimensional measurement program that can appropriately remove the influence of ambient light offset. Another object of the present invention is to provide a three-dimensional measurement device, a three-dimensional measurement method, and a three-dimensional measurement program that can reduce the influence of changes in the ambient light offset. [Means for solving the problem]

[0009] The present invention solves the above problems by the following solving means. Note that, for ease of understanding, the following description will be given with reference numerals corresponding to the embodiments of the present invention, but the present invention is not limited thereto.

[0010] The first invention includes a light projection unit (10, 10B) that projects detection light, an image sensor unit (21, 21B) that receives the detection light, a light reception control unit (31, 31B) that receives light with the image sensor unit (21, 21B) in a state where the detection light is projected and acquires light projection time data, and receives light with the image sensor unit (21, 21B) in a state where the detection light is not projected and acquires non-light projection time data, and a light reception control unit (31, 31B) that divides the light projection time data and the non-light projection time data into a plurality of regions including a plurality of pixels in advance. The three-dimensional measuring device (1, 1B) includes an approximation calculation unit (32, 32B) that approximates a distribution of the amount of received light to a function to generate approximate light-projected data and approximate non-light-projected data for each first specific pixel region set in a divided area, an ambient light component removal calculation unit (33, 33B) that removes or reduces ambient light components from the approximate light-projected data using the calculation results by the approximation calculation unit (32, 32B), and a distance calculation unit (34, 34B) that generates distance distribution data of a measurement object based on the calculation results by the ambient light component removal calculation unit (33, 33B).

[0011] A second invention is a three-dimensional measuring device (1, 1B) according to the first invention, characterized in that the ambient light component removal calculation unit (33, 33B) approximates a difference between the approximate light projection data and the approximate non-light projection data to a function for each second specific pixel area that is pre-divided into a plurality of areas including a plurality of pixels.

[0012] A third invention is a 3D measuring device (1, 1B) according to the first or second invention, characterized in that the light projecting unit (10, 10B) projects detection light of a predetermined projection pattern, and the distance calculation unit (34, 34B) generates distance distribution data by a structured light method.

[0013] A fourth invention is a three-dimensional measuring device (1, 1B) according to the first or second invention, characterized in that the light receiving control unit (31, 31B) acquires, as the light projection data, data on both a state in which the light projecting unit (10, 10B) continuously projects detection light and a state in which the light projecting unit (10, 10B) projects detection light in a pulsed manner, and the distance calculation unit (34, 34B) generates distance distribution data by a time-of-flight method.

[0014] The fifth invention includes a step of a light-projecting unit (10, 10B) projecting detection light, a step of an image sensor unit (21, 21B) receiving the detection light, a step of a light-receiving control unit (31, 31B) receiving light with the image sensor unit (21, 21B) in a state where the detection light is projected and acquiring light-projection time data, and receiving light with the image sensor unit (21, 21B) in a state where the detection light is not projected and acquiring non-light-projection time data, and an approximation calculation unit (32, 32B) performing multiple calculations for the light-projection time data and the non-light-projection time data, the step of approximating a distribution of the amount of received light to a function to generate approximate light-projected data and approximate non-light-projected data for each first specific pixel region that is pre-divided into a plurality of regions including a certain number of pixels; the step of an ambient light component removal calculation unit (33, 33B) removing or reducing ambient light components from the approximate light-projected data using a calculation result by the approximation calculation unit (32, 32B); and the step of a distance calculation unit (34, 34B) generating distance distribution data of the object to be measured based on the calculation result by the ambient light component removal calculation unit (33, 33B).

[0015] A sixth invention includes a step of a light-projecting unit (10, 10B) projecting detection light, a step of an image sensor unit (21, 21B) receiving the detection light, a step of a light-receiving control unit (31, 31B) receiving light with the image sensor unit (21, 21B) in a state where the detection light is projected and acquiring light-projection time data, and receiving light with the image sensor unit (21, 21B) in a state where the detection light is not projected and acquiring non-light-projection time data, and an approximation calculation unit (32, 32B) calculating the light-projection time data and the non-light-projection time data. a step of approximating a distribution of the amount of received light to a function to generate approximate light-projection data and approximate non-light-projection data for each first specific pixel region that is preliminarily divided into a plurality of regions including a plurality of pixels, an ambient light component removal calculation unit (33, 33B) removing or reducing an ambient light component from the approximate light-projection data using a calculation result by the ambient light component removal calculation unit (32, 32B), and a step of generating distance distribution data of the measurement object based on the calculation result by the ambient light component removal calculation unit (33, 33B). This is a 3D measurement program for executing the above. Effect of the Invention

[0016] According to the present invention, it is possible to provide a 3D measurement device, a 3D measurement method, and a 3D measurement program that can appropriately remove the influence of an environmental light offset. Also, according to the present invention, it is possible to provide a 3D measurement device, a 3D measurement method, and a 3D measurement program that can reduce the influence of a change in the environmental light offset. [Brief description of the drawings]

[0017] [Figure 1] 1 is a diagram showing the configuration of a first embodiment of a three-dimensional measuring apparatus 1 according to the present invention. [Diagram 2] FIG. 2 is a diagram showing a more specific configuration of each part of the control unit 30 along with the flow of operation. [Diagram 3] 3 is a diagram for explaining an outline of distance calculation performed by distance calculation unit 34. FIG. [Figure 4] FIG. 13 is a diagram illustrating an example of a dot pattern. [Diagram 5] 11A and 11B are diagrams illustrating a method for detecting an area that matches a dot pattern in a specific rectangular area. [Figure 6] FIG. 13 is a diagram for explaining the definition of a rectangular area. [Figure 7] FIG. 13 is a diagram illustrating a luminance distribution model function. [Figure 8] 13 is a diagram showing the effect of the ambient light component removal calculation unit 33 removing approximate non-light-projection data (ambient light components) from approximate light-projection data. FIG. [Figure 9] FIG. 13 is a diagram for explaining an overview of a three-dimensional measuring apparatus 1B according to a second embodiment. [Figure 10] FIG. 1 shows a possible configuration for Time of Flight. [Figure 11] 13 is a diagram for explaining a method of acquiring data performed by a light reception control unit 31B. FIG. [Figure 12] 13A to 13C are diagrams illustrating data obtained by processing performed by an environmental light component removal calculation unit 33B. [Figure 13] FIG. 11 is a diagram showing a more specific configuration of each part of a control unit 30B along the flow of operation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

[0019] (First embodiment) FIG. 1 is a diagram showing the configuration of a first embodiment of a three-dimensional measuring apparatus 1 according to the present invention. Note that the figures shown below, including FIG. 1, are schematic diagrams, and the size and shape of each part are exaggerated or omitted as appropriate to facilitate understanding. In the following description, specific values, shapes, materials, etc. are given, but these can be changed as appropriate. The three-dimensional measuring device 1 of the first embodiment includes a light projecting unit 10, an image sensor unit 21, and a control unit 30. The three-dimensional measuring device 1 of the first embodiment employs a Structured Light method in which a laser light source of a single wavelength in any of the bands of ultraviolet light, visible light, near-infrared light, or far-infrared light is used, a pattern light is projected that projects light pulses in a circular or rectangular shape, the pattern light projected onto an object is optically observed, and a distance is calculated from the observed parallax of the pattern light using trigonometry.

[0020] The light projection unit 10 includes a light projection lens 10a (see FIG. 3), and as described above, includes a single-wavelength laser light source and a pattern generation unit for projecting a dot pattern, and projects a randomly arranged dot pattern (detection light). The pattern generation unit can be, for example, a diffractive optical element (DOE). The image sensor unit 21 is a camera equipped with an imaging element that receives the dot pattern projected by the light projector unit 10 onto the detection target. The image sensor unit 21 is configured as a part of the camera 20 (see FIG. 3 described later). The camera 20 includes the image sensor unit 21 and a photographing lens 22. The light projecting unit 10 and the image sensor unit 21 are arranged, for example, at a predetermined distance (distance d in FIG. 3, which will be described later) in the horizontal direction.

[0021] The control unit 30 includes a light receiving control unit 31, an approximation calculation unit 32, an environmental light component removal calculation unit 33, and a distance calculation unit 34. The control unit 30 may be configured as a device having a dedicated circuit, or may be realized by causing a computer having a CPU, a storage unit, etc. to execute a three-dimensional measurement program.

[0022] The light-receiving control unit 31 controls the light-projecting unit 10 and the image sensor unit 21 to receive light with the image sensor unit 21 in a state where detection light is projected, and acquires data when light is projected. The light-receiving control unit 31 also controls the light-projecting unit 10 and the image sensor unit 21 to receive light with the image sensor unit 21 in a state where detection light is not projected, and acquires data when no light is projected. More specifically, the light-receiving control unit 31 repeats a state where the light-projecting unit 10 projects detection light and a state where it does not project light for each frame of shooting. This is because the difference between the frames is taken to make it possible to remove environmental light components.

[0023] The approximation calculation unit 32 generates approximate light-projection data and approximate non-light-projection data by approximating the distribution of the amount of received light to a function for each of the first specific pixel regions, which are pre-divided into a plurality of regions including a plurality of pixels, for each of the light-projection data and non-light-projection data.

[0024] The ambient light component removal calculation section 33 uses the calculation result by the approximation calculation section 32 to perform a calculation to remove or reduce the ambient light component from the approximate light projection data. Furthermore, the environmental light component removal calculation unit 33 approximates the difference between the approximate light-projecting data and the approximate non-light-projecting data to a function for each second specific pixel region that is pre-divided into a plurality of regions including a plurality of pixels.

[0025] Distance calculation unit 34 generates distance distribution data of the measurement object based on the calculation result by ambient light component removal calculation unit 33. Note that the distance distribution data is data indicating how the distances based on the positions of light projector unit 10 and image sensor unit 21 are distributed for each position (area) within the measurement object range.

[0026] FIG. 2 is a diagram showing a more specific configuration of each part of the control unit 30 along with the flow of operation. 2 indicates the ambient light component contained in the data detected by the image sensor unit 21 when a dot pattern is projected. The detection data acquired when a dot pattern is projected contains components of the dot pattern and ambient light, and since the influence of ambient light cannot be ignored, in this embodiment, the light-receiving control unit 31 acquires data detected by the image sensor unit 21 of only ambient light without projecting a dot pattern, in addition to data detected by projecting a dot pattern. The data acquired by the light-receiving control unit 31 is stored in the frame buffers.

[0027] The approximation calculation unit 32 approximates the distribution of the amount of received light in the detection data stored in the frame buffer to a function, calculates and generates approximate light-projecting data and approximate non-light-projecting data, and stores them in registers. When using the Structured Light method indoors, the ambient light offset is relatively stable and can be removed by simple subtraction. However, when used under sunlight (outdoors), the ambient light changes significantly even between frames, so there is an issue that the ambient light offset remains when observing the dot pattern, making it impossible to perform accurate distance measurements. In addition, when measuring under sunlight, the ambient light offset is much larger than the reflected light intensity of the detected light, and the resolution of the camera's A / D converter is limited. If the resolution of the camera's A / D converter is assumed to be 16 bits, this resolution is sufficient for precise measurement indoors where there is almost no ambient light offset, but under sunlight, the numerical range will be lost by several tens of percent.

[0028] In addition, in actual measurements, there are spots in brightness due to the effect of the beam directivity of the light projector 10, and the S / N ratio deteriorates in dark areas around the detected light. Furthermore, if the object has a high absorbance, the S / N ratio also deteriorates. Furthermore, the reflected light intensity from the object is maximum when the angle between the object's normal vector and the light vector is 0, and is minimum at 0 when the angle is 90 degrees. Furthermore, since light spreads over an area proportional to the square of the distance, the reflected light intensity attenuates inversely proportional to the square of the distance, and the S / N ratio deteriorates.

[0029] The S / N ratio can be improved by increasing the intensity of the detected light according to the intensity of the ambient light offset, but increasing the intensity of the detected light is expected to have adverse effects on the human body. For example, the light output entering a pupil with a diameter of 7 mm at a distance of 10 cm from the light source must be limited to an output that does not damage the retina.

[0030] Furthermore, for example, when considering the ambient light offset near a plant, the leaves sway when the wind blows, and the ambient light offset changes drastically due to fluctuations in reflected light and shadows cast by the leaves. In order to be able to remove the ambient light offset, it is desirable for the ambient light offset to be the same, but if the frame rate of the image sensor unit 21 is about 1 / 30 seconds / frame, the ambient light offset may change significantly with a difference of one frame. Also, when measuring distances while driving a car, the ambient light offset changes significantly due to the influence of the reflected light intensity and shadows of roadside trees, surrounding buildings, and objects.

[0031] Therefore, in this embodiment, the approximation calculation unit 32 suppresses noise in one frame of data by approximating the model function as a parabolic surface (O(x, y) = ax2 + bx + cy2 + dy + e) ​​using the least squares method for each rectangular pixel region (first specific pixel region) (m × m pixels, where m is an odd number) and calculating the median.

[0032] Furthermore, the ambient light component removal calculation unit 33 uses the approximate light-projecting data and approximate non-light-projecting data calculated and generated by the approximation calculation unit 32 to approximate a model function as a parabolic surface (O(x,y)=S1(x,y)-aS2(x,y)-b) in units of a rectangular pixel region (second specific pixel region) (mx×my pixels, where m is a positive integer) using the least squares method to perform an approximation that results in a minimum square error, and outputs the model function O(x,y) to output the error components of one frame's worth of data. This performs a calculation to remove or reduce the ambient light components from the approximate light-projecting data, and calculates and generates dot pattern data in which the effects of the ambient light offset have been eliminated or reduced. The first specific pixel region and the second specific pixel region may be the same or different.

[0033] FIG. 3 is a diagram for explaining an outline of the distance calculation performed by distance calculation unit 34. As shown in FIG. In the state shown in FIG. 3, the following relationship exists due to the similarity of triangles. Here, the distance between the light projecting lens 10a and the photographing lens 22 is the base length d, the distance from the light projecting lens 10a to the subject is Z, the focal length of the light receiving lens is f, and the position of the subject projected on the image sensor unit is x. Z=d f / x Z min =d f / x max In addition, the number of horizontal pixels of the image sensor is n x The horizontal field of view (FoV) of the camera is defined as θ x If so, the following relationship holds: x max =n x / 2 tan(θ x / 2)=x max / f As described above, the distance calculation unit 34 can perform distance calculations by optical trigonometry.

[0034] A pattern projector can be used as the light projecting unit that projects the random dot pattern. By arranging the pattern projector and the imaging camera at horizontally offset positions as shown in Figure 1, the random dot pattern projected onto the measurement target is always observed at a horizontally offset position according to the distance due to the triangular similarity relationship.

[0035] FIG. 4 is a diagram showing an example of a dot pattern. On the dot pattern shown in FIG. 4, there are only one area that matches the dot pattern in a specific rectangular area, not multiple areas. This type of dot pattern uses random numbers to generate random screen coordinates, and if it does not overlap with an already placed dot within a certain distance, the dot is placed as is, or if it overlaps with an already placed dot, the next random number is obtained and the dot placement is repeated. The autocorrelation characteristics of the random dot pattern obtained in this way are sharp, and it overlaps with itself only in the case of a 0 shift, and the correlation value drops sharply if it is shifted by even one pixel. The cross-correlation between the partial pattern of the dot pattern and the original pattern also has a sharp correlation, and overlaps only in the case of a 0 shift, but otherwise the cross-correlation value is low. The dot pattern projected onto the measurement target is always observed at a horizontally shifted position depending on the distance due to the triangular similarity relationship.

[0036] FIG. 5 is a diagram illustrating a method for detecting an area that matches a dot pattern in a specific rectangular area. To detect an area that matches a dot pattern in a specific rectangular area, first quantization is performed by setting the dots in the dot pattern to +1 and the non-dots to -1. Then, the pixel values ​​in the specific rectangular area are multiplied and accumulated using the following formula. Here, the correlation value representing the similarity is φ j Let us assume that. The pixels of the template image in the specific rectangular area to be extracted are quantized according to the presence or absence of a dot pattern, and the vector value Q i Let us assume that. The pixels of the image of a specific rectangular area to be compared and the correlation to be determined are quantized based on the presence or absence of a dot pattern, and a vector value P i+j where n is the number of pixels in the rectangular area.

[0037]

number

[0038] By calculating the cross-correlation between a specific rectangular area pattern and the captured horizontal dot pattern, the pixel phase position for distance measurement using the optical triangulation method shown in Figure 3 can be obtained from the peak position of the correlation value.

[0039] FIG. 6 is a diagram for explaining the definition of a rectangular area. FIG. 7 is a diagram for explaining the luminance distribution model function. In the approximation calculation section 32, for example, each of the obtained data is approximated to a mathematical formula. As shown in Figures 6 and 7, the depth value of a rectangular region of m x m pixels, where m is an odd number, is approximated by a model function parabolic surface O(x, y) = ax2 + bx + cy2 + dy + e. The actual measured value is represented as S(x, y).

[0040]

number

[0041] Similarly, for a, b, c, d, and e, the following equation holds:

number

[0042] By setting a rectangular small area as shown in Figure 6, the following equation is obtained.

number

[0043] Here, we can extract only the unknown terms a, c, and e.

[0044]

number

[0045] The unknown e is the noise-reduced value of the coordinate origin. Furthermore, the median e(x=0, y=0) can be found using the following formula for the inverse of a 3×3 matrix:

[0046]

number

[0047] Using the formula for the 3x3 inverse matrix above, we can perform approximation using the following product-sum calculation. j is a coefficient that can be calculated in advance.

[0048]

number

[0049] Next, the calculation of the environmental light component removal calculation unit 33 will be described. If detection light is projected in complete darkness, appropriate detection results can be easily obtained, but even if detection light is projected in an environment irradiated by sunlight, the sunlight spectrum has strong energy over the entire wavelength band of interest, so the measurement is performed under conditions with an ambient light offset. Therefore, the dot pattern is measured in a state where the ambient light offset is mixed in.

[0050] Therefore, as described above, in the three-dimensional measurement device 1 of this embodiment, the ambient light component removal calculation unit 33 performs a calculation to remove or reduce the ambient light component from the approximate light projection data using the calculation result by the approximation calculation unit 32. Specifically, for example, the unknowns a (magnification) and b (offset) can be found using the following equation and a 2×2 inverse matrix formula. The actual measured value when light is projected is SR(x,y), the actual measured value of the ambient light is SG(x,y), and the difference between the two, taking into account the magnification and offset at the time of measurement, is e(x,y).

[0051]

number

[0052] FIG. 8 is a diagram showing the effect of the ambient light component removal calculation unit 33 removing the approximate non-light-projection data (ambient light components) from the approximate light-projection data. Fig. 8(a) shows image data received without projecting detection light. Fig. 8(b) shows image data received with projecting detection light. It is difficult to confirm the dot pattern of the detection light just by looking at Fig. 8(b). Fig. 8(c) shows the image data after removing the approximate non-light-projection data (ambient light components) from the approximate light-projection data. In the result of Fig. 8(c), the dot pattern of the detected light is clear, and the effect of the calculation by the ambient light component removal calculation unit 33 can be confirmed.

[0053] As described above, according to the first embodiment, the data handled by the approximation calculation unit 32, the ambient light component removal calculation unit 33, and the distance calculation unit 34 are floating-point format data, which can improve the data resolution and suppress quantization noise. Furthermore, even under sunlight, the accuracy is not lost due to changes in illuminance, and stable three-dimensional measurement can be performed. Furthermore, the elimination or reduction of the ambient light offset functions effectively even when the ambient light offset fluctuates greatly.

[0054] Second embodiment The three-dimensional measuring device of the second embodiment differs from the three-dimensional measuring device of the first embodiment, which employs a Structured Light method, in that the second embodiment employs a Time of Flight method, but other points are the same as those of the first embodiment. Therefore, parts that perform the same functions as those of the first embodiment described above are marked with the letter B at the end, and duplicated explanations will be omitted as appropriate.

[0055] FIG. 9 is a diagram for explaining an overview of a three-dimensional measuring apparatus 1B according to the second embodiment. The three-dimensional measuring device 1B of the second embodiment has a single-wavelength laser light source in any of the ultraviolet, visible, near-infrared, or far-infrared bands, projects detection light that projects a light pulse in a circular or rectangular shape, observes the light that is reflected off the object to be measured and returns on a pixel-by-pixel basis using an optical system, and employs a time-of-flight method in which the distance to the object is determined by measuring the phase delay of the light pulse.

[0056] The light projecting unit 10B is similar to the light projecting unit 10 of the first embodiment except that it projects detection light that is generally uniform or has a Gaussian distribution and does not project a dot pattern as in the first embodiment. The image sensor unit 21B is similar to the image sensor unit 21 in the first embodiment. Light propagates at 300,000 km per second. If the frequency of the optical pulse is assumed to be 50 MHz, then one wavelength is 6 m. The time it takes for light to propagate a round trip distance of 3 m is 20 nanoseconds, so to measure a distance in units of 3 mm it takes 20 picoseconds, and to measure a distance in units of 1 mm it takes about 6.67 picoseconds, and to measure distances by using a counter to measure the time it takes for the optical pulse to return with a resolution of 1 mm, a clock of 150 GHz is required. However, it is difficult to realize such a circuit using ordinary consumer semiconductors.

[0057] FIG. 10 shows a possible implementation of Time of Flight. Here, the duty cycle of the light pulse generated by the light projector 10B, which is a laser light source, is set to 1:1. A switch that stores the photocurrent output by the image sensor unit 21B in a capacitor is provided for each pixel, and the mechanism of this embodiment in which all the switches are turned on simultaneously when the light pulse of the light projector 10B is off is called a global shutter. With the configuration shown in FIG. 10, the output current of the image sensor unit 21B is stored in a capacitor at the timing when the gate (switch) signal is turned on.

[0058] In the second embodiment as well, when used outdoors, measurements are performed under conditions with an ambient light offset. In addition, in actual measurements, there are spots in the brightness due to the effect of the beam directivity of the light projector 10B, and the S / N ratio deteriorates in the dark areas around the detection light. Furthermore, if the object has a high absorbance, the S / N ratio also deteriorates. Furthermore, the reflected light intensity from the object is maximum when the angle between the object's normal vector and the light vector is 0, and is minimum at 0 when the angle is 90 degrees. Furthermore, since light spreads over an area proportional to the square of the distance, the reflected light intensity attenuates inversely proportional to the square of the distance, and the S / N ratio deteriorates. Therefore, in the second embodiment as well, in order to obtain data for removing the ambient light offset, the light reception control section 31B controls the light projection section 10B and the image sensor section 21B to obtain data for a plurality of states.

[0059] FIG. 11 is a diagram for explaining a method of acquiring data performed by the light reception control unit 31B. In step 1, image data of only the ambient light, that is, the ambient light offset, is acquired with the light projector 10B in a non-light projecting (off) state. In step 2, image data including the ambient light and the detection light is acquired while the light projecting unit 10B is in the light projecting (ON) state. In step 3, while light-projecting unit 10B projects pulsed light, image data is acquired that includes the ambient light offset when light-projecting unit 10B is not projecting light (off) and the reflected light intensity (including the ambient light offset) for the phase delay.

[0060] FIG. 12 is a diagram for explaining data obtained by the process performed by the environmental light component removal calculation unit 33B. As shown in Figure 12, the reflected light intensity of the detected light can be obtained using BA. This is the result of accumulating one frame's worth of photocurrent, which includes all of the directivity, absorbance, diffuse reflection, and distance attenuation of the detected light. The reflected light intensity with phase delay can be obtained using CA. Using E / D, the effects of the directivity, absorbance, diffuse reflection, and distance attenuation of the detected light can be corrected to obtain a result proportional to the distance. To improve accuracy, it is necessary to ensure the accuracy of D, the denominator.

[0061] However, the issues of resolution and S / N ratio under sunlight as described in the Structured Light method of the first embodiment, as well as various issues of fluctuations in ambient light offset due to swaying leaves or while driving, also arise in the second embodiment which employs the Time of Flight method.

[0062] Therefore, in the second embodiment as well, the above problem is solved by using approximation calculations by the approximation calculation section 32B and the environmental light component removal calculation section 33B. FIG. 13 is a diagram showing a more specific configuration of each part of the control unit 30B along with the flow of operation.

[0063] Under the control of the light receiving control unit 31B, the frame buffer A holds image data when the detection light is off. The frame buffer B holds image data when the detection light is on. The frame buffer C holds image data accumulated at the off timing when the light projecting unit is optically pulse driven.

[0064] The approximation calculation unit 32B receives image data from each of the frame buffers A, B, and C, and suppresses noise in one frame's worth of data by approximating the model function as a parabolic surface (O(x, y)=ax2+bx+cy2+dy+e) using the least squares method for each rectangular pixel region (m×m pixels, where m is an odd number). From the image data in frame buffer A, approximate non-light projection data is obtained, from the image data in frame buffer B, approximate light projection data is obtained, and from the image data in frame buffer C, approximate pulsed light projection data is obtained.

[0065] The ambient light component removal calculation unit 33B uses the approximate light projection data, approximate non-light projection data, and approximate pulsed light projection data calculated and generated by the approximation calculation unit 32 to approximate a model function as a parabolic surface (O(x, y) = S1(x, y) -aS2(x, y) -b) in a specified rectangular pixel region (mx × my pixels, where m is a positive integer) using the least squares method to perform an approximation that results in a minimum square error, and then outputs the model function O(x, y) and outputs the error components of one frame's worth of data.

[0066] The approximation calculations in the approximation calculation section 32B and the environmental light component removal calculation section 33B are similar to those in the first embodiment.

[0067] As described above, according to the second embodiment, similarly to the first embodiment, the data handled is floating-point data, which can improve the data resolution and suppress quantization noise. Moreover, even under sunlight, the accuracy is not lost due to changes in illuminance, and stable three-dimensional measurement can be performed. Furthermore, the elimination or reduction of the ambient light offset is effective even for large fluctuations in the ambient light offset.

[0068] The embodiments may be used in appropriate combination, but detailed description thereof will be omitted. The present invention is not limited to the embodiments described above. [Explanation of symbols]

[0069] 1. 1B 3D measuring device 10, 10B Light projecting section 10a Projection lens 20 Camera 21, 21B Image sensor section 22 Photographing Lenses 30, 30B Control section 31, 31B Light receiving control section 32, 32B Approximate operation section 33, 33B Ambient light component removal calculation unit 34 Distance calculation section

Claims

1. a light projecting unit that projects detection light; an image sensor unit that receives the detection light; a light-reception control unit that receives the detection light by the image sensor unit while the detection light is being projected, thereby acquiring data during light projection, and that receives the detection light by the image sensor unit while the detection light is not being projected, thereby acquiring data during non-light projection; an approximation calculation unit that generates approximate light-projection data and approximate non-light-projection data by approximating a distribution of the amount of received light to a function for each of a first specific pixel region that is pre-divided into a plurality of regions including a plurality of pixels, for each of the light-projection data and the non-light-projection data; an environmental light component removal calculation unit that removes or reduces environmental light components from the approximate light-projection data by using the approximate light-projection data and the approximate non-light-projection data that are calculation results by the approximation calculation unit; a distance calculation unit that generates distance distribution data of a measurement object based on a calculation result by the environmental light component removal calculation unit; A three-dimensional measuring device comprising:

2. 2. The three-dimensional measuring apparatus according to claim 1, the environmental light component removal calculation unit approximates a difference between the approximate light-projecting data and the approximate non-light-projecting data to a function for each second specific pixel region that is pre-divided into a plurality of regions including a plurality of pixels; A three-dimensional measuring device characterized by the above.

3. 3. The three-dimensional measuring apparatus according to claim 1, The light projecting unit projects detection light of a predetermined projection pattern, the distance calculation unit generates distance distribution data by a structured light method; A three-dimensional measuring device characterized by the above.

4. 3. The three-dimensional measuring apparatus according to claim 1, the light-reception control unit acquires, as the light-projection data, data on the time of light projection for both a state in which the light-projecting unit continuously projects the detection light and a state in which the light-projecting unit projects the detection light in a pulsed manner; the distance calculation unit generates distance distribution data by a time-of-flight method; A three-dimensional measuring device characterized by the above.

5. a step of a light projecting unit projecting detection light; an image sensor unit receiving the detection light; a step of receiving light by the image sensor unit while a light reception control unit is emitting the detection light and acquiring data during light projection, and receiving light by the image sensor unit while not emitting the detection light and acquiring data during non-light projection; an approximation calculation unit approximating a distribution of the amount of received light to a function for each of the light projection data and the non-light projection data, the first specific pixel region being pre-divided into a plurality of regions including a plurality of pixels, to generate approximate light projection data and approximate non-light projection data; an environmental light component removal calculation unit removing or reducing an environmental light component from the approximate light-projection data using the approximate light-projection data and the approximate non-light-projection data which are the calculation results by the approximation calculation unit; A distance calculation unit generates distance distribution data of the measurement object based on a calculation result by the environmental light component removal calculation unit; A three-dimensional measurement method comprising:

6. On the computer, a step of a light projecting unit projecting detection light; an image sensor unit receiving the detection light; a step of receiving light by the image sensor unit while a light reception control unit is emitting the detection light and acquiring data during light projection, and receiving light by the image sensor unit while not emitting the detection light and acquiring data during non-light projection; an approximation calculation unit approximating a distribution of the amount of received light to a function for each of the light projection data and the non-light projection data, the first specific pixel region being pre-divided into a plurality of regions including a plurality of pixels, to generate approximate light projection data and approximate non-light projection data; an environmental light component removal calculation unit removing or reducing an environmental light component from the approximate light-projection data using the approximate light-projection data and the approximate non-light-projection data which are the calculation results by the approximation calculation unit; A distance calculation unit generates distance distribution data of the measurement object based on a calculation result by the environmental light component removal calculation unit; A 3D measurement program for executing the above.

Images