3D shape measuring device and 3D shape measuring method

Abstract

To provide a three-dimensional shape measuring device that, in a plurality of pixels in an image sensor, can obtain data of the measured distance to a measurement object with uniform accuracy to measure a three-dimensional shape.SOLUTION: A three-dimensional shape measuring device 1 comprises: a light emitting unit 12; an image pick-up device 14; an output amount detecting unit 152; an arithmetic unit 153; a measuring unit 156; and a shape information generating unit 157. The output amount detecting unit 152 detects an exposure output amount for every pixel per light emission from the light emitting unit 12. The arithmetic unit 153 acquires a reference exposure output amount for every pixel output at an exposure timing including all rays of reflected light reflected on a measurement object per light emission, and based on the number of times of integration for bringing the reference exposure output amount for every pixel closer to a predetermined target value, acquires the number of times of light emission and exposure for every pixel. The measuring unit 156 measures the distance to the measurement object for every pixel based on the acquired number of times of light emission and exposure. The shape information generating unit 157 generates three-dimensional shape information of the measurement object by using information on the measured distance.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a three-dimensional shape measurement device and a three-dimensional shape measurement method.

Background Art

[0002] Generally, the amount of light exposure to an image sensor when shooting with a digital camera device is determined by six parameters: (i) light source emission intensity, (ii) subject reflectance, (iii) lens T value, (iv) lens aperture value, (v) sensor exposure time, and (vi) signal amplification amount by a circuit.

[0003] When shooting, the photographer changes parameters (iv), (v), and (vi) according to the above-mentioned parameters (i), (ii), and (iii), and in some cases, adjusts parameter (i) to set appropriate imaging conditions. For example, the photographer sets the shooting conditions by adjusting these parameters according to the balance of signal intensities between the main subject and the secondary subject.

[0004] On the other hand, in a three-dimensional shape measurement device using the dToF (direct Time of Flight) method, the amount of exposure output power required for distance measurement of the measurement object is obtained by changing the phase of exposure in the laser emission pulse and the image sensor (image capturing element). At that time, since the laser emission time is very short, it is difficult to obtain sufficient exposure output power with one emission and exposure. Therefore, by repeating emission and exposure a plurality of times at intervals longer than the sensor exposure time, the required exposure output power can be obtained.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] For a 3D shape measuring device using the dToF method to accurately measure the distance of an object, it is necessary to obtain an appropriate exposure output at every pixel of the image sensor. However, there was a problem in that, depending on the measurement conditions, it may not be possible to obtain an appropriate exposure output at every pixel of the image sensor, such as when the object to be measured is made of a material with low reflectivity of laser light, when the distance from the 3D shape measuring device to the object is long, or when there are areas where reflected light from the object cannot be obtained due to the position of the laser emission.

[0007] Furthermore, the analog signal amplification control and exposure control of the image sensor were performed on a per-screen basis, which meant that the exposure output could not be adjusted for each individual pixel.

[0008] The present invention aims to provide a three-dimensional shape measuring device and a three-dimensional shape measuring method that can obtain distance measurement data to an object to be measured with uniform accuracy using multiple pixels in an image sensor, and measure its three-dimensional shape. [Means for solving the problem]

[0009] To achieve the above objective, the present invention provides a three-dimensional shape measuring device comprising: a light-emitting unit that emits laser light toward an object to be measured; an image sensor having a plurality of pixels, each of which receives reflected light from the object to be measured based on predetermined exposure conditions, converts it into an output signal; an output amount detection unit that detects the exposure output amount for each pixel for a single emission from the light-emitting unit based on the output signal output from the image sensor; and, based on the output signal received and output by the image sensor at an exposure timing that includes all the reflected light reflected by the object to be measured for a single emission from the light-emitting unit, the exposure output amount for each pixel detected by the output amount detection unit is acquired as a reference exposure output amount, and the measurement conditions for each pixel for measuring the distance to the object to be measured are calculated based on the acquired reference exposure output amount for each pixel. The system includes a calculation unit, a measurement unit that, under the measurement conditions calculated by the calculation unit, measures the distance to the object to be measured for each pixel based on the output signal received by each image sensor at an exposure timing where the amount of exposure increases as the light reception timing, in which the reflected light reflected from the object to be measured is received by the image sensor for each light emission, is delayed relative to the light emission timing, and generates three-dimensional shape information of the object to be measured using the distance information for each pixel to the object to be measured measured by the measurement unit, the calculation unit calculates the number of integrations required to bring the reference exposure output amount for each pixel closer to a preset target value for the exposure output amount, and calculates the number of light emissions and exposures for each pixel, which are the measurement conditions, based on the calculated number of integrations.

[0010] Furthermore, the present invention provides a three-dimensional shape measurement device comprising: a light-emitting unit that emits laser light toward an object to be measured; an image sensor having a plurality of pixels, each of which receives reflected light from the object to be measured based on predetermined exposure conditions, converts it into an output signal; and an output amount detection unit that detects the exposure output amount for each pixel for a single emission from the light-emitting unit based on the output signal output from the image sensor. The device acquires the exposure output amount for each pixel detected by the output amount detection unit as a reference exposure output amount based on the output signal received and output by each image sensor at an exposure timing that includes all the reflected light reflected from the object to be measured for a single emission from the light-emitting unit, and sets the reference exposure output amount for each pixel in advance. The system calculates the number of integrations required to approach a predetermined target value for exposure output, calculates the number of light emissions and exposures for each pixel, which are the measurement conditions for each pixel used to measure the distance to the object to be measured, and, based on the calculated number of light emissions and exposures, measures the distance to the object to be measured for each pixel based on the output signal received by each image sensor at an exposure timing where the exposure output increases as the timing of receiving reflected light reflected from the object to be measured is delayed relative to the light emission timing, and the exposure output for each pixel detected by the output amount detection unit is used to measure the distance to the object to be measured for each pixel. The system then generates three-dimensional shape information of the object to be measured using the measured information on the distance to the object for each pixel. [Effects of the Invention]

[0011] According to the 3D shape measuring device and 3D shape measuring method of the present invention, distance measurement data to the object to be measured can be obtained with uniform accuracy in multiple pixels within the image sensor, and the 3D shape can be measured. [Brief explanation of the drawing]

[0012] [Figure 1] This is a block diagram showing the configuration of a 3D shape measuring device according to one embodiment of the present invention. [Figure 2](a) is a diagram showing the exposure output from a predetermined pixel when a conventional 3D shape measuring device emits laser light with the object to be measured included in the subject and performs exposure so as to include all of the laser light reflected by the object to be measured; (b) is a diagram showing the exposure output from a predetermined pixel when exposure is performed at a timing in which the exposure output increases as the laser light reflected by the object to be measured is delayed relative to the emission timing; and (c) is a diagram showing the exposure output from a predetermined pixel when exposure is performed without emitting light and the object to be measured is not included in the subject. [Figure 3] This flowchart shows the process when a 3D shape measuring device according to one embodiment of the present invention performs calculation processing for measurement conditions. [Figure 4] This flowchart shows the measurement process performed by a 3D shape measuring device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0013] The configuration of a three-dimensional (3D) shape measuring device and a 3D shape measuring method according to embodiments of the present invention will be described below with reference to the drawings.

[0014] <Configuration of a 3D shape measuring device according to one embodiment> Figure 1 is a block diagram showing the configuration of a 3D shape measuring device 1 according to one embodiment of the present invention. The 3D shape measuring device 1 measures the shape of a predetermined object to be measured and includes an input unit 11, a light-emitting unit 12, a lens unit 13, an image sensor (image sensor) 14, and a CPU 15.

[0015] The input unit 11 receives information about the user's operation. The light-emitting unit 12 emits laser light toward the object to be measured. The lens unit 13 receives the reflected light that has been reflected by the object to be measured from the laser light emitted from the light-emitting unit 12. The image sensor 14 has x pixels p1 to px, and light-receiving units r1 to rx (not shown) and signal amplifiers am1 to amx (not shown) provided for each pixel p1 to px.

[0016] Hereinafter, when it is not specified which one of pixels p1 to px is used, it is described as "pixel p". Similarly, when it is not specified which one of light-receiving parts r1 to rx is used, it is described as "light-receiving part r". Similarly, when it is not specified which one of signal amplifiers am1 to amx is used, it is described as "signal amplifier am".

[0017] The light-receiving part r receives the light incident from the lens part 13 and performs photoelectric conversion based on predetermined exposure conditions. The signal amplifier am converts the voltage of the electric charge generated and accumulated by photoelectric conversion in the corresponding light-receiving part r, amplifies it, and outputs it as an output signal.

[0018] The CPU 15 includes a light emission control part 151, an output amount detection part 152, an arithmetic part 153, a signal amplification amount control part 154, an exposure control part 155, a measurement part 156, and a shape information generation part 157.

[0019] The light emission control part 151 controls the light emission from the light emission part 12. The output amount detection part 152 detects the exposure output amount for each pixel p with respect to one light emission from the light emission part 12 based on the output signal output for each pixel p of the image sensor 14.

[0020] The arithmetic part 153 obtains, as a reference exposure output amount, the exposure output amount for each pixel p detected by the output amount detection part 152 based on the output signal received and output by each light-receiving part r at the exposure timing including all the reflected light reflected by the measurement object with respect to one light emission from the light emission part 12. In the present embodiment, the arithmetic part 153 sets, for each pixel p, the number of light emissions and exposure times for measuring the distance to the measurement object as measurement conditions, and calculates (performs arithmetic operations) the measurement conditions based on the obtained reference exposure output amount for each pixel p.

[0021] The signal amplification amount control part 154 controls the signal amplification amount by the signal amplifier am for each pixel p of the image sensor 14. The exposure control part 155 controls the exposure time and exposure timing for each pixel p of the image sensor 14.

[0022] The measurement unit 156 measures the distance to the measurement object for each pixel p based on the output signal detected by the output amount detection unit 152 under the measurement conditions calculated by the calculation unit 153. The shape information generation unit 157 generates 3D shape information of the measurement object using the information measured by the measurement unit 156.

[0023] 〈Operation of 3D Shape Measurement Device According to One Embodiment〉 Hereinafter, as a general operation of the 3D shape measurement device 1, the process of generating 3D shape information of a predetermined measurement object B using the dToF (Direct Time of Flight) technology will be described.

[0024] When the measurement object B is placed in the measurement target range of the 3D shape measurement device 1 and the light emitting unit 12 emits laser light toward the measurement object B under the control of the light emission control unit 151, the emitted laser light is reflected by the measurement object B and enters from the lens unit 13 and is received by the light receiving unit r of each pixel p of the image sensor 14.

[0025] In the image sensor 14, each light receiving unit r photoelectrically converts the received light, and the corresponding signal amplifier am converts the charge generated by the photoelectric conversion into a voltage, amplifies it, and outputs it as an output signal. The output amount detection unit 152 detects the exposure output amount based on the output signal output for each pixel p of the image sensor 14 and sends it to the measurement unit 156.

[0026] The measurement unit 156 measures the distance to the measurement object B for each pixel p based on the exposure output amount detected by the output amount detection unit 152.

[0027] The parameters used by the measurement unit 156 to measure the distance to the measurement object B for each pixel p will be described. Fig. 2(a) is a diagram showing the exposure output amount S'0 output from a predetermined pixel p when laser light is emitted from the light emitting unit 12 with the measurement object B included in the subject and exposure is performed by setting the exposure time and exposure timing so as to include all the laser light reflected by the measurement object B.

[0028] In Figure 2(a), Tp (time t0~t1) is the emission time of the laser beam from the light-emitting unit 12, and Tq (time t0~t2) is the exposure time at the light-receiving unit r. The exposure output amount S'0 output from a predetermined pixel p within the exposure time Tq includes the exposure output amount S0 for the reflected light of the object to be measured B and the exposure output amount BG for the reflected light of the background other than the object to be measured B. The exposure output amount S0 is used as the reference exposure amount in the calculation processing of measurement conditions and the measurement processing up to the object to be measured B, which will be described later.

[0029] Here, the start time of exposure time Tq is the same as the start time of emission time Tp, time t0, and the end time of exposure time Tq, time t2, is later than the end time of emission time Tp, time t1, and is the time after all of the reflected light from the object B being measured has been exposed.

[0030] Figure 2(b) shows the exposure output S'1 output from a predetermined pixel p when, with the object to be measured B included in the subject, laser light is emitted from the light-emitting unit 12, and exposure is performed at an exposure timing such that the exposure output increases as the timing of receiving the laser light reflected by the object to be measured B and received by the light-receiving unit r is delayed relative to the emission timing.

[0031] In Figure 2(b), Tp (times t5-t6) is the emission time of the laser beam from the light-emitting unit 12, and Tq (times t6-t8) is the exposure time at the light-receiving unit r. The exposure output amount S'1 output from a predetermined pixel p within the exposure time Tq includes the exposure output amount S1 for the reflected light of the object to be measured B and the exposure output amount BG for the reflected light of the background other than the object to be measured B.

[0032] Here, the start timing of the exposure time Tq is the same as the end timing of the light emission time Tp, which is time t6. The exposure output S1 is the exposure output for the reflected light of object B during the period from time t6, which is the start timing of exposure, to time t7, which is the end timing of receiving the reflected light of object B. The exposure time Δt is the time from time t6, which is the start timing of exposure, to time t7, which is the end timing of receiving the reflected light of object B. The exposure time Δt becomes larger as the distance from the corresponding pixel p to object B increases.

[0033] The ratio of exposure time Δt to emission time Tp is equal to the ratio of exposure output S1 to reference exposure output S0, as shown in equation (1) below. Δt / Tp=S1 / S0 (1)

[0034] Figure 2(c) shows the amount of exposure output from a predetermined pixel p when exposure is performed without light emission, with the measurement target part B not included in the subject.

[0035] In Figure 2(c), Tq (times t9~t10) is the exposure time at the light-receiving unit r. The amount of exposure output from a predetermined pixel p within the exposure time Tq is the amount of exposure output BG for reflected light from the background other than the object being measured B.

[0036] The measurement unit 156 performs three types of exposure as described in Figures 2(a), (b), and (c) during a single measurement of the object B to be measured, and acquires the exposure output amounts S'0, S'1, BG, and the values ​​of S0 and S1 calculated from these for each pixel in the image sensor 14. Then, the measurement unit 156 uses the acquired values ​​to measure the distance to the object B to be measured for each pixel p.

[0037] Here, since the emission time Tp is very short (for example, a few nanoseconds), it is difficult to obtain the exposure output amount used to calculate the distance to the object B with a single emission and exposure. Therefore, the measurement unit 156 obtains the exposure output amount used to calculate the distance to the object B by repeating emission and exposure n times at time intervals longer than the exposure time Tq and integrating the exposure output amounts.

[0038] In other words, the measurement unit 156 calculates the reference exposure output amount S0 in Figure 2(a) using the following formula (2). S0 = n × (S'0 - BG) (2) Furthermore, the measurement unit 156 calculates the exposure output amount S1 in Figure 2(b) using the following formula (3). S1 = n × (S'1 - BG) (3) In equations (2) and (3) above, n is an integer.

[0039] Using the various parameters described above, the measurement unit 156 calculates the distance Z to the object B for each pixel p using the following formula (4). Z=C×Δt2=C×Tp / 2×(S1 / S0) (4) Here, C is the speed of light.

[0040] Incidentally, the amount of light exposure to the image sensor when taking a picture with a typical digital camera is determined by six parameters: (i) light source intensity, (ii) subject reflectivity, (iii) lens T value, (iv) lens aperture value, (v) sensor exposure time, and (vi) signal amplification amount by the circuit.

[0041] When taking images with a digital camera, the photographer adjusts parameters (iv), (v), and (vi) in accordance with the parameters (i), (ii), and (iii) described above, and may adjust parameter (i) to set appropriate imaging conditions.

[0042] In contrast, when measuring devices such as 3D shape measuring devices measure the distance to the subject for each pixel of an image sensor using dToF (Direct Time of Flight) technology, the following parameters are added to determine the exposure amount to the image sensor: (vii) flash duration and flash count, and (viii) the number of exposures corresponding to the flash count.

[0043] When using dToF technology, the emission time and exposure time are determined by the distance range from the measuring device to the object being measured, and these parameters are controlled by the number of emission and exposure cycles n. To measure the distance from the measuring device to the object being measured with greater accuracy, it is necessary to set the value of n so that the aforementioned reference exposure output S0 becomes the largest possible target value within the capacity of the light-receiving unit r.

[0044] However, depending on the shape of the object B to be measured, the distance from the 3D shape measuring device 1 differs for each pixel p, so the n value required to bring the reference exposure output amount S0 closer to a predetermined target value differs for each pixel p. Therefore, the 3D shape measuring device 1 of this embodiment calculates appropriate measurement conditions for each pixel p so that the reference exposure output amount S0 detected by all pixels p is as close as possible to a predetermined target value, and then performs the above-described measurement process for each pixel p based on the calculated measurement conditions.

[0045] The following describes the calculation process for measurement conditions performed by the 3D shape measuring device 1. Figure 3 is a flowchart showing the process performed by the CPU 15 when calculating the measurement conditions.

[0046] First, the user performs an operation (reference count setting operation) to set reference values ​​for the emission of laser light from the light-emitting unit 12 and the number of exposures in a single measurement process, as reference exposure conditions to be used in calculating the measurement conditions. At this time, the user places the 3D shape measuring device 1 at a predetermined position in an environment where infrared light is almost not generated from sources other than the 3D shape measuring device 1, that is, in an environment where BG≈0, and places a subject E with high infrared reflectivity and that does not generate infrared light on its own at a distance of the minimum measurable distance of the 3D shape measuring device 1 from that position, and performs the reference count setting operation from the input unit 11 of the 3D shape measuring device 1.

[0047] When the user performs the reference number setting operation, the 3D shape measuring device 1 measures the reference exposure output amount S0 for one flash of light. Specifically, the light emission control unit 151 controls the light emission unit 12 to emit laser light, the light receiving unit r of each pixel p of the image sensor 14 receives the reflected light from the subject E and performs photoelectric conversion, the signal amplifier am converts the charge generated by the photoelectric conversion into a voltage and amplifies it, and outputs it as an output signal. The output amount detection unit 152 detects the reference exposure output amount S0 based on the output signal and sends it to the calculation unit 153.

[0048] The calculation unit 153 calculates the number of times to accumulate the exposure output amount S'0 of the central pixel p of the image sensor 14 so that the value is as close as possible to the lower limit (S0min) of the reference exposure output amount S0 required for proper measurement processing, and sets the reference flash count n = N (step S1). The value of S0min is set in advance based on the charge capacity of the image sensor 14. In other words, the reference flash count N is set so that the reference exposure output amount S0 does not saturate with respect to the charge capacity of the image sensor 14 through the above process.

[0049] Next, the user places the 3D shape measuring device 1 and the object to be measured B in the designated positions and performs a measurement start operation to calculate the measurement conditions. The measurement conditions for the object to be measured B by the 3D shape measuring device 1 are calculated as the number of light emission and exposures for each pixel. When the measurement start operation to calculate the measurement conditions is performed, the calculation unit 153 performs the following calculation processes for the measurement conditions: (a-1) discrimination processing for measurable conditions, (a-2) discrimination processing for effective pixels and invalid pixels, (b) calculation processing for the number of light emission and exposures, and (c) calculation processing for the signal amplification amount. These processes are described below.

[0050] (A-1) Discrimination process for measurable conditions When the measurement start operation is performed to calculate the measurement conditions, the calculation unit 153 first performs a determination process for each pixel p of the image sensor 14 to determine whether or not measurement of the object to be measured B is possible (step S2). In this determination process, the number of light emission and exposures is set as the reference number of light emission N, and the calculation unit 153 measures and integrates the exposure output amount S'0 and BG for each light emission. The integrated value of the exposure output amount S'0 for N times is set as the exposure output amount S'0(N), and the integrated value of the exposure output amount BG is set as the exposure output amount BG(N).

[0051] Based on the charge capacitance of the image sensor 14, the maximum value that the exposure output amount S'0(N) can take is set in advance as S'0max. The calculation unit 153 determines that pixels p that satisfy the measurable conditions shown in the following equation (5), that is, pixels p whose calculated exposure output amount S'0(N) is less than or equal to S'0max, are measurable pixels p. S'0max ≧ BG(N)+S0min=S'0(N) (5)

[0052] Furthermore, for pixels p that do not satisfy the measurable conditions shown in equation (5) above, specifically, pixels p whose calculated exposure output amount S'0(N) is greater than S'0max, the charge accumulated in the image sensor 14 becomes saturated, and the calculation unit 153 determines that these pixels p are unmeasurable.

[0053] The calculation unit 153 stores information indicating that the measurement data for a pixel p determined to be unmeasurable is unmeasurable, such as "0" or a value greater than the maximum expected number of light emission and exposure counts, as the measured value up to the object B corresponding to the pixel p. An unmeasurable pixel p that does not satisfy the above formula (5) is, for example, a pixel located closer to the object B than the predetermined measurable range.

[0054] (A-2) Discrimination process for valid and invalid pixels Next, the calculation unit 153 performs a determination process for each pixel determined to be measurable in (a-1) to distinguish between effective pixels that satisfy the calculated measurement conditions (hereinafter simply referred to as "effective pixels") and invalid pixels that do not satisfy the calculated measurement conditions (hereinafter simply referred to as "invalid pixels") (step S3). In this determination process, the number of light emission and exposure times is set to N times and a × N times, and the calculation unit 153 measures and integrates the exposure output amount S'0 for each light emission. a is an integer, for example, "2". The integrated value of the exposure output amount S'0 for a × N times is taken as the exposure output amount S'0(a × N).

[0055] The calculation unit 153 determines that a pixel p is an effective pixel if the ratio of the exposure output S'0(a×N) when the number of flashes is a×N to the exposure output S'0(N) when the number of flashes is N is equal to or greater than a preset judgment value A, i.e., a pixel p that satisfies the following equation (6). The judgment value A is a value that is set appropriately based on the value of S'0(N), the amount of measurement noise, etc., and the judgment value A increases as a increases. S'0(a×N) / S'0(N) ≥ A (6)

[0056] Furthermore, the calculation unit 153 determines that pixels that do not satisfy the above equation (6), for example, when a=A=2, are invalid pixels if the exposure output amount S'0 is less than doubled even when the number of flashes is doubled.

[0057] The calculation unit 153 stores information indicating that the exposure output amount S'0 data for a pixel p determined to be an invalid pixel is invalid, such as "0" or a value greater than the maximum expected number of light emission and exposure counts, as the measured value up to the object B corresponding to the pixel p. Invalid pixels p that do not satisfy the above formula (6) are, for example, pixels located at a distance from the object B that is farther than the measurable range, or pixels located at a position where it is difficult to receive reflected light due to the angle of the reflective surface of the object B.

[0058] By performing the (a-1) discrimination process for measurable conditions and the (a-2) discrimination process for valid and invalid pixels described above, the measurement process described later will not be performed on pixels that cannot be measured and on pixels whose exposure output amount S'0 data is determined to be invalid, thereby reducing wasted processing time.

[0059] (i) Calculation process for light emission and exposure count, and (iii) Calculation process for signal amplification amount. Next, the calculation unit 153 performs calculation processing for the number of light emission and exposures, and calculation processing for the signal amplification amount for each pixel determined to be valid in (a-2) (step S4). In this calculation processing, the exposure output amount S'0(N) and exposure output amount BG(N) when the number of light emission and exposures is N are used.

[0060] In order to accurately calculate the distance Z to the object B to be measured, the S0 value must satisfy the following equation (7). S0max is the maximum value of S0 that allows for proper calculation of the measurement conditions, and is set based on the charge capacitance of the image sensor 14. S0min is the minimum value of S0 that is assumed to allow for proper calculation of the measurement conditions. S0max ≥ S0 ≥ S0min (7)

[0061] The calculation unit 153 calculates the number of light emission and exposures and the signal amplification amount for each pixel p to be processed, so as to satisfy equation (7). Here, if the target value of the reference exposure output amount S0 is S0typ, the calculation unit 153 calculates coefficients b and f so that the value calculated by the following equation (8) is as close as possible to S0typ. S0typ is set in advance, for example, based on the charge capacitance of the image sensor 14. (S'0(N)-BG(N))×b×f (8)

[0062] The coefficient b is a value that satisfies equation (9) below. S'0(N)×b ≤ S'0max (9)

[0063] The calculation unit 153 calculates the value obtained by multiplying the calculated coefficient b by N as the number of light emission and exposures N1 for the corresponding pixel p, and calculates the coefficient f as the signal amplification amount. Here, by making the value of coefficient b as large as possible (maximum value) and the coefficient f as small as possible within the range that satisfies the above equation (9), the measurement accuracy up to the object B can be improved by approaching the target value S0typ of the reference exposure output amount S0. In this case, the signal amplification amount indicated by coefficient f is set to the same value for all pixels p in the image sensor 14.

[0064] Furthermore, by calculating the number of flashes and exposures N1 without increasing the value of coefficient b to its maximum, and then increasing the value of coefficient f to a certain extent to approach the target value S0typ of the reference exposure output S0, the measurement time to the object B can be shortened. In this case, coefficient f may be set to a different value for each pixel, and the signal amplification control unit 154 may control the signal amplification amount for each pixel p.

[0065] Furthermore, when calculating the coefficient f, an optical correction amount to compensate for light loss due to the lens section 13 may be included in the calculation. By including the optical correction amount in the coefficient f, the S0 value of all pixels p of the image sensor 14 can be brought closer to S0typ, thereby equalizing the distance measurement accuracy between pixels p.

[0066] Moving to step S5, the calculation unit 153 generates multiple pixel groups G1, G2, etc. by grouping together multiple pixels p with similar calculated values ​​based on the light emission and exposure count N1 for each pixel p calculated in step S4. At this time, the calculation unit 153 generates pixel groups by excluding (a-1) pixels p for which information indicating that measurement is not possible is held in the determination process for measurable conditions, and (a-2) pixels for which information indicating that the exposure output amount S'0 data is invalid is held in the determination process for valid pixels and invalid pixels.

[0067] As the number of pixel groups increases, the time required for a single shape measurement also increases; therefore, it is desirable to limit the number of pixel groups to around 4 or 5. In equation (8), by calculating the coefficient b as an integer and a factorial of 2, the number of pixel groups generated here can be reduced.

[0068] The calculation unit 153 determines the number of light emission and exposure counts N2_1, N2_2, etc. for each pixel group G1, G2, etc. Hereafter, if the pixel group is not specified, it will be written as "pixel group G", and if the number of light emission and exposure count is not specified, it will be written as "number of light emission and exposure count N2" or simply "count N2".

[0069] For example, pixels p whose calculated emission and exposure count N1 is between 995 and 1005 times are grouped into one pixel group G1, and the emission and exposure count N2 for that pixel group G1 is set to 1000 times. The count N2_1 for a given pixel group G1 is derived, for example, from a histogram showing the corresponding number of pixels for each of the multiple pixels p belonging to that pixel group G1 whose emission and exposure count N1 has been calculated.

[0070] The calculation unit 153 sets the determined counts N2_1, N2_2, etc. for each pixel group G1, G2, etc. as the number of times each pixel p belonging to the corresponding pixel group G emits light and is exposed, and generates integrated data that links the number of times each pixel p emits light and is exposed N2 with the pixel address indicating the position of each pixel p (step S6).

[0071] The calculation unit 153 sends the generated integrated data to the measurement unit 156. The measurement unit 156 sets the acquired integrated data as measurement conditions for the object B to be measured, and performs measurement processing up to the object B for each pixel group based on these conditions. Figure 4 is a flowchart showing the measurement processing performed by the measurement unit 156.

[0072] The measurement unit 156 first identifies pixel group G1 as the pixel group to be measured (step S11), and then measures the distance to the object B for each pixel p belonging to the corresponding pixel group G1 based on the set measurement conditions (step S12).

[0073] Specifically, the measurement unit 156 causes the light-emitting unit 12 to emit light from the light-emitting control unit 151 to perform the three types of exposure shown in Figures 2(a), (b), and (c) for the number of times N2_1 minutes of the pixel group G1, exposes the image sensor 14 under the control of the exposure control unit 155, amplifies the signal by the signal amplification amount f set by the signal amplification amount control unit 154, and the output signal is detected by the output amount detection unit 152.

[0074] The measurement unit 156 calculates a reference exposure output S0 for each corresponding pixel p by integrating the exposure output S'0-BG for N2_1 cycles at the exposure timing shown in Figure 2(a) based on the exposure output detected by the output amount detection unit 152, and calculates an exposure output S1 by integrating the exposure output S'1-BG for N2_1 cycles at the exposure timing shown in Figure 2(b). Then, the measurement unit 156 calculates the distance to the object B for each corresponding pixel p using the calculated reference exposure output S0 and S1 and the above-described equation (4).

[0075] Similarly, for pixel groups G2 and beyond, the process of calculating the distance to the object B for each corresponding pixel p is performed, and this process is repeated until the measurement process for all pixel groups G is completed (if "NO" is selected in step S13, then steps S11 and S12).

[0076] Once the measurement process for all pixel groups G is complete ("YES" in step S13), the shape information generation unit 157 generates 3D shape information of the object to be measured using the measurement information of all pixels p measured by the measurement unit 156 (step S14).

[0077] In this way, by grouping the pixels p in the image sensor 14 according to the required number of light emission and exposure cycles, and performing measurement processing for each group, highly accurate measurement processing can be performed in a short time.

[0078] According to the above embodiments, the 3D shape measuring device can obtain uniform and highly accurate measurement data in multiple pixels within the image sensor.

[0079] In the embodiment described above, the calculation unit 153 calculates the number of light emission and exposures for each pixel as a process for calculating measurement conditions. However, it is not limited to this, and the light emission intensity for each pixel may also be calculated. In this case, the calculation unit 153 calculates a value obtained by multiplying the coefficient b of equation (8) above by the initial value of the light emission intensity as the light emission intensity of the corresponding pixel p, and groups together multiple pixels with similar calculated values ​​to generate multiple pixel groups. The calculation unit 153 then determines the light emission intensity of each pixel group, sets it as the light emission intensity of each pixel belonging to the corresponding pixel group, and generates integrated data by associating it with the pixel address of each pixel p.

[0080] The measurement unit 156 then sets the generated integrated data as measurement conditions for the object to be measured, and performs measurement processing up to object B for each pixel group, i.e., for each emission intensity, based on these conditions. By performing processing in this manner, the number of emission and exposure times is reduced compared to when the number of emission and exposure times for each pixel is set as measurement conditions, thereby shortening the measurement time.

[0081] In the embodiment described above, the image sensor 14 may be configured to allow setting the number of exposures for each pixel p. By configuring it in this way, each pixel p can be exposed with its corresponding number of exposures in a single sequence in which the light-emitting unit 12 emits light with the highest number of light emission cycles in the integrated data, thereby shortening the measurement time.

[0082] Furthermore, in the embodiment described above, the light-receiving unit r of the image sensor 14 may be configured to subtract the background portion corresponding to the BG signal from the received light information before converting the received light into photoelectricity. By configuring it in this way, accuracy can be improved when performing measurement processing in an outdoor environment with a large amount of BG signal.

[0083] Furthermore, in the above-described embodiment, the image sensor or pixel that performs calculation processing of measurement conditions may be separated from the image sensor or pixel that performs measurement processing. By configuring the image sensor in this way, and by configuring it so that the number of exposures can be set for each pixel as described above, a 3D shape measuring device with high measurement accuracy and capable of handling video can be constructed. [Explanation of Symbols]

[0084] 1. 3D shape measuring device 11 Input section 12 Light-emitting part 13 Lens section 14. Image sensor (imaging element) 15 CPU 151 Light emission control unit 152 Output amount detection unit 153 Arithmetic section 154 Signal amplification control unit 155 Exposure Control Unit 156 Measurement Unit 157 Shape information generation section

Claims

1. A light-emitting unit that emits laser light towards the object to be measured, An image sensor having multiple pixels, wherein each pixel receives reflected light from the object being measured after laser light emitted from the light-emitting unit has been reflected, based on predetermined exposure conditions, and converts it into an output signal. An output amount detection unit detects the exposure output amount for each pixel for a single light emission from the light-emitting unit based on the output signal output from the image sensor, Based on the output signal received and output by the image sensor at an exposure timing that includes all the reflected light reflected by the object to be measured for a single emission from the light-emitting unit, the exposure output amount for each pixel detected by the output amount detection unit is acquired as a reference exposure output amount, and the calculation unit calculates the measurement conditions for each pixel for measuring the distance to the object to be measured based on the acquired reference exposure output amount for each pixel. Based on the measurement conditions calculated by the calculation unit, the measurement unit measures the distance to the object to be measured for each pixel based on the output signal received by each image sensor at an exposure timing where the amount of exposure output increases as the reception timing, in which the reflected light reflected from the object to be measured is received by the image sensor for a single emission from the light-emitting unit, is delayed relative to the emission timing, and based on the exposure output amount for each pixel detected by the output amount detection unit, the measurement unit measures the distance to the object to be measured for each pixel. A three-dimensional shape measuring device comprising: a shape information generation unit that generates three-dimensional shape information of the object to be measured using the distance information of each pixel to the object to be measured, which is measured by the measurement unit;

2. The calculation unit calculates, as the measurement conditions, the number of times the light is emitted and exposed for each pixel based on the number of integrations to bring the reference exposure output amount for each pixel closer to a preset target value of the exposure output amount, or the light emission intensity for each pixel to bring the reference exposure output amount for each pixel closer to a target value of the exposure output amount, as the three-dimensional shape measuring device according to claim 1.

3. The calculation unit determines, among the reference exposure output amounts for each pixel, pixels corresponding to a reference exposure output amount that, when accumulated a predetermined number of times, exceeds the charge capacity of the image sensor, as pixels that cannot be measured, and pixels corresponding to a reference exposure output amount that, even after accumulating a predetermined number of times, does not reach a predetermined value, as invalid pixels. The three-dimensional shape measuring device according to claim 1, wherein the measurement unit does not perform measurement processing on pixels determined to be unmeasurable and pixels determined to be invalid.

4. The calculation unit generates multiple pixel groups that group together pixels with similar values ​​for the calculated measurement conditions, and determines the measurement conditions for each pixel group based on the measurement conditions for each pixel belonging to each pixel group. The three-dimensional shape measuring device according to claim 1, wherein the measuring unit measures the distance to the object to be measured for each pixel belonging to the pixel group, based on the exposure output amount detected for each pixel, under the measurement conditions determined for the pixel group.

5. A light-emitting unit that emits laser light towards the object to be measured, An image sensor having multiple pixels, wherein each pixel receives reflected light from the object being measured after laser light emitted from the light-emitting unit has been reflected, based on predetermined exposure conditions, and converts it into an output signal. A three-dimensional shape measuring device comprising an output amount detection unit that detects the exposure output amount for each pixel for a single light emission from the light emission unit based on the output signal output from the image sensor, Based on the output signal received and output by each front image sensor at an exposure timing that includes all the reflected light reflected by the object to be measured for a single emission from the light-emitting unit, the exposure output amount for each pixel detected by the output amount detection unit is acquired as the reference exposure output amount, the number of integrations required to bring the reference exposure output amount for each pixel closer to a preset target value of exposure output amount is calculated, and based on the calculated number of integrations, the number of emissions and exposures for each pixel, which are the measurement conditions for each pixel to measure the distance to the object to be measured, are calculated. Based on the calculated number of light emission and exposures, the exposure output increases as the reception timing, in which the reflected light reflected from the object to be measured is received by the image sensor for each light emission timing, is delayed relative to the light emission timing. Based on the output signal received and output by each image sensor at the exposure timing, the distance to the object to be measured is measured for each pixel based on the exposure output amount for each pixel detected by the output amount detection unit. A three-dimensional shape measurement method that generates three-dimensional shape information of the object to be measured using information on the distance of each pixel to the measured object.

Images