Distance measuring device and calibration method

Abstract

A distance measurement device according to the present invention comprises: a light emitting unit which emits light; a light receiving sensor which receives light that has been emitted by the light emitting unit and reflected by a target; and a calibration calculating unit which, as a calibration calculation process for finding a correction parameter for distance information calculated via indirect ToF on the basis of a received light signal from the light receiving sensor, performs a calculation process using a received light signal from the light receiving sensor when the light emitting unit emits light at a first light emission frequency, and a received light signal from the light receiving sensor when the light emitting unit emits light at a second light emission frequency that differs from the first light emission frequency.

Description

[Technical field]

[0001] The present technology relates to a distance measuring device and a calibration method thereof, and in particular to a technology for determining correction parameters for distance information calculated by an indirect ToF method. [Background technology]

[0002] Various distance measuring techniques for measuring the distance to a target object are known, and in recent years, distance measuring techniques using a ToF (Time of Flight) method, for example, have been attracting attention. Known ToF methods include a direct ToF method and an indirect ToF method.

[0003] Of these ToF methods, the indirect ToF method measures distance by emitting sine wave light and receiving the light that hits the target object and is reflected. In this case, the light receiving sensor has pixels arranged in a two-dimensional array. Each pixel has a light receiving element and can capture light. Each pixel receives light in synchronization with the phase of the emitted light, thereby obtaining the phase and amplitude of the received sine wave. The phase is based on the emitted sine wave.

[0004] The phase of each pixel corresponds to the time it takes for the light from the light emitting unit to be reflected by the target object and input to the sensor. Therefore, by dividing the phase by 2πf, multiplying it by the speed of light (hereafter referred to as "c") and dividing by 2, the distance to the point of the target (distance measurement point) projected on that pixel can be calculated. Here, f means the frequency of the emitted sine wave.

[0005] Here, in the indirect ToF, the light actually emitted is not strictly a sine wave (for example, a square wave). Therefore, the distance calculated by the above calculation is not strictly the correct distance. The element that causes an error in the distance due to the emitted light not being a sine wave is known as "circular error". If this circular error can be determined, the correct distance can be determined by correcting the distance using the circular error.

[0006] Non-Patent Document 1 below discloses a technique for correcting distance using a correction parameter as this circular error. [Prior art documents] [Patent documents]

[0007] [Non-Patent Document 1] Fuchs, S., May, S.: Calibration and registration for precise surface reconstruction with time-of-flight cameras.Int.J.Intell.Syst.Technol.Appl.5, 274-284(2008) Summary of the Invention [Problem to be solved by the invention]

[0008] Here, conventionally, calibration for obtaining a correction parameter as a circular error has been performed under the condition that the distance to the target object is a known distance, and the target object needs to be precisely positioned at the known distance. Therefore, conventional calibration has been performed using a precision device before the product is shipped, and it has been considered very difficult to perform in an actual usage environment after the product is shipped.

[0009] The present technology has been developed in consideration of the above circumstances, and aims to make it possible to perform calibration for obtaining correction parameters for distance information calculated by the indirect ToF method in an environment in which the device is actually used. [Means for solving the problem]

[0010] A first distance measuring device according to the present technology includes an optical element that emits light, a light receiving sensor that receives light emitted by the optical element and reflected by a target object, and a calibration calculation unit that performs a calibration calculation process to obtain a correction parameter for distance information calculated by an indirect ToF method based on the light receiving signal of the light receiving sensor, using a light receiving signal of the light receiving sensor when the optical element emits light at a first light emission frequency and a light receiving signal of the light receiving sensor when the optical element emits light at a second light emission frequency different from the first light emission frequency. By using a plurality of emission frequencies, it becomes possible to obtain the correction parameters even if the distance to the target object is not fixed.

[0011] In the first distance measuring device according to the present technology described above, it is considered that the calibration calculation unit is configured to obtain the correction parameter by performing calculation processing based on a phase difference between light emission and light reception detected based on the light reception signal. This makes it possible to obtain appropriate correction parameters when performing distance measurement using the indirect ToF method as the phase difference method.

[0012] In the first distance measuring device according to the present technology described above, the calibration calculation unit may be configured to perform an ambiguity resolution process for resolving ambiguity in units of 2π regarding the phase difference. This makes it possible to perform calculation processing of the correction parameters using a phase difference that eliminates the uncertainty in 2π units.

[0013] In the first distance measuring device according to the present technology described above, the calibration calculation unit may be configured to determine, among the phase differences detected from the received light signal when light is emitted at a minimum emission frequency, which is the lowest emission frequency among the emission frequencies of the light emitting unit for the calibration calculation process, the phase difference detected from the received light signal having an amplitude equal to or greater than a predetermined value as the phase difference corresponding to the minimum emission frequency, and to perform a process to eliminate the uncertainty regarding the phase difference corresponding to the emission frequencies other than the minimum emission frequency based on the phase difference corresponding to the determined minimum emission frequency. For the phase difference corresponding to the minimum emission frequency, by selecting the phase difference detected from the received light signal whose amplitude is equal to or greater than a predetermined value as described above, it is possible to eliminate the ambiguity in 2π units, and for the phase differences corresponding to emission frequencies other than the minimum emission frequency, it is possible to identify the true phase difference based on the phase difference corresponding to the minimum emission frequency from which the ambiguity has been eliminated in this manner (i.e., it is possible to eliminate the ambiguity in 2π units).

[0014] In the first distance measuring device according to the present technology described above, the calibration calculation unit may be configured to execute the calibration calculation process based on an elapsed time since the previous execution. This makes it possible to redo the calibration of the correction parameters even if the correction parameters deviate from their true values ​​over time.

[0015] In the first ranging device related to the present technology described above, it is possible to configure the calibration calculation unit to interrupt the calibration calculation process and perform processing for ranging when an instruction for ranging is given while the calibration calculation process is being executed. As a result, even if the calibration calculation process is performed in the background, when an instruction for distance measurement is received, the calibration calculation process is interrupted and a distance measurement operation is performed in response to the instruction.

[0016] A first calibration method according to the present technology is a calibration method for a distance measuring device that has an light-emitting unit that emits light and a light-receiving sensor that receives light emitted from the light-emitting unit and reflected by a target object, and performs distance measurement using an indirect ToF method based on a light-receiving signal of the light-receiving sensor, in which, as a calibration calculation process for obtaining a correction parameter for distance information calculated by the indirect ToF method, a calculation process is performed using the light-receiving signal of the light-receiving sensor when the light-emitting unit emits light at a first emission frequency and the light-receiving signal of the light-receiving sensor when the light-emitting unit emits light at a second emission frequency different from the first emission frequency. With this first calibration method as well, the same effect as with the first distance measuring device according to the present technology described above can be obtained.

[0017] A second distance measuring device according to the present technology includes an optical element that emits light, a light receiving sensor that receives the light emitted from the optical element and reflected by a target object at multiple pixels, and a calibration calculation unit that performs a calculation process using the condition that the respective distance measuring points projected onto the multiple pixels are in a specific positional relationship with each other as a calibration calculation process for obtaining correction parameters for distance information calculated by an indirect ToF method based on the light receiving signal of the light receiving sensor. By using the condition that the distance measurement points have a specific positional relationship as described above, it is possible to find the correction parameters even if the distance to the target object is indefinite.

[0018] In the second ranging device related to the present technology described above, it is possible to configure the calibration calculation unit to perform a calculation process using a condition that the ranging points are located on an object of known shape as the calibration calculation process. If the distance measurement points are located on an object having a known shape, it is possible to define the positional relationship between the distance measurement points as a mathematical formula based on the known shape.

[0019] In the second distance measuring device according to the present technology described above, the calibration calculation unit may be configured to perform, as the calibration calculation process, a calculation process using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first light emitting frequency, and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second light emitting frequency different from the first light emitting frequency. In other words, the calibration calculation process involves performing calculations using multiple light emission frequencies while using the condition that each ranging point is in a specific positional relationship with respect to the others, thereby making it possible to increase the number of equations for the unknowns.

[0020] In the second distance measuring device according to the present technology described above, the calibration calculation unit may be configured to obtain the correction parameter by performing calculation processing based on a phase difference between light emission and light reception detected based on the light reception signal. This makes it possible to obtain appropriate correction parameters when performing distance measurement using the indirect ToF method as the phase difference method.

[0021] In the second distance measuring device according to the present technology described above, the calibration calculation unit may be configured to perform an ambiguity resolution process for resolving ambiguity in units of 2π regarding the phase difference. This makes it possible to perform calculation processing of the correction parameters using a phase difference that eliminates the uncertainty in 2π units.

[0022] In the second distance measuring device according to the present technology described above, the calibration calculation unit may be configured to determine, among the phase differences detected from the received light signal when light is emitted at a minimum emission frequency, which is the lowest emission frequency among the emission frequencies of the light emitting unit for the calibration calculation process, the phase difference detected from the received light signal having an amplitude equal to or greater than a predetermined value as the phase difference corresponding to the minimum emission frequency, and to perform a process to eliminate the uncertainty regarding the phase difference corresponding to the emission frequencies other than the minimum emission frequency based on the phase difference corresponding to the determined minimum emission frequency. For the phase difference corresponding to the minimum emission frequency, by selecting the phase difference detected from the received light signal whose amplitude is equal to or greater than a predetermined value as described above, it is possible to eliminate the ambiguity in 2π units, and for the phase differences corresponding to emission frequencies other than the minimum emission frequency, it is possible to identify the true phase difference based on the phase difference corresponding to the minimum emission frequency from which the ambiguity has been eliminated in this manner (i.e., it is possible to eliminate the ambiguity in 2π units).

[0023] In the second ranging device related to the present technology described above, it is possible to configure the device with a guide display processing unit that performs display processing of a guide image that guides the composition to satisfy the condition that the ranging points are in a specific positional relationship with each other. This makes it possible to increase the possibility that the correction parameters will be calibrated under the condition that the distance measurement points are in a specific positional relationship with each other.

[0024] A second calibration method according to the present technology is a calibration method for a distance measuring device that includes an light-emitting unit that emits light, a light-receiving sensor that receives the light emitted from the light-emitting unit and reflected by a target object at multiple pixels, and performs distance measurement using an indirect ToF method based on the light-receiving signal of the light-receiving sensor, in which a calculation process is performed using the condition that the respective distance measurement points projected onto the multiple pixels are in a specific positional relationship with each other as a calibration calculation process for obtaining correction parameters for distance information calculated by the indirect ToF method. With this second calibration method as well, the same effect as with the second distance measuring device according to the present technology described above can be obtained. [Brief description of the drawings]

[0025] [Figure 1] 1 is a block diagram for explaining an example of the internal configuration of a distance measuring device according to a first embodiment of the present technology. [Diagram 2] FIG. 1 is an explanatory diagram of 2π uncertainty. [Diagram 3] 4 is a flowchart of a calibration calculation process according to the first embodiment. [Figure 4] 13 is a flowchart of a 2π ambiguity resolution process. [Diagram 5] 10 is a flowchart of a process executed by a control unit in a second embodiment. [Figure 6] 13 is a flowchart of a calibration calculation process in the second embodiment. [Figure 7] FIG. 13 is a block diagram for explaining an example of the internal configuration of a distance measuring device according to a third embodiment. [Figure 8] FIG. 13 is a diagram illustrating the state of a distance measuring device when performing calibration according to a third embodiment. [Figure 9] FIG. 2 is an explanatory diagram of a planar photography area. [Figure 10] 13A to 13C are diagrams for explaining an example of a guide display during calibration in the third embodiment. [Figure 11] 13 is a flowchart showing a process flow when performing calibration according to a third embodiment. [Figure 12] 13 is a flowchart of a calibration process according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, with reference to the accompanying drawings, embodiments of the present technology will be described in the following order. <1. First embodiment> [1-1. Configuration of distance measuring device] [On the 1-2.2π uncertainty] [1-3. Calibration method as the first embodiment] <2. Second embodiment> <3. Third embodiment> <4. Modifications> <5. Summary of the embodiment> <6. This Technology>

[0027] <1. First embodiment> [1-1. Configuration of distance measuring device] FIG. 1 is a block diagram for explaining an example of the internal configuration of a distance measuring device 1 according to a first embodiment of the present technology. The distance measuring device 1 performs distance measurement using an indirect ToF (Time of Flight) method. The indirect ToF method is a distance measuring method that calculates the distance to a target object Ob based on a phase difference between light Ls irradiated onto the target object Ob and reflected light Lr obtained by reflecting the irradiated light Ls from the target object Ob. In this example, the distance measuring device 1 is configured as a portable information processing device such as a smartphone or tablet terminal having a distance measuring function using an indirect ToF method.

[0028] As shown in the figure, the distance measuring device 1 includes a light emitting unit 2, a sensor unit 3, a lens 4, a phase difference detection unit 5, a calculation unit 6, an amplitude detection unit 7, a control unit 8, a memory unit 9, a display unit 10, and an operation unit 11. The light emitting unit 2 has one or more light emitting elements as a light source, and emits irradiation light Ls to the target object Ob. In this example, the light emitting unit 2 emits infrared light having a wavelength in the range of 780 nm to 1000 nm as the irradiation light Ls. In the indirect ToF method, the irradiated light Ls is intensity-modulated so that the intensity changes at a predetermined cycle. Specifically, in this example, the irradiated light Ls is repeatedly emitted according to the clock CLK. In this case, the irradiated light Ls is not strictly a sine wave, but is approximately a sine wave. In this example, the frequency of the clock CLK is variable, and this makes the emission frequency of the irradiated light Ls variable as well. The emission frequency of the irradiated light Ls can be changed within a predetermined frequency range, for example, with 10 MHz (megahertz) as the basic frequency.

[0029] The sensor unit 3 has a plurality of pixels arranged in a two-dimensional array. Each pixel has a light receiving element such as a photodiode, which receives the reflected light Lr. A lens 4 is attached to the front of the sensor unit 3, and the reflected light Lr is collected by the lens 4 so that it can be efficiently received by each pixel in the sensor unit 3.

[0030] A clock CLK is supplied to the sensor unit 3 as a timing signal for the light receiving operation, and the sensor unit 3 thereby performs the light receiving operation in synchronization with the cycle of the irradiated light Ls emitted by the light emitting unit 2. The sensor unit 3 accumulates the reflected light Lr for tens of thousands of cycles of the irradiated light Ls, and outputs data proportional to the accumulated amount of received light. The reason for accumulation is that although the amount of light received one time is small, the amount of received light can be increased by accumulating tens of thousands of times, and significant data can be obtained. Therefore, the interval at which distance measurement is performed is an interval of tens of thousands of light emission cycles of the irradiated light Ls.

[0031] The phase difference detection unit 5 detects a phase difference corresponding to the time difference between the emission timing of the irradiated light Ls and the reception timing of the reflected light Lr, using data proportional to the accumulated amount of received light output from each pixel of the sensor unit 3. This phase difference is proportional to the distance to the target object Ob. Although not illustrated, in the indirect ToF method, two FDs (floating diffusions) are provided for one light receiving element in each pixel of the sensor unit 3, and the accumulated charge of the light receiving element is distributed to these FDs within one emission cycle of the irradiated light Ls. Then, data proportional to the charge accumulated in these FDs over a period of tens of thousands of emission cycles of the irradiated light Ls is output from each pixel. The phase difference detection unit 5 detects the phase difference based on the data for each FD output from each pixel in this manner.

[0032] The calculation unit 6 calculates the distance for each pixel based on the phase difference detected for each pixel by the phase difference detection unit 5. Specifically, the calculation unit 6 calculates the distance for each pixel by multiplying the phase difference detected by the phase difference detection unit 5 by {c÷(4πf)}, where f is the emission frequency (sine wave frequency) of the irradiation light Ls. Hereinafter, the information indicating the distance for each pixel obtained by the calculation unit 6 will be referred to as a "distance image."

[0033] The amplitude detection unit 7 uses data proportional to the accumulated amount of received light output from each pixel of the sensor unit 3 to detect the amplitude of the received reflected light Lr (sine wave).

[0034] The control unit 8 is configured with a microcomputer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc., and performs overall control of the distance measuring device 1, for example by executing processing according to a program stored in the above-mentioned ROM. For example, the control unit 8 controls the operation of the light emitting unit 2, including the control of the emission frequency of the irradiated light Ls, controls the light receiving operation of the sensor unit 3, and controls the execution of distance calculation processing by the calculation unit 6.

[0035] In addition, the control unit 8 controls the display operation of the display unit 10 and performs various processes according to operation input information from the operation unit 11. The display unit 10 is a display device capable of displaying images, such as a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays various types of information according to instructions from the control unit 8. The operation unit 11 collectively represents operators such as various buttons, keys, a touch panel, etc. provided on the distance measuring device 1. The operation unit 11 outputs operation input information corresponding to an operation input from a user to the control unit 8. The control unit 8 executes processing corresponding to the operation input information, thereby realizing the operation of the distance measuring device 1 corresponding to the operation input from the user.

[0036] The memory unit 9 is composed of, for example, a non-volatile memory, and is used to store various data handled by the control unit 8 and the calculation unit 6. In this embodiment, the memory unit 9 stores information on correction parameters used for distance correction, which will be described later, as parameter information 9a, which will be described later.

[0037] The control unit 8 has a function as a calibration calculation unit 8a. The function of the calibration calculation unit 8a determines correction parameters used for correcting the distance, which will be described later.

[0038] [On the 1-2.2π uncertainty] The uncertainty of the phase difference in units of 2π (hereinafter referred to as "2π uncertainty") will be described with reference to FIG. Fig. 2A shows the change over time in the emission intensity of the irradiated light Ls (sine wave) emitted from the light emitting unit 2. Fig. 2B shows the change over time in the received light intensity of the reflected light Lr from the target object Ob. The phase difference (δ) between Fig. 2A and Fig. 2B is proportional to the distance between the distance measuring device 1 and the target object Ob. If the target object Ob is located further away, the phase difference may shift by an additional 2π (see FIG. 2C) or 4π (see FIG. 2D). Since the phase difference detection unit 5 detects only the phase difference, it is not possible to distinguish between the cases of Figures 2B, 2C, and 2D. In other words, it is not possible to determine whether the phase difference is δ+2sπ (where s is an integer equal to or greater than 0). In terms of distance, it is not possible to determine whether the phase difference is δ+2sπ (where s is an integer equal to or greater than 0). This inability to determine whether the phase difference is δ+2sπ is referred to here as 2π uncertainty. In addition, in the indirect ToF method, the value of s = 0 is generally output. In other words, δ × c ÷ (4πf) is output as the distance. Here, δ is a value greater than or equal to 0 and less than 2π.

[0039] [1-3. Calibration method as the first embodiment] Here, as mentioned above, the irradiated light Ls is not actually a perfect sine wave, and therefore a correction is required in the calculation of the distance in the calculation unit 6. Parameters for the correction calculation are stored as parameter information 9a in the memory unit 9. Therefore, the calculation unit 6 performs a complex calculation in addition to simply "multiplying the phase difference by {c÷(4πf)}". This "complex calculation" will be described below.

[0040] As the parameter information 9a, values ​​A1 to An, B1 to Bn, ag, bg, and cg are stored. These are parameters for performing correction calculations. Note that N is a predetermined value, for example, N=20.

[0041] As described in Chapter 4 of Non-Patent Document 1, it is necessary to correct circular error and signal propagation delay.

[0042] Since circular error has periodicity, it can be expressed by a trigonometric function. Here, the component of circular error at a frequency n times the phase observed by the sensor unit 3 is denoted as An, and the phase shift at that frequency is denoted as Bn. Here, n can take a value from 1 to N.

[0043] The signal propagation delay mainly takes into consideration the signal propagation delay for each pixel in the sensor unit 3. The signal propagation delay for each pixel is caused by the difference in the time until charge resetting occurs depending on the pixel position. As described in Chapter 4 of Non-Patent Document 1, the signal propagation delay is linear with respect to the pixel position. Therefore, the phase shift of the entire pixel is ag, the gradient of the delay amount with respect to the row direction (horizontal direction) position of the pixel position is bg, and the gradient of the delay amount with respect to the column direction (vertical direction) position is cg. Note that in Chapter 4 of Non-Patent Document 1, ag is written as b0, bg as b1, and cg as b2.

[0044] Here, the number of pixels in the sensor unit 3 is U×V, and the pixel positions in the sensor unit 3 are (u, v). In this case, U ranges from u=1, and V ranges from v=1. Moreover, the phase difference observed at a pixel position (u, v) (that is, the phase difference calculated by the phase difference detection unit 5) is defined as θ(u, v). The distance L(u,v) corresponding to a pixel position (u,v) is calculated by the following [Equation 1], which includes the above-mentioned correction parameters An, Bn, ag, bg, and cg.

number

[0045] That is, the calculation unit 6 does not simply "multiply the phase difference θ by {c÷(4πf)}", but performs the calculation shown in [Formula 1] using parameters A1 to An, B1 to Bn, and ag, bg, and cg. The calculation result L(u,v) is obtained as the distance measurement result for the pixel position (u,v).

[0046] The parameters A1 to An, B1 to Bn, ag, bg, and cg are obtained by measurements using precision equipment at the time of product shipment. These obtained values ​​are stored in advance in the memory unit 9 as parameter information 9a.

[0047] Here, there is a risk that the values ​​of the parameters A1 to An and B1 to Bn may deviate from the true values ​​and become inappropriate due to aging. Therefore, it is advisable to perform calibration even while the user is using the distance measuring device 1, and update the parameters A1 to An and B1 to Bn stored as the parameter information 9a. To achieve this, it is desirable to be able to perform calibration easily (to be able to calculate the values ​​of the parameters A1 to An and B1 to Bn) without using a precision device. Of course, when a product is shipped, the method according to the embodiment may be used to perform calibration, and the resulting parameters A1 to An and B1 to Bn may be stored as parameter information 9a before shipping. The advantage of using the method according to the embodiment in this case is that calibration can be performed without installing a precision device in the factory.

[0048] The calibration calculation process according to the first embodiment will be described with reference to the flowchart of Fig. 3. This process is a process of the calibration calculation unit 8a shown in Fig. 1, and is executed by the control unit 8 based on a program stored in a predetermined storage device such as the above-mentioned ROM.

[0049] The amplitude detected by the amplitude detection unit 7 and the phase difference detected by the phase difference detection unit 5 are input to the calibration calculation unit 8a. Then, the calibration calculation unit 8a calculates the values ​​of parameters A1 to An and B1 to Bn (described later) and stores them as parameter information 9a in the memory unit 9 (the values ​​of parameters A1 to An and B1 to Bn are overwritten). As a result, appropriate values ​​of parameters A1 to An and B1 to Bn are always stored as the parameter information 9a, and when the user causes the distance measuring device 1 to perform distance measurement, a correct distance measurement result is obtained according to [Equation 1].

[0050] It is to be noted that the calculation by the calibration calculation unit 8a and the overwriting process to the memory unit 9 may be performed automatically, for example, when a predetermined trigger condition is met when the user turns on the power of the distance measuring device 1.

[0051] This embodiment is characterized in that a plurality of frequencies f (light emission frequencies) are used in the calibration. Specifically, T frequencies f (T is a natural number equal to or greater than 2) are used. Hereinafter, the frequency is referred to as f(t), where t ranges from 1 to T. For example, f(1)=10 MHz, f(2)=11 MHz, f(3)=12 MHz, etc. The frequency f(1) at t=1 is the lowest frequency. For T, for example, T=15.

[0052] Here, the circular error and the signal propagation delay depend on t. That is, for each t, ​​the circular error and the signal propagation delay are stored as correction parameters in the memory unit 9 as parameter information 9a. Hereinafter, the parameters of the circular error at t are denoted as A1(t) through An(t) and B1(t) through Bn(t).

[0053] In addition, the signal propagation delay parameters a(t), b(t), and c(t) at each frequency f(t) are assumed to be measured at the time of shipment from the factory. The signal propagation delay parameters a(t), b(t), and c(t) measured in advance are also assumed to be stored as parameter information 9a in the memory unit 9. In addition, in Chapter 4 of Non-Patent Document 1, a(t) is written as b0, b(t) as b1, and c(t) as b2.

[0054] The process shown in FIG. 3 will now be described. First, in step S101, the calibration calculation unit 8a sets h=1, and then proceeds to step S102. In step S102, the calibration calculation unit 8a determines whether h is equal to or less than H. If h is equal to or less than H, the process proceeds to step S103.

[0055] Here, H is the number of measurements (a predetermined value) for calibration, for example, H=40. Moreover, since measurements are performed at intervals of a predetermined time k, the calibration takes time H×k. Different target objects (different distances) are measured H times.

[0056] In step S103, the calibration calculation unit 8a sets t = 1. Then, the process proceeds to step S104. In step S104, the calibration calculation unit 8a determines whether t is equal to or less than T. If t is equal to or less than T, the process proceeds to step S105.

[0057] In step S105, the calibration calculation unit 8a controls the execution of light emission and reception at frequency f(t). That is, the light emission unit 2 emits irradiated light Ls at frequency f(t) and the sensor unit 3 receives reflected light Lr. In step S106 following step S105, the calibration calculation unit 8a causes the phase difference detection unit 5 to detect the phase difference at each pixel position (u, v) and obtains each phase difference as p(h, t, u, v). Then, the process proceeds to step S107.

[0058] In step S107, the calibration calculation unit 8a increments t by 1 to obtain data for the next frequency f, and then returns to step S104.

[0059] In step S104, if it is determined that t is not equal to or less than T, that is, if the phase difference p(h, t, u, v) is obtained in step S106 for each emission frequency from t=1 to T, the calibration calculation unit 8a proceeds to step S108. In step S108, the calibration calculation unit 8a performs a discard process of the phase difference p(h,t,u,v) based on the magnitude of the amplitude. Specifically, if any one of the "amplitudes (T pieces from t=1 to T) of the light receiving signal at frequency f(t)" at each pixel position (u,v) is less than a predetermined value, the phase difference p(h,t,u,v) (a total of T pieces from t=1 to T) for that (h,u,v) is discarded. In other words, in step S108, if the amplitudes at all frequencies f(t) (t=1 to T) of all pixel positions (u,v) are equal to or greater than a predetermined value, no processing is performed. A small amplitude means that there is little light reflected from the object Ob, and so the measurement data is less reliable. Therefore, such data is discarded. Here, if data is discarded in step S108, all measurements of the h-th phase difference p(h,t,u,v) become invalid, and therefore in this example, if data is discarded, h=h-1 is set.

[0060] In step S109 following step S108, the calibration calculation unit 8a waits for a predetermined time k to perform the next measurement (h+1th measurement), and then in step S110, increments h by 1 and returns to the previous step S102. This results in H measurements of the phase difference p(h,t,u,v) for each of the T emission frequencies.

[0061] If it is determined in step S102 that h is not equal to or less than H, the calibration calculation unit 8a proceeds to step S111 and performs 2π ambiguity resolution processing. Specifically, in step S111, processing is performed to resolve the 2π ambiguity of the phase difference p(h,t,u,v) for each h, each t, ​​and each (u,v). The phase difference after the 2π ambiguity is resolved is defined as θ(h,t,u,v). The 2π ambiguity resolution process in step S111 will be described in detail later (see FIG. 4).

[0062] In step S112 following step S111, the calibration calculation unit 8a obtains parameters of the circular error (parameters A1(t) to An(t) and B1(t) to Bn(t)) that satisfy [Equation 3] described later. The obtained parameters are stored in the memory unit 9 as parameter information 9a (the values ​​of parameters A1(t) to An(t) and B1(t) to Bn(t) are overwritten).

[0063] The calibration calculation unit 8a ends the series of processes shown in FIG. 3 in response to the execution of the process of step S112.

[0064] Here, a supplementary explanation will be given regarding the calculation process in step S112. The following [Equation 2] expresses the relationship between the phase difference θ (h,t,u,v) at pixel position (u,v) in the h-th measurement and the distance L(h,u,v) to the point of the target (distance measurement point) projected onto pixel position (u,v), where t ranges from 1 to T.

number

[0065] From the analogy of [Equation 1], it is clear that [Equation 2] holds. The thing to note about [Equation 2] is that the distance L(h, u, v) does not depend on t. Even if the measurement is performed by changing the frequency f(t), the distance to the target object Ob naturally does not change, so L(h, u, v) does not depend on t. Note that since the distance to the target object Ob is unknown, L(h, u, v) is an unknown quantity.

[0066] In addition, in [Equation 2], the parameters a(t), b(t), and c(t) of the signal propagation delay at each frequency f(t) can be known by reading out the parameters stored as parameter information 9a, where t ranges from 1 to T. In this example, the parameters An and Bn are obtained by calibration, but the factory default values ​​of the signal propagation delay parameters a(t), b(t), and c(t) are assumed to continue to be used.

[0067] Therefore, the parameters A1(t) to An(t) and B1(t) to Bn(t) that satisfy [Equation 2] can be found by using the data other than (h, u, v) discarded in step S108. In practice, the least squares method is used to find these parameters. Specifically, the parameters A1(t) to An(t), B1(t) to Bn(t), and L(h, u, v) that minimize [Equation 3] can be found.

number

[0068] Here, we would like to add a supplementary explanation for the effectiveness of this method. [Equation 3] is established for each (h, t, u, v) from h=1 to H, t=1 to T, u=1 to U, and v=1 to V. That is, at the time of proceeding to step S112, H×T×U×V equations have been obtained. Meanwhile, the unknown parameters are An(t) (n=1 to N, t=1 to T), Bn(t) (n=1 to N, t=1 to T), and L(h, u, v) (h=1 to H, u=1 to U, v=1 to V), totaling (2×N×T)+(H×U×V). Therefore, if (2×N×T)+(H×U×V)≦H×T×U×V, the number of equations is greater than the number of unknowns, and the equations can be solved. In fact, by increasing T, that is, by increasing the number of light emission frequencies of the light-emitting unit 2, it is possible to obtain (2×N×T)+(H×U×V)≦H×T×U×V. Alternatively, by increasing H, i.e., measuring the phase difference for various scenes, it is possible to make (2×N×T)+(H×U×V)≦H×T×U×V.

[0069] That is, this embodiment utilizes the fact that it is possible to satisfy (2×N×T)+(H×U×V)≦H×T×U×V if T is 2 or more. In other words, the feature of this embodiment is that "the phase difference is measured using a plurality of emission frequencies (at least two different emission frequencies) for the same object." This makes it possible to obtain unknown parameters An(t) (n=1 to N, t=1 to T), Bn(t) (n=1 to N, t=1 to T), and L(h, u, v) (h=1 to H, u=1 to U, and v=1 to V) even if the distance to the object is unknown. That is, it is possible to obtain circular error (An(t) (n=1 to N, t=1 to T), Bn(t) (n=1 to N, t=1 to T)).

[0070] In addition, in step S108 of FIG. 3, a process of setting h=h-1 is executed when discarding is performed, but if T and H are determined so that H×T×U×V is sufficiently larger than (2×N×T)+(H×U×V) (so that there is a margin in the number of measurement data), the process of setting h=h-1 can be omitted.

[0071] FIG. 4 is a flowchart showing the 2π ambiguity resolution process in step S111. 2, there is a 2π uncertainty in the phase difference p(h,t,u,v) measured for each pixel of the sensor unit 3. In other words, for each (h,t,u,v), it is unclear which of the following [Equation 4] the true phase difference θ(h,t,u,v) is.

number

[0072] Here, as the distance to the target object Ob increases, the amount of light reflected from the light emitting unit 2 by the target object Ob and reaching the sensor unit 3 also decreases. That is, the amplitude of the received light signal becomes small. Also, since the data with small amplitude is discarded in step S108, the distance to the target object Ob corresponding to (h, t, u, v) targeted in step S11 is not so far. Therefore, it can be said that the distance to the target object Ob corresponding to (h, t, u, v) targeted in step S111 satisfies [Equation 5].

number

[0073] As mentioned above, f(1) in [Equation 5] is the lowest frequency among the frequencies f(t) from t=1 to T.

[0074] Therefore, for t=1, the true phase difference θ(h,t,u,v) is s(h,t,u,v)=0 in [Equation 4], and can be determined from the phase difference p(h,t,u,v) measured for each pixel of the sensor unit 3 by the following [Equation 6].

number

[0075] Furthermore, although the irradiated light Ls emitted by the light-emitting unit 2 is not a perfect sine wave, it has a waveform similar to a sine wave and therefore has a small amount of circular error.

number

[0076] In [Equation 7], when t = 1, it is determined that s(h, t, u, v) = 0. Therefore, the following [Equation 8] holds.

number

[0077] By transforming [Equation 8], we obtain the following [Equation 9].

number

[0078] For t=2 to T, s(h,t,u,v) can be determined from [Equation 9]. In other words, the integer closest to the following [Equation 10] should be set as s(h,t,u,v).

number

[0079] For t=2 to T, if s(h,t,u,v) is determined, the true phase difference θ(h,t,u,v) can also be determined from the above [Equation 4].

[0080] Taking the above into consideration, the processing in FIG. 4 will be explained. First, in step S1111, the calibration calculation unit 8a sets θ(h,1,u,v) at frequency f(1) to p(h,1,u,v). That is, θ(h,1,u,v)=p(h,1,u,v). As described above, the frequency f(1) at t=1 is a lower frequency than the other frequencies (f(2) to f(T)).

[0081] In step S1112 following step S1111, the calibration calculation unit 8a finds the integer closest to the value of [Equation 10] for each t from t=2 to T, and sets the found integer as s(h, t, u, v).

[0082] Furthermore, in step S1113 following step S1112, the calibration calculation unit 8a calculates [Equation 4] for each t from t=2 to T to obtain a true phase difference θ(h, t, u, v). In response to executing the process of step S1113, the calibration calculation unit 8a ends the 2π ambiguity resolution process of step S111.

[0083] Here, the ambiguity resolution process described above can be rephrased as follows. In other words, among the phase differences detected from the light receiving signal when light is emitted at the minimum light emission frequency (frequency f(1)), which is the lowest light emission frequency among the light emission frequencies for the calibration calculation process, the phase difference detected from the light receiving signal whose amplitude is equal to or greater than a predetermined value is determined as the phase difference corresponding to the minimum light emission frequency, and processing is performed to eliminate 2π ambiguity for the phase differences corresponding to light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency.

[0084] <2. Second embodiment> Next, a second embodiment will be described. In the second embodiment, calibration is performed to obtain correction parameters in the background. In the second embodiment, the hardware configuration of the distance measuring device 1 is the same as that of the first embodiment, and therefore is not illustrated. In the following description, the same reference numerals are used to designate the same parts as those already described, and the description thereof will be omitted.

[0085] FIG. 5 is a flowchart of the process executed by the control unit 8 in the second embodiment. The process shown in FIG. 5 is started in response to the establishment of a predetermined trigger condition, such as when the power of the distance measuring device 1 is turned on or when an application for distance measurement is started.

[0086] In this case, in step S201, the control unit 8 determines whether a predetermined time (e.g., one year) has passed since the previous calibration. If the predetermined time has passed, there is a possibility that aging has occurred. Therefore, when the control unit 8 determines in step S201 that the predetermined time has passed, it executes the process as the calibration calculation unit 8a shown in FIG. 6. On the other hand, if the predetermined time has not elapsed, it is assumed that no deterioration with age has occurred, and the process shown in FIG. 6 is not executed.

[0087] If it is determined that the predetermined time has not elapsed, the control unit 8 proceeds to step S202 and performs a process of waiting for a distance measurement instruction from the user via, for example, the operation unit 11. If a distance measurement instruction is received, the control unit 8 proceeds to step S203 and performs distance measurement processing. That is, the control unit 8 causes the light-emitting unit 2 to emit the irradiation light Ls and the sensor unit 3 to receive the reflected light Lr, and also causes the phase difference detection unit 5 to detect the phase difference and the calculation unit 6 to calculate the distance. After the distance measurement process in step S203 is executed, the control unit 8 returns to step S202.

[0088] In the second embodiment, the calibration is performed by the process shown in FIG. 6 between distance measurement processes performed in accordance with a distance measurement instruction from the user. The process shown in FIG. 6 differs from that shown in FIG. 3 in that steps S204 and S205 are inserted between steps S108 and S109. In this case, the control unit 8 (calibration calculation unit 8a) proceeds to step S204 in response to executing the discarding process of step S108 to determine whether a distance measurement instruction has been issued. If there is no distance measurement instruction, the control unit 8 proceeds to step S109. In other words, if there is no distance measurement instruction, the same process as in FIG. 3 (the flow of proceeding to step S109 after the process of step S108) is performed.

[0089] If a distance measurement instruction has been issued, the control unit 8 proceeds to step S205 to execute distance measurement processing (the processing is the same as in step S203), and then proceeds to step S109.

[0090] The process flow in FIG. 6 is basically the same as that in FIG. 3, except that if a distance measurement instruction is received between steps S108 and S109 in FIG. 3, the calibration process is temporarily interrupted and distance measurement is performed (step S205).

[0091] In this case, in response to having executed the process of step S112, the control unit 8 advances the process to step S202 in FIG.

[0092] As described above, in the second embodiment, while the user is using the distance measuring device 1, calibration for obtaining correction parameters can be performed in the background.

[0093] <3. Third embodiment> In the third embodiment, calibration is performed on the condition that the distance measurement points are in a specific positional relationship with each other. FIG. 7 is a block diagram for explaining an example of the internal configuration of a distance measuring device 1A according to the third embodiment. The difference from the distance measuring device 1 is that a control unit 8A is provided instead of the control unit 8. The control unit 8A has the same hardware configuration as the control unit 8, but is different in that it performs calculation processing by a method different from that of the first embodiment as the calibration calculation processing. Here, the function that performs the calibration calculation processing by the method of the third embodiment described below is denoted as a calibration calculation unit 8aA.

[0094] In the third embodiment, calibration is performed by obliquely photographing (measuring the phase difference) a flat plate 20 whose distance is unknown, as illustrated in Fig. 8. Since the distance to the flat plate 20 does not need to be known, no precision device is required. FIG. 8 shows a schematic diagram of a state in which a part of the flat plate 20 is projected onto the distance measuring device 1A.

[0095] Here, in the third embodiment as well, the number of pixels in the sensor unit 3 is U×V, and each pixel position is (u, v) (u=1 to U, v=1 to V). In this example, the area (u,v) (u=U0 to U0+U1, v=V0 to V0+V1) is called the planar imaging area Ar. For example, U0=U / 4, V0=V / 3, U1=U / 2, V1=V / 3. These positional relationships are shown in Figure 9.

[0096] During calibration, the user photographs the same plane of the plate 20 in the plane photographing area Ar of the sensor unit 3 . In this example, the control unit 8A causes the display unit 10 to display a guide image to guide the user in taking a photograph so that the same plane on the flat plate 20 is captured within the planar shooting area Ar (i.e., to guide the shooting composition).

[0097] FIG. 10 is a diagram for explaining an example of guide display during calibration, including the display of such a guide image. First, the calibration inquiry screen shown in Fig. 10A is displayed. On this calibration inquiry screen, a "Yes" button B1 and a "No" button B2 are displayed together with an inquiry message such as "Do you want to perform calibration?" asking whether or not to perform calibration. When the user wishes to instruct execution of calibration, he or she operates the "Yes" button B1.

[0098] When the "Yes" button B1 is operated, a frame screen shown in Fig. 10B is displayed. On the frame screen, a frame W indicating the size of the planar imaging area Ar described above is displayed, along with a message such as "Please fit the same surface of the plate within the frame" encouraging the user to fit the same plane of the plate 20 within the frame W, and a "Shoot" button B3 for instructing the user to start measuring the phase difference for calibration.

[0099] In this example, similarly to the first embodiment, H measurements are taken at different distances during calibration. When the "shoot" button B3 is operated to perform the first measurement on the frame screen shown in Fig. 10B, the frame screen shown in Fig. 10C is displayed on the display unit 10. The difference with the frame screen in FIG. 10B is that a message such as "Please change your position and take a picture" is displayed to encourage taking a picture at a different distance.

[0100] When H measurements have been performed and the calibration calculation process is complete, a calibration completion screen shown in Fig. 10D is displayed. As shown in the figure, the calibration completion screen displays a message such as "Calibration is complete" to notify that the calibration calculation process is complete.

[0101] 10B and 10C, an image (e.g., a distance image) obtained by the light receiving operation of the sensor unit 3 is displayed in real time. This makes it easy for the user to adjust the composition to an appropriate one while looking at the screen of the display unit 10.

[0102] The object used during calibration is not limited to the flat plate 20. For example, it may be a wall of a user's house or an outer wall of a building.

[0103] FIG. 11 is a flowchart showing a process flow when performing calibration according to the third embodiment. The process shown in FIG. 11 is started in response to the establishment of a predetermined trigger condition, such as when the power of the distance measuring device 1A is turned on or when an application for distance measurement is started.

[0104] First, in step S301, the control unit 8A performs processing for displaying a calibration inquiry screen such as that shown in FIG. 10A on the display unit 10 as a display processing for the calibration inquiry screen.

[0105] In step S302 following step S301, the control unit 8A waits until the above-mentioned "YES" button B1 is operated, and if the "YES" button B1 is operated, the process proceeds to step S303 and performs the display process of the frame screen exemplified in FIG. 10B. When the "No" button B2 is operated on the calibration inquiry screen, a process of transitioning to a predetermined screen such as a distance measurement screen may be performed.

[0106] In step S304 following step S303, the control unit 8A waits until the "shoot" button B3 on the frame screen is operated, and if the "shoot" button B3 is operated, executes a calibration process in step S305 and proceeds to step S306. The calibration process in step S305 is performed under the condition that the distance measurement points are in a specific positional relationship with each other, and details of this will be described later.

[0107] In step S306, the control unit 8A executes a process for displaying a calibration completion screen exemplified in FIG. 10D, and ends the series of processes shown in FIG.

[0108] FIG. 12 is a flowchart of the calibration process in step S305. As shown in the figure, the calibration process shown in FIG. 12 differs from the calibration process previously described in FIG. 3 in that the standby process (time k) of step S109 is omitted, that the process of step S310 (shooting button standby process) is executed in response to execution of the process of step S108, and that the process of step S311 is executed instead of the process of step S112.

[0109] First, in the determination process of step S102, H is set to, for example, 40 in this case as well. In the third embodiment, the phase difference is measured H times with different compositions (that is, the user moves the distance measuring device 1A). That is, in the third embodiment, it is assumed that a plane at a different distance is measured each time the value of h is incremented.

[0110] In this example, the calibration calculation process uses a condition that the distance measurement points are in a specific positional relationship with each other, and employs a method using multiple emission frequencies as in the first embodiment. Therefore, in the third embodiment as well, the phase difference is measured for multiple ts from t=1 to T.

[0111] In the process of FIG. 12, in response to executing the discarding process in step S108, the control unit 8A proceeds to step S310 and waits until the "shoot" button B3 is operated. Although not illustrated, in the third embodiment, the control unit 8A performs a process of updating the frame screen to the frame screen illustrated in Fig. 10C after the "Shoot" button B3 on the frame screen illustrated in Fig. 10B is operated, until the discard process of step S108 is executed for the first time. Therefore, the "Shoot" button B3 waiting for operation in step S310 is the "Shoot" button B3 on the frame screen illustrated in Fig. 10C.

[0112] If it is determined in step S310 that the "shoot" button B3 has been operated, the control unit 8A advances the process to step S110.

[0113] In the third embodiment, the target for detecting the phase difference in the process of step S106 is the range of u=U0 to U0+U1 and v=V0 to V0+V1 among the pixel positions (u, v) of the sensor unit 3. This allows the phase difference detected for each ranging point on the same plane to be used in the calculation process of the correction parameters.

[0114] The processing in step S311 is basically the same as step S112 shown in FIG. 3, in that it is a process of finding the parameters of the circular error (parameters A1(t) to An(t) and B1(t) to Bn(t)) that satisfy [Equation 3]. After executing the process of step S311, the control unit 8A ends the calibration process of step S305.

[0115] Regarding the process of step S311, in the third embodiment, there is a certain condition when solving [Equation 3].

[0116] The calculation in step S311 will be described in detail below. First, the direction in which the pixel position (u, v) is photographed is (d x (u,v),d y (u,v),d z For example, let lens 4 have no distortion and its focal length be F L Then, the direction in which the pixel position (u, v) is photographed is given by the following [Equation 11].

number

[0117] The pixel position (u,v) corresponds to the shooting direction (d x (u,v),d y (u,v),d z (u,v) is determined by the characteristics of the lens 4. For example, the characteristics are determined when the lens 4 is designed, and therefore can be considered known.

[0118] In addition, the three-dimensional vector (d x (u,v),d y (u,v),d z (u,v) is assumed to be normalized, i.e., to satisfy the following [Equation 12].

number

[0119] If the distance to the point on plate 20 projected onto pixel position (u, v) when plate 20 is photographed for the hth time is L(h, u, v), then the position in three-dimensional space of the point on plate 20 projected onto pixel position (u, v) is given by [Equation 13].

number

[0120] Consider the position of the flat plate 20 in three-dimensional space at the h-th time. In this example, the positions of the object in three-dimensional space projected onto all pixel positions (u, v) (u=U0 to U0+U1, v=V0 to V0+V1) are on a certain plane. In other words, on the plane that passes through the positions of the object in three-dimensional space projected onto the three pixel positions (U0, V0), (U0+1, V0), and (U0, V0+1), there is also the position of the object in three-dimensional space projected onto the pixel position of another position (u, v). Therefore, the following [Equation 14] is satisfied.

number

[0121] To summarize the above, the pixel position (u, v) corresponds to the shooting direction (d x (u,v),d y (u,v),d z (u,v) are known. In the h-th shooting (measurement of phase difference), the same plane is shot within the plane shooting area Ar, so [Equation 14] is satisfied for pixels from u=U0 to U0+U1 and v=V0 to V0+V1. Note that L(h,u,v) in [Equation 14] is the distance to the plate 20 projected at the pixel position (u,v) when the plate 20 is shot the h-th time.

[0122] Now, [Equation 2] expresses the relationship between the phase difference θ(h,t,u,v) at the pixel position (u,v) in the h-th measurement and the distance L(h,u,v) to the range-finding point projected on the pixel position (u,v), where t ranges from 1 to T.

[0123] As mentioned above, it is clear that [Equation 2] holds true from the analogy of [Equation 1]. Since the distance to the target object Ob is unknown, L(h, u, v) is an unknown quantity. However, L(h, u, v) satisfies [Equation 14] as described above.

[0124] Therefore, it is sufficient to find parameters A1(t) to An(t) and B1(t) to Bn(t) that satisfy [Equation 2] under the condition that [Equation 14] is satisfied. In reality, the least squares method is also used in this case, and therefore it is sufficient to find parameters A1(t) to An(t), B1(t) to Bn(t), and L(h, u, v) that minimize [Equation 3] given above under the condition that [Equation 14] is satisfied.

[0125] That is, the calculation in step S311 is to find A1(t) to An(t), B1(t) to Bn(t), and L(h, u, v) that minimize [Equation 3] for each (u, v) (where u = U0 to U0 + U1, and v = V0 to V0 + V1) under the condition that [Equation 14] is satisfied. Then, the found parameters A1(t) to An(t) and B1(t) to Bn(t) are set as the parameters of the circular error.

[0126] Here, we will add that the calibration method of the third embodiment (a method that uses the condition that the distance measurement points are in a specific positional relationship with each other) is effective. In step S311, "a solution that satisfies the equations shown in [Equation 2] and [Equation 14] is found." [Equation 2] holds for each (h, t, u, v) from h=1 to H, t=1 to T, u=U0 to U0+U1, and v=V0 to V0+V1. In other words, when the process proceeds to step S311, H×T×U1×V1 equations have been obtained.

[0127] Furthermore, [Equation 14] holds for each (h, u, v) from h=1 to H, u=U0 to U0+U1, and v=V0 to V0+V1, except for the pairs (u, v)=(U0, V0), (u, v)=(U0+1, V0), and (u, v)=(U0, V0+1). In other words, H×(U1×V1-3) ​​equations are obtained.

[0128] Therefore, when the process proceeds to step S311, (H×T×U1×V1)+(H×(U1×V1-3)) equations have been obtained.

[0129] On the other hand, the unknown parameters are An(t) (n=1 to N, t=1 to T), Bn(t) (n=1 to N, t=1 to T), and L(h,u,v) (h=1 to H, u=U0 to U0+U1, v=V0 to V0+V1), totaling (2×N×T)+(H×U1×V1). Therefore, if (2×N×T)+(H×U1×V1)≦(H×T×U1×V1)+(H×(U1×V1-3)), the number of equations will be greater than the number of unknowns, and they can be solved. In fact, if at least one of U1, V1, or H is sufficiently large, it can be determined that (2×N×T)+(H×U1×V1)≦(H×T×U1×V1)+(H×(U1×V1-3)). Note that the above inequality can be satisfied even when T = 1. That is, in the first embodiment, T needs to be a natural number of 2 or more, but in the third embodiment, T may be any natural number of 1 or more.

[0130] In addition, in the process of FIG. 12, the 2π ambiguity resolution process in step S111 is the same as that explained in FIG. 4, so a duplicate explanation will be avoided.

[0131] <4. Modifications> It should be noted that the embodiment is not limited to the specific example described above, and various modifications are possible. For example, in the above example, an example was given in which the distance measuring device according to the present technology is applied to a portable information processing device such as a smartphone, but the distance measuring device according to the present technology is not limited to application to portable information processing devices, and can be widely and suitably applied to various electronic devices.

[0132] In addition, in the process of Fig. 3 described in the first embodiment and the process of Fig. 12 described in the third embodiment, it is desirable that the positional relationship with the target object Ob has changed when performing the h+1th measurement from the h-th measurement. Therefore, for example, in the process of Fig. 3, between steps S109 and S110, a process for determining whether the distance measuring device 1 is moving based on a detection signal of an acceleration sensor or an angular velocity sensor built into the distance measuring device 1 can be provided. In this case, if the distance measuring device 1 is moving, the process proceeds to step S110, and if not, the determination process is performed again. This ensures that the h+1th measurement can be performed on an object at a different distance from the h-th measurement. As for the processing in FIG. 12, for example, a process for determining whether the distance measuring device 1A is moving could be provided between steps S310 and S110, and if the distance measuring device 1A is moving, the process could proceed to step S110, and if not, the determination process could be performed again.

[0133] <5. Summary of the embodiment> As described above, the first distance measuring device (1) as an embodiment includes an emission unit (2) that emits light, a light receiving sensor (sensor unit 3) that receives light emitted by the emission unit and reflected by a target object, and a calibration calculation unit (8a) that performs a calculation process using the light receiving signal of the light receiving sensor when the light emission unit emits light at a first emission frequency and the light receiving signal of the light receiving sensor when the light emission unit emits light at a second emission frequency different from the first emission frequency, as a calibration calculation process to obtain a correction parameter for distance information calculated by the indirect ToF method based on the light receiving signal of the light receiving sensor. By using a plurality of emission frequencies, it becomes possible to obtain the correction parameters even if the distance to the target object is not fixed. Therefore, the prerequisites for establishing the calibration can be relaxed, and the calibration can be performed even in the actual environment in which the device is used. By being able to perform calibration even in an actual usage environment, it becomes possible to absorb changes in correction parameters due to aging, thereby preventing deterioration of distance measurement accuracy over time.

[0134] In the first distance measuring device as the embodiment, the calibration calculation unit obtains the correction parameters by performing calculation processing based on the phase difference between the emitted light and the received light detected based on the light reception signal. This makes it possible to obtain appropriate correction parameters when performing distance measurement using the indirect ToF method as the phase difference method.

[0135] Furthermore, in the first distance measuring device as the embodiment, the calibration calculation unit performs an ambiguity resolution process for resolving the ambiguity of the phase difference in units of 2π (see step S111). This makes it possible to perform calculation processing of the correction parameters using a phase difference that eliminates the uncertainty in 2π units. Therefore, the accuracy of the correction parameters can be improved, and the distance measurement accuracy can be improved.

[0136] Furthermore, in the first distance measuring device as an embodiment, the calibration calculation unit determines, among the phase differences detected from the light receiving signal when light is emitted at the minimum light emission frequency, which is the lowest light emission frequency among the light emission frequencies of the light emitting unit for the calibration calculation process, the phase difference detected from the light receiving signal whose amplitude is equal to or greater than a predetermined value as the phase difference corresponding to the minimum light emission frequency, and performs processing to eliminate uncertainty regarding the phase differences corresponding to light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency. For the phase difference corresponding to the minimum emission frequency, by selecting the phase difference detected from the received light signal whose amplitude is equal to or greater than a predetermined value as described above, it is possible to eliminate the ambiguity in 2π units, and for the phase differences corresponding to emission frequencies other than the minimum emission frequency, it is possible to identify the true phase difference based on the phase difference corresponding to the minimum emission frequency from which the ambiguity has been eliminated in this manner (i.e., it is possible to eliminate the ambiguity in 2π units). Therefore, the correction parameters can be calculated based on the phase difference from which the uncertainty in units of 2π has been eliminated, and the accuracy of the correction parameters can be improved, thereby improving the accuracy of distance measurement.

[0137] In the first distance measuring device as the embodiment, the calibration calculation unit executes the calibration calculation process based on the elapsed time since the previous execution (see step S201). This makes it possible to redo the calibration of the correction parameters even if the correction parameters deviate from their true values ​​over time. Therefore, it is possible to prevent the distance measurement accuracy from decreasing over time due to changes in the correction parameters over time.

[0138] Furthermore, in the first distance measuring device as an embodiment, when an instruction for distance measurement is given during execution of the calibration calculation process, the calibration calculation unit interrupts the calibration calculation process and performs processing for distance measurement (see FIG. 6). As a result, even if the calibration calculation process is performed in the background, when an instruction for distance measurement is received, the calibration calculation process is interrupted and a distance measurement operation is performed in response to the instruction. Therefore, usability can be improved.

[0139] Furthermore, a first calibration method as an embodiment is a calibration method for a distance measuring device that has an light-emitting unit that emits light and a light-receiving sensor that receives light emitted from the light-emitting unit and reflected by a target object, and performs distance measurement using an indirect ToF method based on a light-receiving signal of the light-receiving sensor, and is a calibration method that performs a calculation process using the light-receiving signal of the light-receiving sensor when the light-emitting unit emits light at a first light-emitting frequency and the light-receiving signal of the light-receiving sensor when the light-emitting unit emits light at a second light-emitting frequency different from the first light-emitting frequency, as a calibration calculation process for obtaining a correction parameter for distance information calculated by the indirect ToF method. This first calibration method also provides the same effects and advantages as the first distance measuring device described above.

[0140] The second distance measuring device (1A) as an embodiment includes an emission unit (2) that emits light, a light receiving sensor (sensor unit 3) that receives the light emitted from the emission unit and reflected by a target object at multiple pixels, and a calibration calculation unit (8aA) that performs a calculation process using the condition that the respective distance measuring points projected onto the multiple pixels are in a specific positional relationship with each other as a calibration calculation process to obtain correction parameters for distance information calculated by the indirect ToF method based on the light receiving signal of the light receiving sensor. By using the condition that the distance measurement points have a specific positional relationship as described above, it is possible to find the correction parameters even if the distance to the target object is indefinite. Therefore, the prerequisites for establishing the calibration can be relaxed, and the calibration can be performed even in the actual environment in which the device is used. By being able to perform calibration even in an actual usage environment, it becomes possible to absorb changes in correction parameters due to aging, thereby preventing deterioration of distance measurement accuracy over time.

[0141] In the second distance measuring device according to the embodiment, the calibration calculation unit performs the calibration calculation process under the condition that the distance measuring points are located on an object having a known shape. If the distance measurement points are located on an object having a known shape, it is possible to define the positional relationship between the distance measurement points as a mathematical formula based on the known shape. Therefore, the prerequisites for establishing the calibration can be relaxed, and the calibration can be performed even in the actual environment in which the device is used. Furthermore, by making it possible to perform calibration in an actual usage environment, it becomes possible to absorb changes in correction parameters due to aging, thereby preventing deterioration of distance measurement accuracy over time.

[0142] Furthermore, in the second distance measuring device of the embodiment, the calibration calculation unit performs a calibration calculation process using the light receiving signal of the light receiving sensor when the light emitting unit emits light at a first light emitting frequency and the light receiving signal of the light receiving sensor when the light emitting unit emits light at a second light emitting frequency different from the first light emitting frequency (see Figure 12). In other words, the calibration calculation process involves performing calculations using multiple light emission frequencies while using the condition that each ranging point is in a specific positional relationship with respect to the others, thereby making it possible to increase the number of equations for the unknowns. Therefore, the correction parameters can be obtained more robustly, and the distance measurement accuracy can be improved.

[0143] Furthermore, in the second distance measuring device as the embodiment, the calibration calculation section obtains the correction parameters by performing calculation processing based on the phase difference between the emitted light and the received light detected based on the light reception signal. This makes it possible to obtain appropriate correction parameters when performing distance measurement using the indirect ToF method as the phase difference method.

[0144] In the second distance measuring device according to the embodiment, the calibration calculation unit performs an ambiguity resolution process for resolving the ambiguity of the phase difference in units of 2π. This makes it possible to perform calculation processing of the correction parameters using a phase difference that eliminates the uncertainty in 2π units. Therefore, the accuracy of the correction parameters can be improved, and the distance measurement accuracy can be improved.

[0145] Furthermore, in the second distance measuring device as an embodiment, the calibration calculation unit determines, among the phase differences detected from the received light signals when light is emitted at a minimum emission frequency, which is the lowest emission frequency among the emission frequencies of the light emitting unit for the calibration calculation process, the phase difference detected from the received light signal whose amplitude is equal to or greater than a predetermined value as the phase difference corresponding to the minimum emission frequency, and performs processing to eliminate uncertainty regarding the phase differences corresponding to emission frequencies other than the minimum emission frequency based on the phase difference corresponding to the determined minimum emission frequency. For the phase difference corresponding to the minimum emission frequency, by selecting the phase difference detected from the received light signal whose amplitude is equal to or greater than a predetermined value as described above, it is possible to eliminate the ambiguity in 2π units, and for the phase differences corresponding to emission frequencies other than the minimum emission frequency, it is possible to identify the true phase difference based on the phase difference corresponding to the minimum emission frequency from which the ambiguity has been eliminated in this manner (i.e., it is possible to eliminate the ambiguity in 2π units). Therefore, the correction parameters can be calculated based on the phase difference from which the uncertainty in units of 2π has been eliminated, and the accuracy of the correction parameters can be improved, thereby improving the accuracy of distance measurement.

[0146] Furthermore, in the second distance measuring device of the embodiment, a guide display processing unit is provided that performs display processing of a guide image that guides the composition to satisfy the condition that the distance measuring points are in a specific positional relationship (control unit 8A, see Figures 10 and 11). This makes it possible to increase the possibility that the correction parameters will be calibrated under the condition that the distance measurement points are in a specific positional relationship with each other. Therefore, the accuracy of the correction parameters can be improved, and the distance measurement accuracy can be improved.

[0147] In addition, a second calibration method as an embodiment is a calibration method in a distance measuring device that includes an light-emitting unit that emits light, a light-receiving sensor that receives the light emitted from the light-emitting unit and reflected by a target object at multiple pixels, and performs distance measurement using an indirect ToF method based on the light-receiving signal of the light-receiving sensor, in which a calculation process is performed using the condition that the respective distance measurement points projected onto the multiple pixels are in a specific positional relationship with each other as a calibration calculation process for obtaining correction parameters for the distance information calculated by the indirect ToF method. This second calibration method also provides the same effects and advantages as the second distance measuring device described above.

[0148] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0149] <6. This Technology> The present technology can also be configured as follows. (1) A light emitting unit that emits light; a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object; a calibration calculation unit that performs a calculation process using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency, as a calibration calculation process for obtaining a correction parameter for distance information calculated by an indirect ToF method based on a light receiving signal of the light receiving sensor. Ranging device. (2) The calibration calculation unit The correction parameter is obtained by performing a calculation process based on the phase difference between the emitted light and the received light detected based on the received light signal. A distance measuring device as described in (1) above. (3) The calibration calculation unit The phase difference is subjected to an ambiguity resolution process for resolving the ambiguity in units of 2π. A distance measuring device as described in (2) above. (4) The calibration calculation unit Among the phase differences detected from the light reception signal when light is emitted at a minimum light emission frequency that is the lowest light emission frequency among the light emission frequencies of the light-emitting unit for the calibration calculation process, the phase difference detected from the light reception signal having an amplitude equal to or greater than a predetermined value is determined as a phase difference corresponding to the minimum light emission frequency, and a process is performed to eliminate the uncertainty regarding the phase difference corresponding to the light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency. A distance measuring device as described in (3) above. (5) The calibration calculation unit The calibration calculation process is performed based on the elapsed time since the previous execution. A distance measuring device according to any one of (1) to (4). (6) The calibration calculation unit When an instruction for distance measurement is given during the execution of the calibration calculation process, the calibration calculation process is interrupted and a process for distance measurement is performed. A distance measuring device according to any one of (1) to (5) above. (7) A calibration method for a distance measuring device that includes a light emitting unit that emits light and a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object, and performs distance measurement by an indirect ToF method based on a light receiving signal of the light receiving sensor, comprising: As a calibration calculation process for obtaining a correction parameter for distance information calculated by the indirect ToF method, a calculation process is performed using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency. Calibration method. (8) A light emitting unit that emits light; a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object at a plurality of pixels; a calibration calculation unit that performs a calculation process using a condition that each of the distance measurement points projected onto the plurality of pixels has a specific positional relationship as a calibration calculation process for obtaining a correction parameter for distance information calculated by an indirect ToF method based on a light receiving signal of the light receiving sensor; Ranging device. (9) The calibration calculation unit As the calibration calculation process, a calculation process is performed under the condition that the distance measurement points are located on an object of known shape. The distance measuring device according to (8) above. (10) The calibration calculation unit As the calibration calculation process, a calculation process is performed using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency. A distance measuring device according to (8) or (9). (11) The calibration calculation unit The correction parameter is obtained by performing a calculation process based on the phase difference between the emitted light and the received light detected based on the received light signal. A distance measuring device according to any one of (8) to (10) above. (12) The calibration calculation unit The phase difference is subjected to an ambiguity resolution process for resolving the ambiguity in units of 2π. The distance measuring device according to (11) above. (13) The calibration calculation unit Among the phase differences detected from the light reception signal when light is emitted at a minimum light emission frequency that is the lowest light emission frequency among the light emission frequencies of the light-emitting unit for the calibration calculation process, the phase difference detected from the light reception signal having an amplitude equal to or greater than a predetermined value is determined as a phase difference corresponding to the minimum light emission frequency, and a process is performed to eliminate the uncertainty regarding the phase difference corresponding to the light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency. The distance measuring device according to (12) above. (14) a guide display processing unit that performs processing for displaying a guide image that guides a composition to satisfy the condition that the distance measuring points are in a specific positional relationship; A distance measuring device according to any one of (8) to (13) above. (15) A calibration method for a distance measuring device that includes a light emitting unit that emits light, a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object at a plurality of pixels, and performs distance measurement by an indirect ToF method based on a light receiving signal of the light receiving sensor, comprising: As a calibration calculation process for obtaining a correction parameter for the distance information calculated by the indirect ToF method, a calculation process is performed using a condition that each of the distance measurement points projected onto the plurality of pixels has a specific positional relationship with each other. Calibration method. [Explanation of symbols]

[0150] 1,1A range finder 2 Light emitting unit 3 Sensor section 4 Lenses 5 Phase difference detection section 6 Arithmetic section 7 Amplitude detection section 8,8A Control unit 8a, 8aA Calibration calculation section 9 Memory section 9a Parameter Information 10 Display 11 Control section Ob Target object Ls irradiation light Lr reflected light 20 flat plate W frame Ar Plane Shooting Area

Claims

1. A light emitting unit that emits light; a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object; a calibration calculation unit that performs a calculation process using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency, as a calibration calculation process for obtaining a correction parameter for distance information calculated by an indirect ToF method based on a light receiving signal of the light receiving sensor. Ranging device.

2. The calibration calculation unit The correction parameter is obtained by performing a calculation process based on a phase difference between the light emission and the light reception detected based on the light reception signal.

2. A distance measuring device according to claim 1.

3. The calibration calculation unit The phase difference is subjected to an ambiguity resolution process for resolving the ambiguity in units of 2π.

3. A distance measuring device according to claim 2.

4. The calibration calculation unit Among the phase differences detected from the light reception signal when light is emitted at a minimum light emission frequency that is the lowest light emission frequency among the light emission frequencies of the light-emitting unit for the calibration calculation process, the phase difference detected from the light reception signal having an amplitude equal to or greater than a predetermined value is determined as a phase difference corresponding to the minimum light emission frequency, and a process is performed to eliminate the uncertainty regarding the phase difference corresponding to the light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency.

4. A distance measuring device according to claim 3.

5. The calibration calculation unit The calibration calculation process is performed based on the elapsed time since the previous execution.

2. A distance measuring device according to claim 1.

6. The calibration calculation unit When an instruction for distance measurement is given during the execution of the calibration calculation process, the calibration calculation process is interrupted and a process for distance measurement is performed.

2. A distance measuring device according to claim 1.

7. A calibration method for a distance measuring device that includes a light emitting unit that emits light and a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object, and performs distance measurement by an indirect ToF method based on a light receiving signal of the light receiving sensor, comprising: As a calibration calculation process for obtaining a correction parameter for distance information calculated by the indirect ToF method, a calculation process is performed using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency. Calibration method.

8. A light emitting unit that emits light; a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object at a plurality of pixels; a calibration calculation unit that performs a calculation process using a condition that each of the distance measurement points projected onto the plurality of pixels has a specific positional relationship as a calibration calculation process for obtaining a correction parameter for distance information calculated by an indirect ToF method based on a light receiving signal of the light receiving sensor, the calibration calculation unit including: Ranging device.

9. The calibration calculation unit As the calibration calculation process, a calculation process is performed under the condition that the distance measurement points are located on an object of known shape.

9. A distance measuring device according to claim 8.

10. The calibration calculation unit As the calibration calculation process, a calculation process is performed using a light receiving signal of the light receiving sensor when the light emitting unit emits light at a first emission frequency and a light receiving signal of the light receiving sensor when the light emitting unit emits light at a second emission frequency different from the first emission frequency.

9. A distance measuring device according to claim 8.

11. The calibration calculation unit The correction parameter is obtained by performing a calculation process based on a phase difference between the light emission and the light reception detected based on the light reception signal.

9. A distance measuring device according to claim 8.

12. The calibration calculation unit The phase difference is subjected to an ambiguity resolution process for resolving the ambiguity in units of 2π. A distance measuring device according to claim 11.

13. The calibration calculation unit Among the phase differences detected from the light reception signal when light is emitted at a minimum light emission frequency that is the lowest light emission frequency among the light emission frequencies of the light-emitting unit for the calibration calculation process, the phase difference detected from the light reception signal having an amplitude equal to or greater than a predetermined value is determined as a phase difference corresponding to the minimum light emission frequency, and a process is performed to eliminate the uncertainty regarding the phase difference corresponding to the light emission frequencies other than the minimum light emission frequency based on the phase difference corresponding to the determined minimum light emission frequency. A distance measuring device according to claim 12.

14. a guide display processing unit that performs processing for displaying a guide image that guides a composition to satisfy the condition that the distance measuring points are in a specific positional relationship; 9. A distance measuring device according to claim 8.

15. A calibration method for a distance measuring device that includes a light emitting unit that emits light, a light receiving sensor that receives light emitted from the light emitting unit and reflected by a target object at a plurality of pixels, and performs distance measurement by an indirect ToF method based on a light receiving signal of the light receiving sensor, comprising: As a calibration calculation process for obtaining a correction parameter for the distance information calculated by the indirect ToF method, a calculation process is performed using a condition that each of the distance measurement points projected onto the plurality of pixels has a specific positional relationship with each other. Calibration method.

Images