A two-phase sampling calibration and correction method and device for an iToF module

Abstract

The present application relates to a kind of two-phase sampling calibration and correction method and device of iToF module.The present application obtains the change curve of the measured phase difference value of two-phase sampling and four-phase sampling measurement phase variation by the sampling timing under the two-phase correlation sampling of the iToF module of the configuration single pixel with two taps, and is stored as look-up table, obtains the proportional coefficient of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in look-up table, and is stored as proportional coefficient table, in two-phase sampling mode, according to the test result of two-phase sampling obtained by sampling timing, the correction value of test result is obtained by looking up the look-up table, the proportional coefficient of current pixel is obtained by looking up proportional coefficient table to further correct the correction value, to correct each pixel under two-phase sampling.The present application two-phase sampling calibration can guarantee ranging accuracy and anti motion blur ability, and reduce hardware cost.

Description

Technical Field

[0001] This invention relates to the field of Time-of-Flight (ToF) ranging technology, and in particular to a two-phase sampling calibration and correction method and apparatus for an iToF module. Background Technology

[0002] Binocular ranging, structured light, and time-of-flight (ToF) are the three mainstream 3D imaging technologies today. Among them, ToF, due to its advantages such as simple principle, simple and stable structure, and long measurement distance, has been gradually applied to fields such as gesture recognition, 3D modeling, autonomous driving, and machine vision. ToF technology is a method for accurately measuring the distance of objects. There are direct-time-of-flight (dToF) ranging technology, which directly measures the flight time of light to calculate the distance of objects, and indirect-time-of-flight (iToF) ranging technology, which calculates the distance of objects by periodically modulating and demodulating the light intensity and using phase information.

[0003] Please refer to the following: Figure 1 , Figure 2 ,in, Figure 1 This is a schematic diagram of the iToF imaging principle. Figure 2 This is a schematic diagram of continuous wave modulation ranging. Specifically, the iToF module controls the light-emitting module 12 to actively emit continuously modulated light pulses as emitted light 13 through the modulation module 11. The emitted light 13 is emitted onto the surface of the target object 19, and the reflected light 14 formed after reflection by the target object 19 is captured by the photosensitive pixel array 15 of the image sensor. The phase shift between the emitted light 13 and the reflected light 14 is calculated. To obtain the depth of target 19. The light-emitting module 12 can be a vertical-cavity surface-emitting laser (VCSEL), an infrared emitter (IR emitter), or a light-emitting diode (LED), etc. Since the speed of light c and the modulation frequency f of the emitted light are known quantities, the phase shift is obtained... Based on this, the depth d of the target object 19 can be obtained using the following formula:

[0004]

[0005] For iToF technology, due to non-ideal factors such as ambient light and manufacturing process errors, multi-phase sampling techniques (commonly four-phase sampling, but other single-pixel multi-tap sampling techniques also exist) are generally used to eliminate these non-ideal factors and ensure the accuracy and robustness of phase information, i.e., distance information, in order to achieve accurate phase measurement. However, since the time points of multi-phase sampling are not the same, the actual object may have moved. In this case, the depth calculated from the information of multiple samplings will be affected by motion blur, which greatly affects the accuracy of ranging. In order to simultaneously ensure ranging accuracy and anti-motion blur capability, placing more taps within a single pixel and using correction methods is a common solution. This places higher demands on the design of iToF modules, correction methods, and storage of correction information. The complex correction methods and data storage requirements also increase the overall cost of iToF modules.

[0006] Therefore, how to effectively ensure ranging accuracy and anti-motion blurring capability without increasing the overall cost of the iToF module is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] The purpose of this invention is to provide a two-phase sampling calibration and correction method and device for iToF modules, which effectively ensures ranging accuracy and anti-motion ambiguity capability through two-phase sampling calibration, and reduces hardware costs.

[0008] To achieve the above objectives, this invention provides a two-phase sampling calibration and correction method for an iToF module, comprising the following steps: configuring a sampling timing sequence for two-phase correlated sampling of an iToF module with two taps per pixel, wherein the sampling timing sequence is to perform first and second phase sampling when the active light source of the iToF module is on, and to perform a third sampling to detect ambient light intensity when the active light source is off; obtaining the change curve of the measured phase difference value of two-phase sampling and four-phase sampling as a function of the measured phase change of two-phase sampling, and storing it as a lookup table; obtaining the proportional coefficient of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and storing it as a proportional coefficient table; and in the two-phase sampling mode, obtaining the test result of two-phase sampling according to the sampling timing sequence, obtaining the correction value of the test result by looking up the lookup table, and obtaining the proportional coefficient of the current pixel by looking up the proportional coefficient table to further correct the correction value, so as to correct each pixel under two-phase sampling.

[0009] To achieve the above objectives, the present invention also provides a two-phase sampling calibration and correction device for an iToF module, comprising: a sampling timing configuration module, used to configure the sampling timing of a two-phase associated sampling of an iToF module with two taps per pixel, wherein the sampling timing is to perform first and second phase sampling when the active light source of the iToF module is on, and to perform a third sampling to detect ambient light intensity when the active light source is off; a lookup table acquisition module, used to acquire the change curve of the measured phase difference value of two-phase sampling and four-phase sampling with the change of the measured phase of two-phase sampling, and store it as a lookup table; a scaling factor table acquisition module, used to acquire the scaling factor of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and store it as a scaling factor table; and a two-phase correction module, used to obtain the test result of two-phase sampling according to the sampling timing in two-phase sampling mode, obtain the correction value of the test result by looking up the lookup table, and obtain the scaling factor of the current pixel by looking up the scaling factor table to further correct the correction value, so as to correct each pixel under two-phase sampling.

[0010] This invention improves the ranging accuracy of two-phase sampling by calibrating and correcting the additional error generated by two-phase sampling compared to four-phase sampling at the pixel level. With lower storage costs and less algorithm overhead, it achieves higher resistance to motion blur, ambient light interference, and process errors in iToF modules under two-phase sampling. This invention does not require additional hardware to store the image of the signal offset value difference of pixel taps, or the correction data of the charge-voltage conversion gain difference of pixel taps, thus reducing the system's storage costs. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the iToF imaging principle;

[0013] Figure 2 This is a schematic diagram of continuous wave modulation ranging.

[0014] Figure 3 This is a schematic diagram of the four-phase sampling phase of the iToF module;

[0015] Figure 4 This is a schematic diagram of the four-phase sampling timing of the iToF module;

[0016] Figure 5 This is a schematic diagram illustrating the steps of the two-phase sampling calibration and correction method for the iToF module provided by the present invention;

[0017] Figure 6 This is a schematic diagram of a two-phase sampling timing provided in an embodiment of the present invention;

[0018] Figure 7 A flowchart for calculating and obtaining a LUT is provided in one embodiment of the present invention;

[0019] Figure 8 This is a schematic diagram illustrating the measurement phase difference between two-phase sampling and four-phase sampling under the square wave algorithm provided in an embodiment of the present invention.

[0020] Figure 9 A flowchart for obtaining the scaling factor is provided in one embodiment of the present invention;

[0021] Figure 10 This is a structural block diagram of the two-phase sampling calibration and correction device for the iToF module provided by the present invention. Detailed Implementation

[0022] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please refer to the following: Figure 3 , Figure 4 ,in, Figure 3 This is a schematic diagram of the four-phase sampling phase of the iToF module. Figure 4 This diagram illustrates the four-phase sampling timing of an iToF module. In the diagram, Mod represents the modulation waveform of the emitted light, and Demod represents the demodulation waveform of the reflected light captured by the iToF module's photosensitive pixel array. To eliminate the influence of ambient light and non-ideal factors in the manufacturing process, a four-phase sampling method is generally used for iToF modules with two taps A and B per pixel. In each sampling, two taps A and B will obtain a set of measurement values. The demodulated waveforms are delayed by 1 / 4 of a modulation period in each of the four samplings, resulting in four sets of values ​​for taps A and B, namely A0, B0, and A... 90 B 90 A 180 B 180 A 270 B 270 This can eliminate the influence of ambient light and non-ideal factors in the process.

[0024] For a modulated signal waveform that is a sine wave (sin), the values ​​of taps A and B at different phase sampling times can be expressed by the following formula:

[0025]

[0026]

[0027] In this context, the subscript φ represents 0°, 90°, 180°, and 270°, and G... A G B Amp represents the charge-to-voltage conversion gain of pixel taps A and B. (环境+DC) A is the sum of the ambient light intensity received by the pixel and the DC component of the light intensity emitted by the iToF module. offset B offset These represent the signal offset values ​​of taps A and B in this pixel, respectively, and V represents the AC component amplitude of the light intensity emitted by the iToF module. Since taps A and B are in the same pixel, the DC component of the sum of the ambient light intensity received by tap A and the light intensity emitted by the iToF module is the same as the DC component of the sum of the ambient light intensity received by tap B and the light intensity emitted by the iToF module, both represented by Amp. (环境+DC) express.

[0028] The measured values ​​of taps A and B at different phase sampling times are obtained using the above formulas. Then, I and Q are obtained using the following formulas:

[0029]

[0030]

[0031] After obtaining I and Q, the measured phase value can be directly calculated using the sin algorithm or the square wave algorithm.

[0032]

[0033]

[0034] However, due to the time difference between adjacent phase sampling points, and the difference being an integer multiple of Δt1 + Δt2, such as Figure 4 As shown. High-speed moving objects can cause the actual four-phase sampling result for the same pixel to differ from the image of the same object, resulting in motion blur. To improve resistance to motion blur, only one phase sampling method can be used. Figure 3 The sampling of the first two phases yields two sets of tap values ​​A and B, namely A0 and B0. 90 B 90 And calculate I, Q and using the above formula. While this method improves resistance to dynamic blurring, the influence of ambient light and non-ideal factors in the process will significantly affect ranging accuracy. However, if A is obtained through pre-calibration... offset B offset And subtract from the initial test signal, as well as the pre-calibrated G for each pixel. A G B As a result, the subsequent compensation and correction algorithm for gain mismatch requires storing A. offset B offset Different images, and G A G B Correction data for gain differences increases the cost of storing correction data.

[0035] To achieve higher resistance to motion blur, ambient light interference, and process errors in iToF modules with two-phase sampling, this invention provides a two-phase sampling calibration and correction method for iToF modules by performing pixel-level calibration and correction compensation for the additional errors generated by two-phase sampling compared to four-phase sampling, thereby improving the ranging accuracy of two-phase sampling. This method is implemented with lower storage costs and less algorithm overhead.

[0036] Please refer to the following: Figures 5-9 ,in, Figure 5 This is a schematic diagram illustrating the steps of the two-phase sampling calibration and correction method for the iToF module provided by the present invention. Figure 6 This is a schematic diagram of a two-phase sampling timing provided in an embodiment of the present invention. Figure 7 This is a flowchart of the calculation to obtain the LUT according to an embodiment of the present invention. Figure 8 This is a schematic diagram illustrating the measurement phase difference between two-phase sampling and four-phase sampling under the square wave algorithm provided in an embodiment of the present invention. Figure 9 This is a flowchart for obtaining the scaling factor according to an embodiment of the present invention.

[0037] like Figure 5As shown, the two-phase sampling calibration and correction method of the iToF module in this embodiment includes the following steps: S1, configuring the sampling timing sequence of the iToF module with two taps per pixel under two-phase associated sampling, wherein the sampling timing sequence is to perform the first phase and the second phase sampling when the active light source of the iToF module is on, and to perform the third sampling when the active light source is off to detect the ambient light intensity; S2, obtaining the change curve of the measured phase difference value of two-phase sampling and four-phase sampling with the change of the measured phase of two-phase sampling, and storing it as a lookup table; S3, obtaining the proportional coefficient of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and storing it as a proportional coefficient table; and S4, in the two-phase sampling mode, obtaining the test result of two-phase sampling according to the sampling timing sequence, obtaining the correction value of the test result by looking up the lookup table, and obtaining the proportional coefficient of the current pixel by looking up the proportional coefficient table to further correct the correction value, so as to correct each pixel under two-phase sampling.

[0038] Regarding step S1, the sampling timing of the iToF module with two taps per pixel under two-phase correlated sampling is configured. The sampling timing is to perform the first phase and the second phase sampling when the active light source of the iToF module is turned on, and to perform the third sampling when the active light source is turned off, so as to detect the ambient light intensity.

[0039] like Figure 6 As shown, the four-phase sampling timing (reference) Figure 4 Unlike the previous version, the two-phase sampling timing configuration of this invention discards the sampling of the fourth phase and turns off the active light source of the iToF module that samples the third phase in order to detect the ambient light intensity. The configuration of other parts is consistent with that of the four-phase sampling.

[0040] In some embodiments, the sampling results of the first phase and the second phase are denoted as A. φ B φ :

[0041]

[0042]

[0043] In this context, the subscript φ represents 0° and 90°, and G... A G B Amp represents the charge-to-voltage conversion gain of taps A and B. (环境+DC) A is the sum of the ambient light intensity received by the pixel and the DC component of the light intensity emitted by the iToF module. offset B offsetThese represent the signal offset values ​​for taps A and B, respectively, and V represents the AC component amplitude of the light intensity emitted by the iToF module. Since taps A and B are located in the same pixel, the DC component of the sum of the ambient light intensity received by tap A and the light intensity emitted by the iToF module is the same as the DC component of the sum of the ambient light intensity received by tap B and the light intensity emitted by the iToF module, both represented by Amp. (环境+DC) express.

[0044] The result of the third sampling is denoted as A. 环境 B 环境 :

[0045] A 环境 =G A ×Amp 环境 +A offset B 环境 =G B ×Amp 环境 +B offset .

[0046] Among them, Amp 环境 Let Amp be the ambient light intensity received by the pixel. Since taps A and B are in the same pixel, the ambient light intensity received by tap A is the same as that received by tap B, both denoted by Amp. 环境 express.

[0047] I and Q are obtained using the following formula:

[0048]

[0049]

[0050] Among them, (G) A -G B )×Amp DC This residual signal is caused by the difference in charge-voltage conversion gain between different taps of the same pixel. This residual signal in I and Q will affect the measured phase value of that pixel, ultimately deteriorating the accuracy of two-phase ranging. To improve the accuracy of two-phase sampling ranging, this invention further calibrates and corrects each pixel individually to compensate for the ranging error caused by the residual signal.

[0051] Regarding step S2, obtain the curve of the change of the measured phase difference value between two-phase sampling and four-phase sampling as the measured phase of two-phase sampling changes, and store it as a lookup table.

[0052] In some embodiments, step S2 further includes: 1) when the difference in charge-voltage conversion gain between the two taps is a preset percentage, obtaining a curve showing the change in the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase as a function of the two-phase sampling measurement phase, using an actual optical waveform, an ideal square wave waveform, or an ideal sine wave waveform; 2) based on the piecewise linear characteristics of the curve, obtaining and storing the inflection points of each segment of the curve to form the lookup table. The preset percentage can be 3%, 3.7%, 7%, or other difference percentages.

[0053] like Figure 7 As shown, with G A G B Taking a difference percentage of 3% as an example, the lookup table acquisition method of this invention will be further illustrated. The specific steps are as follows: S71, using an actual light waveform (obtained by actual measurement) or an ideal square wave waveform or an ideal sine wave waveform; S72, considering the charge-voltage conversion gain G of pixel taps A and B. A G B When the difference percentage is 3%, S73, theoretical calculation of the measurement phase of two-phase sampling. and the measurement phase of four-phase sampling S74. Obtain the difference between the two ( S75, Obtain the difference value ( The curve showing the change in phase with the two-phase sampling measurement is generated. Using this curve as a lookup table (LUT), it is stored. Based on the piecewise linear characteristic of the curve, only the inflection points of each segment of the curve need to be stored; intermediate values ​​can be obtained using interpolation.

[0054] Regarding step S3, obtain the scaling factor of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and store it as a scaling factor table.

[0055] The calibration and adjustment principles of this invention under two-phase sampling will be explained further. Although non-ideal factors in the process may cause G... A ≠G B Furthermore, the differences exist between different pixels, but in reality, the impact of these non-ideal factors on ranging accuracy, i.e., wiggling error, follows certain patterns. For example... Figure 8 As shown, the square wave phase algorithm is used to measure different G values. A G B In the case of the difference percentage, theoretical calculations are performed on the two-phase sampling measurement phase and the four-phase sampling measurement phase respectively to obtain the difference value of the measurement phase calculated by the two methods. Figure 8The horizontal axis represents the converted coordinate value when π is 3.14, and the vertical axis represents the difference between the two-phase measured phase and the four-phase measured phase. Figure 8 We found that, whether it is G A >G B Still G A <G B In this case, the difference value exhibits the same characteristics as the measurement phases of the two-phase sampling, namely, there are two periods where the difference value is constant, located in the intervals where the measurement phase falls within [π / 2, π] and [(3π) / 2, 2π], respectively. Simultaneously, the difference value between these two constant periods is linearly connected, and the difference value is zero at the measurement phases equal to π / 4 and (5π) / 4. A G B The percentage difference only affects the amplitude of the difference curve. In practical applications, to make the difference curve more stable, an appropriate phase calculation formula, such as the sin algorithm or square wave algorithm, can be selected based on the actual light source waveform of the iToF module when calculating the phase using I and Q values. When the calculation formula matches the actual light source waveform of the iToF module, the oscillation error curve of the four-phase sampling will also remain at a small level.

[0056] Because the curve of this difference value changing with the measurement phase is different in G A G B Since similarity exists regardless of percentage difference, this invention stores a lookup table (LUT) representing this characteristic and calibrates each pixel to obtain a scaling factor for the change curve of the measured phase difference value between two-phase sampling and four-phase sampling for each pixel relative to the change curve in the lookup table. Using the scaling factor and the lookup table, individual correction can be performed on each pixel under two-phase sampling, ultimately correcting the measured phase of two-phase sampling to be equal to the measured phase value under four-phase sampling. Since the accuracy of four-phase sampling measurement phase is unaffected by ambient light and non-ideal factors in the process, the corrected two-phase sampling measurement phase also eliminates most of the influence of ambient light and non-ideal factors in the process, enabling the accuracy of the two-phase sampling measurement phase result to reach the accuracy performance of four-phase sampling.

[0057] In some embodiments, step S3 further includes: 1) placing the iToF module directly opposite the calibration whiteboard at a first distance, adjusting the time delay between the modulated light signal and the demodulated light signal of the iToF module, so that the two-phase sampling measurement phase at the center point of the image sensor of the iToF module is a first preset value, and acquiring and storing the first four-phase average value of the whole-area measurement phase of the four-phase sampling and the first two-phase average value of the whole-area measurement phase of the two-phase sampling under this condition; 2) adjusting the time delay between the modulated light signal and the demodulated light signal of the iToF module or adjusting the distance between the iToF module and the calibration whiteboard, so that the two-phase sampling measurement phase at the center point of the image sensor of the iToF module is a first preset value. 2. Under the given conditions, acquire and store the second four-phase average value and the second two-phase average value of the whole-area measurement phase of four-phase sampling for a preset number of images; 3) Acquire the first difference value between the first two-phase average value and the first four-phase average value for each pixel, and the second difference value between the second two-phase average value and the second four-phase average value. Based on the first and second preset values, search the lookup table to obtain the first and second lookup values. Based on the first and second difference values, the first and second lookup values, obtain the proportional coefficient of the measurement phase difference value between two-phase and four-phase sampling for each pixel relative to the change curve in the lookup table, and store it as a proportional coefficient table. For example, the ratio of the difference between the first and second difference values ​​to the difference between the first and second lookup values ​​is used as the proportional coefficient for that pixel.

[0058] In some embodiments, the first distance is a distance that ensures the difference between the two phases of the sampling measurements of all pixels is less than or equal to π / 2. For example, the first distance can be 50cm. The iToF module is placed 50cm away from a flat calibration whiteboard to ensure that the difference between the two phases of the sampling measurements of all pixels does not exceed π / 2; otherwise, the distance between the calibration whiteboard and the iToF module is reduced.

[0059] In some embodiments, the first preset value is a first constant stage that ensures the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase of the entire pixel area is constant. Specifically, the first constant stage is a stage where the measurement phase falls within the interval [π / 2, π], and the first preset value can be approximately (3π) / 4. The second preset value is a second constant stage that ensures the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase of the entire pixel area is constant. Specifically, the second constant stage is a stage where the measurement phase falls within the interval [(3π) / 2, 2π], and the second preset value can be approximately (7π) / 4. The preset number of images can be 100, or it can be changed according to the calibration time limit.

[0060] In some embodiments, the scaling factor is further expressed by the following formula:

[0061]

[0062] Wherein, ratio is the proportionality coefficient. The average value of the first two phases. The average value of the first four phases. This is the average value of the second two phases. This is the average value of the second and fourth phases. For the first preset value, LUT( The first lookup value is obtained by searching the lookup table based on the first preset value. The second preset value, LUT( The second lookup value is obtained by searching the lookup table based on the second preset value.

[0063] like Figure 9 As shown, taking the first preset value as approximately equal to (3π) / 4 and the second preset value as (7π) / 4 as examples, the method for obtaining the proportional coefficient of the present invention will be further illustrated. The specific steps are as follows: S901, place the iToF module directly opposite the calibration white board at a first distance; S902, determine whether the difference between the two-phase sampling measurement phases of the entire pixel is less than or equal to π / 2. If yes, proceed to the next step; otherwise, return to step S901 to adjust the distance between the iToF module and the calibration white board; S903, adjust the time delay between the modulation light signal and the demodulation light signal of the iToF module; S904, determine whether the two-phase sampling measurement phases of the center point of the image sensor of the iToF module are approximately equal to (3π) / 4. If yes, proceed to the next step; otherwise, return to step S903 to adjust the time delay between the modulation light signal and the demodulation light signal; S905, acquire and store the first four-phase average value of the entire measurement phase of 100 four-phase samples under this condition. And switch to two-phase sampling to acquire and store the first two-phase average value of the full-area measurement phase of 100 two-phase samples under this condition. S906, Obtain the difference between the average value of the first two phases and the average value of the first four phases for each pixel. And store; return to step S903 to adjust the time delay between the modulated optical signal and the demodulated optical signal (or adjust the distance between the iToF module and the calibration whiteboard), and execute S907, determine whether the two-phase sampling measurement phase of the image sensor center point of the iToF module is approximately equal to (7π) / 4. If so, proceed to the next step; otherwise, return to step S903 to adjust the time delay between the modulated optical signal and the demodulated optical signal; S908, acquire and store the second four-phase average value of the full-area measurement phase of 100 four-phase samples under this condition. And switch to two-phase sampling to acquire and store the second two-phase average value of the full-area measurement phase of 100 two-phase samples under this condition. S909, Obtain the difference between the second two-phase average value and the second four-phase average value for each pixel. And store; S910, obtain the difference 1 between the difference value 1 and the difference value 2 of each pixel (difference 1 = difference value 1 - difference value 2); S911, find the lookup value 1 with a measurement phase equal to (3π) / 4 and the lookup value 2 with a measurement phase equal to (7π) / 4 in the lookup table; S912, obtain the difference 2 between the lookup value 1 and the lookup value 2 (difference 2 = lookup value 1 - lookup value 2); S913, obtain the ratio of the difference between the difference value 1 and the difference value 2 of each pixel to the difference between the lookup value 1 and the lookup value 2 as the ratio coefficient of the pixel (ratio = difference 1 / difference 2), which is the ratio coefficient of the measurement phase difference value of the two-phase sampling and four-phase sampling of the pixel relative to the change curve in the lookup table; S914, store the ratio coefficient of each pixel to form a ratio coefficient table.

[0064] In other words, the present invention only needs to store the lookup table and the proportional coefficient table, which has low storage cost and low algorithm cost.

[0065] Regarding step S4, in the two-phase sampling mode, the test result of the two-phase sampling is obtained according to the sampling timing. The correction value of the test result is obtained by looking up the lookup table. The proportional coefficient of the current pixel is obtained by looking up the proportional coefficient table to further correct the correction value, so as to correct each pixel under the two-phase sampling.

[0066] In some embodiments, the correction of each pixel under two-phase sampling is further performed using the following formula:

[0067]

[0068] in, To correct the results, The test results are as follows. The correction value is ratio, the scaling factor, and wiggling. 四相位采样For the FPPN, the pre-saved four-phase sampling swing error calibration data of the iToF module 四相位采样 The FPPN calibration data of four-phase sampling is pre-saved for the iToF module.

[0069] Before the iToF module leaves the factory, it undergoes pre-calibration of the swing error of four-phase sampling and FPPN, and obtains the corresponding calibration data. The iToF module of this invention can have two usage modes: four-phase sampling mode and two-phase sampling mode. In application scenarios where time delay and motion blur are not sensitive, the four-phase sampling mode can be used. The specific working principle can refer to the existing four-phase sampling mode. In application scenarios where time delay and motion blur are more sensitive, such as the application scenario of shooting high-speed moving objects, it can be switched to two-phase sampling mode, using the two-phase sampling calibration and correction method of this invention to reduce the sampling interval and reduce the influence of motion blur.

[0070] As can be seen from the above, this invention improves the ranging accuracy of two-phase sampling by calibrating and correcting the additional error generated by two-phase sampling at each pixel level, and achieves high resistance to motion blur, ambient light interference and process error in iToF module two-phase sampling with low storage cost and small algorithm cost.

[0071] Based on the same inventive concept, this invention also provides a two-phase sampling calibration and correction device for an iToF module. The provided two-phase sampling calibration and correction device for an iToF module can be used as follows: Figures 5-9 The two-phase sampling calibration and correction method shown completes the two-phase sampling calibration and correction of the iToF module.

[0072] Please see Figure 10 This is a structural block diagram of the two-phase sampling calibration and correction device for the iToF module provided by the present invention. Figure 10 As shown, the two-phase sampling calibration and correction device of the iToF module includes: a sampling timing configuration module 101, a lookup table acquisition module 102, a scaling factor table acquisition module 103, and a two-phase correction module 104.

[0073] Specifically, the sampling timing configuration module 101 is used to configure the sampling timing under two-phase correlated sampling of an iToF module with two taps per pixel. The sampling timing is to perform first and second phase sampling when the active light source of the iToF module is on, and to perform a third sampling when the active light source is off to detect the ambient light intensity. The lookup table acquisition module 102 is used to acquire the change curve of the measured phase difference value of two-phase sampling and four-phase sampling with the change of the measured phase of two-phase sampling, and store it as a lookup table. The scaling factor table acquisition module 103 is used to acquire the scaling factor of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and store it as a scaling factor table. The two-phase correction module 104 is used to obtain the test result of two-phase sampling according to the sampling timing in two-phase sampling mode, obtain the correction value of the test result by looking up the lookup table, and obtain the scaling factor of the current pixel by looking up the scaling factor table to further correct the correction value, so as to correct each pixel under two-phase sampling.

[0074] The working methods of each module can be found in the following references. Figures 5-9 The description of the corresponding steps in the two-phase sampling calibration and correction method of the iToF module shown is not repeated here.

[0075] Based on the same inventive concept, the present invention also provides an electronic device, including a memory, a processor, and a computer-executable program stored in the memory and executable on the processor; when the processor executes the computer-executable program, it implements as follows: Figures 5-9 The steps of the two-phase sampling calibration and correction method for the iToF module are shown.

[0076] Within the scope of this inventive concept, embodiments can be described and illustrated based on modules that perform one or more of the described functions. These modules can be physically implemented by analog and / or digital circuitry, such as logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, etc., and can optionally be driven by firmware and / or software. The circuitry can be implemented, for example, in one or more semiconductor chips. The circuitry constituting a module can be implemented by dedicated hardware, or by a processor (e.g., one or more programmed microprocessors and associated circuitry), or by a combination of dedicated hardware performing some functions of the module and a processor performing other functions of the module. Without departing from the scope of this inventive concept, each module of an embodiment can be physically divided into two or more interactive and discrete modules. Similarly, without departing from the scope of this inventive concept, the modules of an embodiment can be physically combined into more complex modules.

[0077] Generally, terms can be understood at least partially from their usage in context. For example, the term "one or more" as used herein depends at least in part on the context and can be used to describe a feature, structure, or characteristic in a singular sense, or in a plural sense to describe a combination of features, structures, or characteristics. Additionally, the term "based on" can be understood not necessarily to express an exclusive set of factors, but rather, alternatively, also depends at least in part on the context, allowing for the presence of other factors that are not necessarily explicitly described.

[0078] It should be noted that the terms "comprising" and "having," and their variations, used in this invention document are intended to cover non-exclusive inclusion. The terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence, unless explicitly indicated by the context. It should be understood that such data used interchangeably where appropriate. Furthermore, embodiments and features within embodiments of this invention can be combined with each other unless otherwise specified. In addition, descriptions of well-known components and technologies have been omitted in the above description to avoid unnecessarily obscuring the concepts of this invention. In the various embodiments described above, each embodiment focuses on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A two-phase sampling calibration and correction method for an iToF module, characterized in that, Includes the following steps: The sampling timing sequence of a two-phase correlated sampling of an iToF module with two taps per pixel is configured. The sampling timing sequence is to perform the first phase and the second phase sampling when the active light source of the iToF module is turned on, and to perform the third sampling when the active light source is turned off, so as to detect the ambient light intensity. Obtain the curves showing the change of the measured phase difference between two-phase sampling and four-phase sampling as a function of the measured phase in two-phase sampling, and store them as a lookup table; Obtain the scaling factor of the measured phase difference value of two-phase sampling and four-phase sampling for each pixel relative to the change curve in the lookup table, and store it as a scaling factor table; as well as In the two-phase sampling mode, the test results of the two-phase sampling are obtained according to the sampling timing. The correction value of the test results is obtained by looking up the lookup table. The proportional coefficient of the current pixel is obtained by looking up the proportional coefficient table to further correct the correction value, so as to correct each pixel under the two-phase sampling.

2. The method according to claim 1, characterized in that, The step of performing first and second phase sampling when the active light source of the iToF module is on, and performing a third sampling when the active light source is off, to detect the ambient light intensity, further includes: Let A be the sampling result of the first phase and the second phase. B : A =G A ×Amp （环境+DC） +A offset +2G A ×V×cos(φ- ), B =G B ×Amp （环境+DC） +B offset -2G B ×V×cos(φ- ), Among them, the subscript G represents 0° and 90°. A G B Amp represents the charge-to-voltage conversion gain of taps A and B. （环境+DC） A is the sum of the ambient light intensity received by the pixel and the DC component of the light intensity emitted by the iToF module. offset B offset These represent the signal offset values ​​of taps A and B, respectively; V represents the AC component amplitude of the light intensity emitted by the iToF module; and φ is the phase shift of the emitted and reflected light. The result of the third sampling is denoted as A. 环境 B 环境 : A 环境 =G A ×Amp 环境 +A offset ,B 环境 =G B ×Amp 环境 +B offset , Among them, Amp 环境 The intensity of ambient light received by the pixel; I and Q are obtained using the following formula: I=(A0-B0)-(A 环境 -B 环境 )=(G A -G B )×Amp DC +2(G A +G B )×Vcosφ, Q=(A 90 -B 90 )-(A 环境 -B 环境 )=(G A -G B )×Amp DC +2(G A +G B )×Vsinφ, Where A0 is the phase sample value of tap A at 0°, A 90 B0 is the phase sample value of tap A at 90°, and B0 is the phase sample value of tap B at 0°. 90 The phase sample value of tap B at 90°, (G) A -G B )×Amp DC This is the residual signal caused by the difference in charge-voltage conversion gain between different taps of the same pixel.

3. The method according to claim 1, characterized in that, The step of obtaining the curves showing the change of the measured phase difference between two-phase sampling and four-phase sampling as a function of the measured phase in two-phase sampling, and storing them as a lookup table, further includes: When the difference between the charge-voltage conversion gain of the two taps is a preset percentage, the curve of the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase is obtained as the two-phase sampling measurement phase changes, using the actual optical waveform, ideal square wave waveform, or ideal sine wave waveform. Based on the segmented linear characteristics of the curve, the inflection points of each segment of the changing curve are obtained and stored to form the lookup table.

4. The method according to claim 1, characterized in that, The step of obtaining the scaling factor of the measured phase difference curve of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and storing it as a scaling factor table, further includes: Place the iToF module directly opposite the calibration whiteboard at a first distance, adjust the time delay between the modulation light signal and the demodulation light signal of the iToF module, so that the two-phase sampling measurement phase of the center point of the image sensor of the iToF module is a first preset value, acquire and store the first four-phase average value of the whole-area measurement phase of the four-phase sampling of a preset number of images under this condition and the first two-phase average value of the whole-area measurement phase of the two-phase sampling. Adjust the time delay between the modulated and demodulated optical signals of the iToF module or adjust the distance between the iToF module and the calibration whiteboard so that the two-phase sampling measurement phase at the center point of the image sensor of the iToF module is a second preset value. Obtain and store the second four-phase average value of the whole-area measurement phase of the four-phase sampling with a preset number of images under this condition and the second two-phase average value of the whole-area measurement phase of the two-phase sampling. Obtain the first difference value between the first two-phase average value and the first four-phase average value of each pixel, and the second difference value between the second two-phase average value and the second four-phase average value. According to the first preset value and the second preset value, look up the lookup table to obtain the first lookup value and the second lookup value. According to the first difference value, the second difference value, the first lookup value, and the second lookup value, obtain the scaling factor of the measurement phase difference value of the two-phase sampling and the four-phase sampling of each pixel relative to the change curve in the lookup table, and store it as a scaling factor table.

5. The method according to claim 4, characterized in that, The first distance is the distance that makes the difference between the two phases of the sampling measurement of the entire pixel less than or equal to π / 2.

6. The method according to claim 4, characterized in that, The first preset value is a first constant stage that makes the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase of the entire pixel constant, and the second preset value is a second constant stage that makes the difference between the two-phase sampling measurement phase and the four-phase sampling measurement phase of the entire pixel constant.

7. The method according to claim 6, characterized in that, The first constant phase is the phase in which the measured phase falls within the interval [π / 2, π], and the second constant phase is the phase in which the measured phase falls within the interval [(3π) / 2, 2π].

8. The method according to claim 4, characterized in that, The proportionality coefficient is further expressed by the following formula: ratio=((φ 两相位1 -f 四相位1 )-(φ 两相位2 -f 四相位2 )) / (LUT (φ1)-LUT (φ2)), Where ratio is the proportionality coefficient, φ 两相位1 φ is the average value of the first two phases. 四相位1 φ is the average value of the first four phases. 两相位2 φ is the average value of the second two phases. 四相位2 φ1 is the first preset value, LUT(φ1) is the first lookup value obtained by searching the lookup table according to the first preset value, φ2 is the second preset value, and LUT(φ2) is the second lookup value obtained by searching the lookup table according to the second preset value.

9. The method according to claim 1, characterized in that, The correction of each pixel under two-phase sampling is further performed using the following formula: φ r =φ m -LUT(φ m )×ratio-wiggling 四相位采样 -FPPN 四相位采样 , Where φ is the phase shift between the emitted and reflected light, φ r For the correction result, φ m For the test results, LUT(φ) m ) is the correction value, ratio is the scaling factor, and wiggling is the scaling factor. 四相位采样 For the FPPN, the pre-saved four-phase sampling swing error calibration data of the iToF module 四相位采样 The FPPN calibration data of four-phase sampling is pre-saved for the iToF module.

10. A two-phase sampling calibration and correction device for an iToF module, characterized in that, include: The sampling timing configuration module is used to configure the sampling timing of the two-phase correlated sampling of the iToF module with two taps for a single pixel. The sampling timing is to perform the first phase and the second phase sampling when the active light source of the iToF module is turned on, and to perform the third sampling when the active light source is turned off, so as to detect the ambient light intensity. The lookup table acquisition module is used to obtain the curve of the measurement phase difference value of two-phase sampling and four-phase sampling as a function of the measurement phase of two-phase sampling, and store it as a lookup table; The scaling factor table acquisition module is used to acquire the scaling factor of the measured phase difference value of two-phase sampling and four-phase sampling of each pixel relative to the change curve in the lookup table, and store it as a scaling factor table; and The two-phase correction module is used to obtain the test results of two-phase sampling according to the sampling timing in the two-phase sampling mode, obtain the correction value of the test results by looking up the lookup table, and obtain the scaling factor of the current pixel by looking up the scaling factor table to further correct the correction value, so as to correct each pixel under two-phase sampling.

Images