Calibration method and device of ToF depth camera, and electronic device

Abstract

The application relates to a calibration method and device of a ToF depth camera and electronic equipment. The method comprises the following steps: obtaining average measurement phases by performing time domain averaging on measurement phases for a first preset number of times at an initial delay value; taking a center point measurement phase as a seed point, searching for a pixel point with the same measurement phase, and obtaining a target pixel coordinate; adjusting the delay value by a preset step, and after each adjustment of the delay value, capturing a preset frame of current measurement phases, obtaining the current measurement phases same as the target pixel coordinate, and averaging to obtain a current center point measurement phase corresponding to the current delay value; and after completing the delay value adjustment and the corresponding center point measurement phase acquisition of a preset period, generating a mapping table of the measurement phase and the real phase, and completing wiggling calibration. The application saves calibration time and reduces calibration cost by replacing the time domain with the space domain, and further matches the real swing error in a linear interpolation manner.

Description

Technical Field

[0001] This application relates to the field of ToF ranging technology, and in particular to a calibration method and apparatus for a ToF depth camera, as well as electronic equipment. Background Technology

[0002] Binocular ranging, structured light, and time-of-flight (ToF) are the three major mainstream 3D imaging technologies today. Among them, ToF, due to its advantages such as simple principle, simple and stable structure, and long measurement distance, has been gradually applied to fields such as gesture recognition, 3D modeling, autonomous driving, and machine vision. Figure 1 As shown, the working principle of ToF technology is as follows: The ToF depth camera controls the light-emitting module 12 (VCSEL, infrared emitter, or LED, etc.) to emit continuously modulated light through the control module (Controller) 11. After the emitted light shines on the surface of the target object 19, it is reflected back and captured by the image sensor (sensor) 13 of the ToF depth camera. By calculating the phase difference between the emitted light and the reflected light, the depth information of the target object 19 from the ToF depth camera is obtained.

[0003] Due to inherent imaging limitations and external environmental interference, data acquired directly from ToF depth cameras typically contains certain errors. Therefore, a series of calibrations are required before application to improve measurement accuracy. Common calibration methods include Wiggling calibration, FPPN calibration, and lens calibration, with Wiggling calibration addressing distance-related cyclic errors. Because the actual light waveform differs from the ideal situation, the measured phase exhibits a periodic variation with the ideal phase. Figure 2 As shown in the figure. The horizontal axis represents the real phase, and the vertical axis represents the wriggling error.

[0004] In existing technologies, a mapping relationship is typically established by moving a calibration plate, measuring different distances and the actual distance, and then calculating the oscillation error of the entire cycle through fitting. Figure 3 As shown. Alternatively, by changing the delay value through a delay circuit to scan a complete cycle, the mapping relationship between the measured distance and the actual distance can be obtained, such as... Figure 4The increase in time delay is equivalent to an increase in real distance of each image sensor by an offset. However, the ToF depth camera measures various random noises due to the pixel itself, and needs to be averaged in the time domain; therefore, a certain amount of data needs to be captured at each distance for averaging, which requires a certain amount of calibration time and thus increases the calibration cost. Chinese Patent Publication CN113281726A discloses an error calibration method and device, which simulates a virtual calibration board through an internal delay circuit to replace the traditional moving calibration board to generate a series of virtual real distances, establishes a mapping relationship according to the measured distance and the virtual real distance, and calculates the wiggling error of the entire cycle through fitting to complete the wiggling calibration of the entire cycle. However, capturing multiple data for averaging at each virtual real distance increases the time cost and results in an increase in calibration cost. Moreover, the fitting model determined through fitting is not applicable to all measurement points, which results in a large residual error for some points.

[0005] Therefore, how to save calibration time and reduce calibration cost is a technical problem to be solved at present. SUMMARY

[0006] The present application aims to provide a ToF depth camera calibration method and device and electronic equipment, which solve the technical problems of long calibration time and high calibration cost of the existing ToF depth camera calibration, and save calibration time and reduce calibration cost by replacing the time domain with the spatial domain.

[0007] To achieve the above-mentioned purpose, the present application provides a ToF depth camera calibration method, which comprises the following steps: at an initial delay value, performing time domain averaging of each measurement phase in the image sensor of the ToF depth camera a first preset number of times to obtain the corresponding average measurement phase, so as to remove random noise; taking the center point measurement phase in all the average measurement phases as a seed point, traversing all the average measurement phases, finding the pixel points with the same average measurement phase as the center point measurement phase, and obtaining the target pixel coordinates of all the found pixel points; adjusting the delay value by a preset step, and after each adjustment of the delay value, capturing a preset number of current measurement phases, obtaining the current measurement phases with the same pixel coordinates as the target pixel coordinates, and averaging them as the current center point measurement phase corresponding to the current delay value; and after completing the delay value adjustment and the center point measurement phase acquisition of a preset period, generating a mapping table of measurement phases and real phases, and completing the wiggling calibration.

[0008] Optionally, the step of generating the mapping table of measurement phases and real phases further comprises: increasing the pixel points by linear interpolation to supplement the mapping table.

[0009] To achieve the above object, the application further provides a calibration device of a ToF depth camera, comprising: a first acquisition module, configured to acquire, at an initial delay value, average measurement phases by time domain averaging a first preset number of times for each measurement phase in an image sensor of the ToF depth camera, so as to remove random noise; a second acquisition module, configured to take a center point measurement phase in all the average measurement phases as a seed point, traverse all the average measurement phases, find pixel points with the same average measurement phase as the center point measurement phase, and acquire target pixel coordinates of all the found pixel points; a third acquisition module, configured to adjust the delay value by a preset step, and after each adjustment of the delay value, acquire current measurement phases of a preset frame, acquire current measurement phases with the same pixel coordinates as the target pixel coordinates, and average the current measurement phases, so as to take the average measurement phases as current center point measurement phases corresponding to the current delay value; and a generation module, configured to generate a mapping table of measurement phases and real phases after completing the delay value adjustment and the center point measurement phase acquisition of a preset period, and complete wiggling calibration.

[0010] To achieve the above object, the application further provides an electronic device, comprising a memory, a processor, and a computer executable program stored in the memory and executable on the processor, wherein the processor executes the computer executable program to implement the steps of the calibration method of the ToF depth camera.

[0011] The application saves calibration time and reduces calibration cost by replacing time domain with space domain, solves the technical problem of long calibration time and high calibration cost of the existing ToF depth camera calibration, and further matches the real wiggling error in a linear interpolation manner, avoiding residual error caused by the existing fitting manner. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 Fig. 1 is a schematic diagram of the working principle of ToF;

[0014] Figure 2 Fig. 2 is a wiggling error existing in the measurement process;

[0015] Figure 3 Fig. 3 is a schematic diagram of wiggling calibration of a moving calibration board;

[0016] Figure 4A schematic diagram of wiggling calibration by simulating a virtual calibration board through a time delay circuit;

[0017] Figure 5 A schematic diagram of steps of a calibration method of a ToF depth camera provided by an embodiment of the present application;

[0018] Figure 6 A schematic diagram of a found pixel point set provided by an embodiment of the present application;

[0019] Figure 7 A schematic diagram of measurement of a phase and a real phase mapping relationship provided by an embodiment of the present application;

[0020] Figure 8 A schematic diagram of a linear interpolation method provided by an embodiment of the present application;

[0021] Figure 9 A flowchart of a calibration method of a ToF depth camera provided by an embodiment of the present application;

[0022] Figure 10 A structural block diagram of a calibration device of a ToF depth camera provided by an embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0024] Generally, the phase can be calculated by using a four-phase method, and by calculating the phase difference between the emitted light and the reflected light, the depth information of the measured target object from the ToF depth camera can be obtained.

[0025] The four-phase method for calculating the phase can use the following calculation formula:

[0026] phase=arctan[(Q3-Q4) / (Q1-Q2)];

[0027] wherein Q1 is the phase when the phase delay is 0°, Q2 is the phase when the phase delay is 90°, Q3 is the phase when the phase delay is 180°, and Q4 is the phase when the phase delay is 270°.

[0028] The following calculation formula can be used to obtain the depth according to the phase:

[0029] d=c*[phase / (2*f*phase_max)]+d_max*n;

[0030] wherein c is the speed of light, phase is the phase, phase_max is one phase period (e.g. 2π), f is the modulation frequency of the emitted light, d_max = (c / 2)*(1 / f), n is a preset frame number and n∈[0, 1, …].

[0031] Due to the difference between the actual light waveform and the ideal case, the measured phase and the ideal phase present a certain periodic variation rule, as shown in Figure 2 In order to solve the technical problems of long calibration time and high calibration cost of the existing ToF depth camera, the present application saves the calibration time and reduces the calibration cost by replacing the time domain with the spatial domain based on the principle analysis of the random noise of the ToF depth camera; when considering the swing error model, the scanning step is reduced and the linear interpolation method is used to replace the existing fitting method, so that the mapping table of the generated measured phase and the real phase is more matched with the actual swing error of the image sensor. The following gives a detailed explanation.

[0032] An embodiment of the present application provides a calibration method of a ToF depth camera.

[0033] Please refer to Figures 5 to 8 , wherein, Figure 5 is a calibration method of a ToF depth camera provided by an embodiment of the present application, Figure 6 is a schematic diagram of a found pixel point set provided by an embodiment of the present application, Figure 7 is a schematic diagram of a measured phase and a real phase mapping relationship provided by an embodiment of the present application, Figure 8 is a schematic diagram of a linear interpolation method provided by an embodiment of the present application.

[0034] As shown in Figure 5 , the calibration method described in the embodiment includes the following steps: S1, at the initial delay value, performing time domain averaging on each measured phase in the image sensor of the ToF depth camera for a first preset number of times to obtain the corresponding average measured phase, so as to remove the random noise; S2, taking the center point measured phase in all the average measured phases as a seed point, traversing all the average measured phases, finding the pixel points with the same average measured phase as the center point measured phase, and obtaining the target pixel coordinates of all the found pixel points; S3, adjusting the delay value by a preset step, and after adjusting the delay value each time, grabbing the current measured phase of a preset frame, obtaining the current measured phase with the same pixel coordinates as the target pixel coordinates and averaging, to serve as the current center point measured phase corresponding to the current delay value; and S4, after completing the delay value adjustment and the center point measured phase acquisition of a preset period, generating a mapping table of the measured phase and the real phase, and completing the wiggling calibration.

[0035] Regarding step S1, at the initial delay value, a first preset number of time-domain averages of each measurement phase in the image sensor of the ToF depth camera are obtained to remove random noise. Specifically, the measurement phase of each pixel point in the image sensor of the ToF depth camera contains certain random noise. Since the transfer function of each pixel point is consistent, it converges to a fixed value after a preset number of time-domain averages. Therefore, the first preset number of time-domain averages of each measurement phase are obtained to remove random noise. The first preset number can be set according to the calibration accuracy requirement, and only a small number of frames are needed to obtain the phase after adjusting the delay, that is, the time domain is replaced by the spatial domain. Since the subsequent frame rate does not need to be set too high, the calibration time and cost can be saved, and additional errors caused by temperature drift can be prevented.

[0036] In some embodiments, the initial delay value is 0. That is, when the delay value is 0, the first preset number of time-domain averages of each measurement phase are obtained to remove random noise.

[0037] In some embodiments, after the first preset number of time-domain averages of each measurement phase in the image sensor, the random noise converges to a fixed value (for example, 0). Specifically, the obtained average measurement phase satisfies the following formula:

[0038] phase n，i = phase n + ε n，m ;

[0039] wherein phase n，i is the i-th measurement phase of the n-th pixel point, phase n is the theoretical measurement value of the n-th pixel point after removing random noise, ε n，m is the fixed value to which the random noise converges after m times of time-domain averaging. Preferably, based on the normal distribution of noise, ε n，m = 0.

[0040] As to step S2, a center point measurement phase is taken as a seed point, all the average measurement phases are traversed, pixel points with the same average measurement phase as the center point measurement phase are searched, and target pixel coordinates of all the searched pixel points are obtained. Specifically, after time domain averaging for a preset number of times, an average depth image can be formed or the average measurement phases can be stored (for example, in the form of a depth image, which facilitates subsequent searching based on the measurement phases); then, the center point measurement phase of the pixel center point at the initial delay value is taken as a seed point, the average depth image formed after averaging is traversed, all pixel points with the same average measurement phase as the center point measurement phase are searched, and target pixel coordinates <posx, posy> of all the searched pixel points are obtained. The searched pixel point set is as shown in Figure 6

[0041] In some embodiments, the searching of the pixel points with the same average measurement phase as the center point measurement phase and the obtaining of the target pixel coordinates of all the searched pixel points in step S2 further include: if it is determined that the number of the searched pixel points is less than a preset value, the center point measurement phase is adjusted, and the traversing is performed again to increase the number of the searched pixel points. For example, if the number of the searched pixel points with the same center point measurement phase is small, the center point measurement phase can be added or subtracted by 1, and the traversing is continued to obtain more pixel coordinates.

[0042] As to step S3, the delay value is adjusted by a preset step, the current measurement phases of a preset number of frames are captured after each adjustment of the delay value, the current measurement phases with the same pixel coordinates as the target pixel coordinates are obtained and averaged to serve as a current center point measurement phase corresponding to the current delay value. Specifically, the delay value can be increased by a preset step each time on the basis of the initial delay value; the obtaining of the current center point measurement phase is performed after each increase of the delay value, so that the wiggling calibration can be completed after scanning a whole period. The preset step can be any one of π / 4, π / 8 and π / 16; the preset period can be 2π. In order to improve the calibration accuracy, the preset step can also be a smaller value; the delay value adjustment step can be set according to the comprehensive consideration of the calibration accuracy and the calibration time.

[0043] In some embodiments, the preset number of frames is 1. That is, one frame of depth image is captured and all the current measurement phases of the frame of depth image are obtained after each adjustment of the delay value. Since only a small number of frames need to be captured for phase acquisition, the calibration time can be saved, the calibration cost can be reduced, and additional errors caused by temperature drift can be prevented.

[0044] ​In some embodiments, the step S3 of capturing the current measurement phase of the preset frame after adjusting the delay value each time, and obtaining the current measurement phase with the same pixel coordinate as the target pixel coordinate further comprises: 1) capturing the current measurement phase of the preset frame after adjusting the delay value each time and storing; and 2) searching for the current pixel point with the same pixel coordinate as the target pixel coordinate in the stored current measurement phase, and obtaining the corresponding current measurement phase. For example, 1 frame of depth image is captured after adjusting the delay value each time, and all the current measurement phases of the frame of depth image are obtained and stored; all the target pixel coordinates <posx, posy> obtained in the step S2 are searched in the stored current measurement phase, and the corresponding current pixel points are found respectively, each current pixel point corresponding to a measurement phase, which is the measurement phase of the pixel point output by the ToF depth camera and is a known value.

[0045] The implementation principle that the average of the measurement phases of the pixel points with the same measurement phase (i.e., the spatial domain average) can replace the average in the time domain is explained and described below.

[0046] The pixel point wiggling correction model can adopt formula (1) as shown:

[0047] phase real = wiggling_lut[phase measure ] - fppn i，j (1)

[0048] wherein phase real is a real phase corresponding to a real distance of a pixel point pixel(i, j), wiggling_lut is a wiggling calibration mapping table, phase measure is a measurement phase of the pixel point, and fppn i，j is an fppn error corresponding to the pixel point.

[0049] After m times of averaging, the measurement phase phase measure of the center point is taken as a seed point to search for a pixel point with the same measurement phase as the measurement phase of the center point in the averaged depth image. Since the measurement phase phase' measure of each pixel point searched is the same as the measurement phase phase measure of the center point, the wiggling value of the pixel point is the same as the wiggling value of the center point, i.e.:

[0050] wiggling_lut[phase' measure ] = wiggling_lut[phase measure ].

[0051] The found pixel point wiggling correction model satisfies formula (2) :

[0052] phase' real = wiggling_lut[phase measure ]- fppn' x，y (2)

[0053] Wherein, phase' real is the real distance corresponding to the real phase of the found pixel point pixel (x, y), and fppn' x，y is the fppn error corresponding to the pixel point.

[0054] Formula (1) - formula (2) can obtain formula (3) :

[0055] phase real - phase' real = fppn' x，y - fppn i，j (3)

[0056] Since the real distance increases the same offset by increasing a delay value through the delay circuit, the fppn values of the pixel points with the same pixel coordinates in the images obtained before and after the delay value is increased are also the same, i.e. fppn' i，j = fppn i，j ; therefore, formula (3) can obtain:

[0057] wiggling_lut[phase measure1 ] = wiggling_lut[phase' measure1 ] (4)

[0058] Wherein, wiggling_lut[phase measure1 ] is the real phase mapped by the measured phase phase measure1 before the delay value is increased, and wiggling_lut[phase' measure1 ] is the real phase mapped by the measured phase phase' measure1 after the delay value is increased.

[0059] Since the wiggling calibration mapping table is one-to-one mapped with the whole period and the real phase, formula (4) can obtain:

[0060] phase measure1 = phase' measure1 (5)

[0061] That is, by measuring phase average (i.e. spatial domain average) of the pixel points with the same measuring phase, the average in time domain can be replaced. By changing the delay value, when scanning a whole cycle, the wiggling calibration can be completed

[0062] Regarding step S4, after the delay value adjustment of the preset cycle and the center point measuring phase acquisition are completed, a mapping table of the measuring phase and the real phase is generated, and the wiggling calibration is completed. Specifically, according to the measuring phase average corresponding to one delay value, the real phase corresponding to each change of the delay value is calculated, and then the mapping table of the measuring phase and the real phase can be generated. The mapping relationship between the measuring phase and the real phase is shown in Figure 7 .

[0063] Since the adjustment step of the delay value may not cover all real phases, the application further increases the pixel points by using a linear interpolation method to supplement the mapping table (lut), so as to supplement a complete mapping table. Compared with the existing fitting method, the increased pixel points of the application are more consistent with the real wiggling error, avoiding the existence of individual outliers that cannot match the fitting model.

[0064] In some embodiments, the step of increasing the pixel points by using the linear interpolation method to supplement the mapping table further comprises: 1) calculating the slope k of the interval segment where the pixel points to be increased are located by using the following formula: k = (y j -y i ) / (x j -x i ), wherein x i , x j are the measuring phases at both ends of the interval segment, y i is the real phase corresponding to the measuring phase x i , and y j is the real phase corresponding to the measuring phase x j ; 2) calculating the intercept b by using any of the following formulas: y i =kx i +b or y j =kx j +b; and 3) calculating the real phase of the pixel points to be increased by using the following formula: y = kx + b, wherein y is the real phase to be calculated, x is the measuring phase of the pixel points to be increased, k is the slope, and b is the intercept. According to the obtained slope k and intercept b, the real phase of any pixel point in the interval segment corresponding to x i , x j can be quickly calculated according to y = kx + b. The linear interpolation method is shown in Figure 8As shown, the horizontal axis of the graph is the measured phase, and the vertical axis is the real phase. P represents a pixel to be added.

[0065] In some embodiments, the method further comprises, before generating the mapping table of the measured phase and the real phase: restoring the delay value of the image sensor to a default value. Since the delay value of the image sensor is changed during the calibration process, the image sensor is restored after the calibration is completed, so that the subsequent ToF depth camera can be used normally.

[0066] The following will be described in detail Figure 9 The calibration method of the ToF depth camera is further explained. The specific process of the embodiment is as follows: S91, when the delay value is 0, m depth images of the image sensor of the ToF depth camera and the existing measured phase are obtained, and m times of average are performed to obtain one depth image formed after the average; S92, according to the center point measured phase of the pixel center point of the depth image formed after the average, all the pixel points with the same average measured phase as the center point measured phase are searched in the depth image formed after the average, and the target pixel coordinates <posx, posy> of all the searched pixel points are obtained; S93, the delay value is increased by a preset step (for example, π / 8); S94, one frame of depth image is captured, and all the current measured phases of the frame of depth image are obtained. All the target pixel coordinates <posx, posy> are searched in the current measured phase, and the corresponding current pixel points are found respectively. Each current pixel point corresponds to a current measured phase. All the current measured phases are averaged to obtain the current center point measured phase corresponding to the current delay value; S95, it is determined whether the current delay value is less than or equal to the maximum delay value (delay<=max_delay_count; wherein max_delay_count is a preset period, for example, 2π). If yes, return to step S93 to continue to increase the delay value and subsequent operations. If the current delay value is greater than the maximum delay value, the calibration is completed, and step S96 is executed; S96, the delay value of the image sensor is restored to a default value; and S97, a mapping table of the measured phase and the real phase is generated.

[0067] According to the above, it can be seen that the present application saves the calibration time and reduces the calibration cost by replacing the time domain with the space domain, and further matches the real swing error in a linear interpolation manner, thereby solving the technical problems of the existing ToF depth camera calibration, such as long calibration time, high calibration cost, and residual error caused by the fitting manner.

[0068] Based on the same inventive concept, the present application also provides a calibration device for a ToF depth camera. The calibration device for the ToF depth camera can adopt the calibration method of the ToF depth camera as described above. Figure 5The calibration method of the ToF depth camera shown completes calibration of the ToF depth camera.

[0069] Referring to Figure 10 , which is a structural block diagram of a calibration device of a ToF depth camera according to an embodiment of the present application. As shown in Figure 10 , the calibration device of the ToF depth camera comprises a first acquisition module 101, a second acquisition module 102, a third acquisition module 103 and a generation module 104.

[0070] Specifically, the first acquisition module 101 is configured to, at an initial delay value, acquire a corresponding average measurement phase by performing time-domain average on each measurement phase in the image sensor of the ToF depth camera for a first preset number of times, so as to remove random noise. The second acquisition module 102 is configured to, taking a center point measurement phase in all the average measurement phases as a seed point, traverse all the average measurement phases, find pixel points with the same average measurement phase as the center point measurement phase, and acquire target pixel coordinates of all the found pixel points. The third acquisition module 103 is configured to adjust the delay value by a preset step, and after each adjustment of the delay value, acquire current measurement phases of a preset frame, acquire current measurement phases with the same pixel coordinates as the target pixel coordinates, and average the current measurement phases, so as to obtain a current center point measurement phase corresponding to the current delay value. The generation module 104 is configured to, after completing adjustment of the delay value and acquisition of the center point measurement phase for a preset period, generate a mapping table of measurement phases and real phases, and complete wiggling calibration.

[0071] Based on the same inventive concept, the present application further provides an electronic device comprising a memory, a processor and a computer executable program stored in the memory and executable on the processor; the processor executes the computer executable program to implement steps of the calibration method of the ToF depth camera shown in Figure 5 .

[0072] In the field of the inventive concept are embodiments that can be described and explained in terms of modules that execute one or more functions described. These modules, which can also be referred to herein as units, among others, can be implemented in physical hardware, such as analog and / or digital circuits, for example, logic gates, integrated circuits, microprocessors, microcontrollers, memory circuits, passive electronic components, active electronic components, optical components, hardwired circuits, and the like, and can optionally be driven by firmware and / or software. The circuits, for example, can be implemented in one or more semiconductor chips. Circuits constituting a module can be implemented by dedicated hardware, or by a processor, such as one or more programmed microprocessors and associated circuitry, or by a combination of dedicated hardware and a processor implementing some functions of the module. Each module of an embodiment can be physically separated into two or more interacting and discrete modules without departing from the scope of the inventive concept. Likewise, modules of an embodiment can be physically combined into a more complex module without departing from the scope of the inventive concept.

[0073] In general, terms can be understood to be contextually defined. For example, the term "one or more," as used herein, can be used in either a singular sense or a plural sense depending on the context in which the term is used. Additionally, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but instead, as allowing for existence of other factors that are not necessarily explicitly described. Also, the term "based on" can be understood as not necessarily intended to convey an exclusive set of factors, but instead, as allowing for existence of other factors that are not necessarily explicitly described.

[0074] It should be noted that the terms "comprises" and "comprising" and variations thereof in the specification and claims are not intended to exclude any feature, component, or step from the methods or compositions described. The terms "first," "second," and the like, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. Also, the terms "a" and "an" and "the" and similar reference used in the context of an embodiment do not exclude a plurality, and are used herein as shorthand notation for one or more unless otherwise indicated. Further, the use of "including," "containing," or "comprising" and variations thereof throughout the specification and claims, does not exclude the presence of other elements or steps. Furthermore, the description and drawings are to be regarded as illustrative in nature and that variations are to be regarded as being within the scope of the application. Moreover, descriptions of a particular embodiment or embodiments does not indicate that the application is limited to only that particular embodiment or embodiments.

[0075] The above description is only the preferred embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as falling within the scope of the present application.

Claims

1. A method of calibrating a ToF depth camera, the method comprising: The method comprises the following steps: acquiring a corresponding average measurement phase by performing a first preset number of time domain averages on each measurement phase in an image sensor of a ToF depth camera at an initial delay value, to remove random noise; taking a center point measurement phase in all the average measurement phases as a seed point, traversing all the average measurement phases, finding pixel points with the same average measurement phase as the center point measurement phase, and acquiring target pixel coordinates of all the found pixel points; adjusting the delay value by a preset step size, and after each adjustment of the delay value, acquiring current measurement phases of a preset frame, acquiring current measurement phases with the same pixel coordinates as the target pixel coordinates, and averaging the current measurement phases to obtain a current center point measurement phase corresponding to the current delay value; and generating a mapping table of measurement phases and real phases after completing delay value adjustment and center point measurement phase acquisition in a preset period, to complete wiggling calibration.

2. The method of claim 1, wherein, The initial delay value is 0.

3. The method of claim 1, wherein, After the first preset number of time domain averages, the random noise converges to a fixed value.

4. The method of claim 3, wherein, The acquired average measurement phase satisfies the following formula: phase n，i = phase n + ε n，m ; wherein phase n，i is the phase of the i-th measurement of the n-th pixel, phase n is the theoretical measurement value of the n-th pixel after removing the random noise, ε n，m is a fixed value to which the random noise converges after m times of time domain averaging.

5. The method of claim 1, wherein, The step of finding pixel points with the same average measurement phase as the center point measurement phase and acquiring target pixel coordinates of all the found pixel points further comprises: If it is determined that the number of found pixel points is less than a preset value, adjusting the center point measurement phase and traversing again to increase the number of found pixel points.

6. The method of claim 1, wherein, The preset frame is 1 frame; the preset step size is any one of π / 4, π / 8, and π / 16; and the preset period is 2π.

7. The method of claim 1, wherein, The step of acquiring current measurement phases of a preset frame after each adjustment of the delay value and acquiring current measurement phases with the same pixel coordinates as the target pixel coordinates further comprises: acquiring current measurement phases of a preset frame after each adjustment of the delay value and storing the current measurement phases; and finding current pixel points with the same pixel coordinates as the target pixel coordinates in the stored current measurement phases and acquiring corresponding current measurement phases.

8. The method of claim 1, wherein, The step of generating a mapping table of measurement phases and real phases further comprises supplementing the mapping table by increasing pixel points in a linear interpolation manner.

9. The method of claim 8, wherein, The step of supplementing the mapping table by increasing pixel points in a linear interpolation manner further comprises: calculating a slope k of an interval segment in which a pixel point to be added is located by using the following formula: k = (y j - y i ) / (x j - x i ), wherein x i , x j are the measured phases at the ends of the interval segment, y i is the true phase corresponding to the measured phase x i , and y j is the true phase corresponding to the measured phase x j . calculating an intercept b by using any one of the following formulas: y i = kx i + b or y j = kx j + b; calculating a real phase of the pixel point to be added by using the following formula: y=kx+b, where y is the real phase to be calculated, x is the measurement phase of the pixel point to be added, k is the slope, and b is the intercept.

10. The method of claim 1, wherein, The method further comprises the following step before generating a mapping table of measurement phases and real phases: restoring the delay value of the image sensor to a default value.

11. A device for calibrating a ToF depth camera, characterized in that The method comprises the following steps: a first acquisition module is configured to acquire a corresponding average measurement phase by performing a first preset number of time domain averages on each measurement phase in an image sensor of a ToF depth camera at an initial delay value, to remove random noise; a second obtaining module, configured to take a center point measurement phase in all the average measurement phases as a seed point, traverse all the average measurement phases, find pixel points with the same average measurement phase as the center point measurement phase, and obtain target pixel coordinates of all the found pixel points; a third obtaining module, configured to adjust the delay value by a preset step, and after each adjustment of the delay value, capture a preset frame of current measurement phases, obtain current measurement phases with the same pixel coordinates as the target pixel coordinates, and average the current measurement phases to obtain a current center point measurement phase corresponding to the current delay value; and a generating module, configured to generate a mapping table of measurement phases and real phases after completing the adjustment of the delay value and the obtaining of the center point measurement phase for a preset period, and complete wiggling calibration.

12. An electronic device comprising a memory, a processor, and a computer- executable program stored on the memory and executable on the processor, wherein, The processor implements the steps of the calibration method of the ToF depth camera according to any one of claims 1-10 when executing the computer executable program.

Images