Time-of-flight camera calibration method, device and system and storage medium

Abstract

The invention belongs to the technical field of time-of-flight cameras, and provides a time-of-flight camera calibration method, device and system and a storage medium. The method comprises the following steps: acquiring a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions arranged at equal intervals; calculating a measurement phase based on the plurality of diffuse reflection target images; obtaining the distance between the time-of-flight camera and the diffuse reflection target at each position; calculating a real phase according to the distance; according to the measurement phase and the real phase, a fixed template noise phase is obtained through calculation; generating a depth calibration file according to the fixed template noise phase; and writing the depth calibration file into the flight time camera to complete calibration. According to the method provided by the invention, the depth calibration file with higher precision can be obtained, internal errors of different TOF camera systems can be eliminated, and the accuracy and consistency of depth measurement of each produced TOF camera are ensured.

Description

Technical Field

[0001] The present application belongs to the technical field of time-of-flight cameras, and in particular, relates to a calibration method, device, system and storage medium for a time-of-flight camera. Background Art

[0002] With the demand for depth measurement in the industrial field, depth cameras based on TOF (Time of Flight) technology are widely used. The principle is that the transmitter of the TOF camera (TOF camera) transmits a modulated single pulse or continuous pulse signal, and the image sensor of the TOF camera receives the modulated signal reflected by the object, and calculates the current distance of the object through the phase difference between the transmitted signal and the received signal. In the production process of the TOF camera, the calibration method of the TOF camera is particularly important, which determines the accuracy of the depth measurement.

[0003] The depth calibration file obtained by the existing time-of-flight camera calibration method has low accuracy and cannot guarantee the accuracy and consistency of the measurement of each time-of-flight camera. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a calibration method, device, system and storage medium for a time-of-flight camera to solve the problem that the depth calibration file obtained using the traditional calibration method has low accuracy and cannot guarantee the accuracy and consistency of the measurement of each time-of-flight camera.

[0005] A first aspect of an embodiment of the present application provides a calibration method for a time-of-flight camera, comprising:

[0006] Acquire multiple diffuse reflection target images acquired by a time-of-flight camera at multiple positions set at equal intervals;

[0007] Calculating a measurement phase based on the multiple diffuse reflection target images;

[0008] Obtaining the distance between the time-of-flight camera and the diffuse reflection target at each of the positions;

[0009] Calculate the real phase according to the distance;

[0010] Calculate a fixed template noise phase according to the measured phase and the true phase;

[0011] Generate a depth calibration file according to the fixed template noise phase;

[0012] The depth calibration file is written into the time-of-flight camera to complete the calibration.

[0013] A second aspect of an embodiment of the present application provides a calibration device for a time-of-flight camera, comprising:

[0014] A communication module, used to obtain a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions set at equal intervals;

[0015] A measurement phase calculation module, used for calculating the measurement phase based on the multiple diffuse reflection target images;

[0016] A distance acquisition module, used for acquiring the distance between the time-of-flight camera and the diffuse reflection target at each position;

[0017] A real phase calculation module, used to calculate the real phase according to the distance;

[0018] A fixed template noise phase calculation module, used to calculate the fixed template noise phase according to the measured phase and the true phase;

[0019] A calibration file generating module, used for generating a depth calibration file according to the fixed template noise phase;

[0020] The writing module is used to write the depth calibration file into the time-of-flight camera to complete the calibration.

[0021] A third aspect of an embodiment of the present application provides a calibration system for a time-of-flight camera, including a calibration device, a camera fixing and adjusting device, a focus target, and a diffuse reflection target;

[0022] The time-of-flight camera is fixed on the camera fixing and adjusting device, the focus target or diffuse reflection target is arranged toward the lens of the time-of-flight camera, and the optical axis of the time-of-flight camera passes through the center of the focus target or diffuse reflection target, and the calibration device is connected to the time-of-flight camera and the camera fixing and adjusting device;

[0023] The calibration device is used to control the camera fixing and adjusting device to move to a plurality of positions set at equal intervals;

[0024] The time-of-flight camera is used to collect image data of the focus target or the diffuse reflection target and output it to the calibration device;

[0025] The calibration device is used to execute a calibration method for a time-of-flight camera provided in the first aspect of the embodiment of the present application.

[0026] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the calibration method provided in the first aspect of the embodiment of the present application.

[0027] The first aspect of the embodiment of the present application provides a calibration method for a time-of-flight camera, which is obtained by acquiring multiple diffuse reflection target images collected by the time-of-flight camera at multiple positions set at equal intervals; calculating the measured phase based on the multiple diffuse reflection target images; obtaining the distance between the time-of-flight camera and the diffuse reflection target at each position; calculating the true phase based on the distance; calculating the fixed template noise phase based on the measured phase and the true phase; generating a depth calibration file based on the fixed template noise phase; writing the depth calibration file into the time-of-flight camera to complete the calibration. A higher-precision depth calibration file can be obtained, and internal errors of different TOF camera systems can be eliminated to ensure the accuracy and consistency of depth measurement of each TOF camera produced.

[0028] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0030] Figure 1 It is a flow chart of a calibration method provided by an embodiment of the present application;

[0031] Figure 2 It is a flow chart of a calibration method provided by an embodiment of the present application;

[0032] Figure 3 is a flow chart of a calibration method provided by another embodiment of the present application;

[0033] Figure 4 is a flow chart of a calibration method provided by another embodiment of the present application;

[0034] Figure 5 This is a schematic diagram of a focused target image obtained by acquisition provided by an embodiment of the present application;

[0035] Figure 6 This is a schematic diagram of a focus target image with the highest definition provided by an embodiment of the present application;

[0036] Figure 7 is a schematic diagram of the structure of a calibration device provided in an embodiment of the present application;

[0037] Figure 8It is a structural diagram of the calibration system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0039] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0040] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0041] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0042] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0043] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0044] The calibration method of the time-of-flight camera provided in the embodiment of the present application can be executed by the control module of the calibration device of the time-of-flight camera when running a computer program with corresponding functions, by acquiring multiple diffuse reflection target images collected by the time-of-flight camera at multiple positions set at equal intervals; calculating the measured phase based on the multiple diffuse reflection target images; acquiring the distance between the time-of-flight camera and the diffuse reflection target at each position; calculating the true phase according to the distance; calculating the fixed template noise phase according to the measured phase and the true phase; generating a depth calibration file according to the fixed template noise phase; writing the depth calibration file into the time-of-flight camera to complete the calibration. A higher-precision depth calibration file can be obtained, and the internal errors of different TOF camera systems can be eliminated, ensuring the accuracy and consistency of the depth measurement of each TOF camera produced.

[0045] In applications, the control module may be a processor, and the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0046] In the application, the calibration device can be a computing device with at least one function of image processing, distance measurement, image analysis, camera control, motor control, etc., for example, a robot, a mobile phone, a personal digital assistant (PDA), a self-service terminal, a camera, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a tablet computer, a laptop computer, a personal computer (PC), a netbook, a server, a machine vision system, a smart camera, etc. The embodiments of the present application do not impose any restrictions on the specific type of the calibration device.

[0047] like Figure 1 As shown, the calibration method of the time-of-flight camera provided in the embodiment of the present application includes the following steps S101 to S107:

[0048] Step S101 : acquiring a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions set at equal intervals.

[0049] In applications, diffuse reflection targets are usually composed of patterns of different gray levels, which maintain consistent reflection characteristics under different lighting conditions. The purpose of this is to ensure the consistency and repeatability of the image, thereby facilitating the subsequent calibration process. The multiple diffuse reflection target image data in this step are obtained by placing the time-of-flight camera at different positions (n ​​positions can be set, n is a positive integer greater than 1), and ensuring that there are equal distances between these positions, and then collecting a series of diffuse reflection target images at different positions.

[0050] In the application, multiple frames of images can be collected at each position, such as 20 frames, 30 frames, etc., and then the average grayscale value or color value of each pixel in all 20 or 30 frames of images is calculated to generate a new image. Each pixel value of this image is the average value of the corresponding pixel values ​​of 20 or 30 frames of images. If the time-of-flight camera outputs a grayscale image, the grayscale value of each pixel is averaged; if it is a color image, the RGB three channels of each pixel are averaged separately. This method can improve the signal-to-noise ratio of the image.

[0051] In the application, the diffuse reflection target uses a diffuse reflection target with high diffuse reflectivity, and the high diffuse reflectivity means that the diffuse reflectivity is greater than or equal to 80%, such as 80%, 85%, 90%, 95%, etc. The diffuse reflection target with high diffuse reflectivity can be any one of a uniform grayscale target with a single color or grayscale level and a reflectivity greater than or equal to 80%, a geometric shape target containing a series of geometric figures of known sizes (such as circles, squares, etc.), and these figures have a reflectivity greater than or equal to 80%, and a multi-grayscale target composed of multiple areas with different grayscale levels, and the reflectivity of each area is greater than or equal to 80%. The embodiment of the present application does not limit the specific form of the diffuse reflection target.

[0052] In the application, this step ensures that the collected data covers the effective working range of the camera and can capture the performance changes at different distances. By collecting data at equally spaced locations, evenly distributed sample points can be obtained, which helps to fully understand the behavior characteristics of the camera throughout the entire working range.

[0053] Step S102: Calculate and obtain a measurement phase based on a plurality of diffuse reflection target images.

[0054] In the application, by analyzing the collected multiple diffuse reflection target images, the phase difference at each pixel point, that is, the measured phase, can be calculated. The measured phase can be calculated by any one of the methods such as inverse tangent calculation, Fourier transform method, phase unwrapping algorithm, etc.

[0055] In the application, the measurement phase is calculated based on multiple diffuse reflection target images, including using inverse tangent calculation to obtain the measurement phase based on the multiple diffuse reflection target images. The formula is as follows:

[0056]

[0057] By using this formula, the measured phase data set can be calculated Among them, I0,I 90 ,I 180 ,I 270 They are respectively the signal strength collected when the phase is the same as the transmitted signal, the signal strength collected when the phase is 90 degrees different from the transmitted signal, the signal strength collected when the phase is 180 degrees different from the transmitted signal, and the signal strength collected when the phase is 270 degrees different from the transmitted signal.

[0058] Step S103: Obtain the distance between the time-of-flight camera and the diffuse reflection target at each position.

[0059] In the application, when calibrating, a measuring tool with known precise size (such as any one of ultrasonic sensors, infrared ranging sensors, and laser ranging sensors) can be used to obtain the actual distance d1, d2, d3…d between the time-of-flight camera and the diffuse reflection target at each position. n , and record it. Where d1 is the actual distance between the time-of-flight camera and the diffuse reflection target at the first position...d n is the actual distance between the time-of-flight camera and the diffuse reflection target at the nth position.

[0060] Step S104: Obtain the real phase according to the distance calculation.

[0061] In the application, the formula Calculate the true phase, where d is the distance, c is the speed of light, and f is the modulation frequency of the transmitted signal. According to this calculation formula, the true phase data set can be obtained

[0062]

[0063] Step S105: Calculate the fixed template noise phase according to the measured phase and the true phase.

[0064] In applications, there is usually a certain difference between the measured phase and the true phase, which is mainly caused by system noise and other non-ideal factors. A statistical method (such as least squares method) can be used to fit the deviation between the true phase and the measured phase to obtain a phase deviation template that characterizes the influence of noise, that is, a fixed template noise phase. In this step, by comparing the difference between the measured phase and the true phase, a fixed phase deviation template can be calculated. This template can reflect the systematic deviation of the camera at different distances, thereby providing a calibration basis for subsequent depth measurement.

[0065] Step S106: Generate a depth calibration file according to the fixed template noise phase.

[0066] In the application, the fixed template noise phase at each pixel position is organized into a format suitable for time-of-flight camera calibration. For example, the phase error data is converted into a two-dimensional array, where each element corresponds to a pixel of the camera. The organized data is saved as a file with a structure consistent with the format specified by the time-of-flight camera manufacturer, such as .txt or .csv format, which is the depth calibration file.

[0067] Step S107: Write the depth calibration file into the time-of-flight camera to complete the calibration.

[0068] In applications, the calibration method of the time-of-flight camera, from data acquisition to calculation to final application, can obtain a more accurate depth calibration file, which can eliminate the internal errors of different TOF camera systems, ensure the accuracy and consistency of the depth measurement of each TOF camera produced, and ensure that the time-of-flight camera can obtain accurate and reliable depth measurement results at various distances. This method not only improves the performance of the camera itself, but also simplifies the user's operation process, allowing users to obtain high-quality measurement data without manual adjustment. In addition, this calibration method based on the difference between the true phase and the measured phase can effectively reduce the measurement errors caused by system noise and other non-ideal factors, and enhance the robustness and adaptability of the camera in actual application scenarios.

[0069] In one embodiment, Figure 2 As shown, step S105 includes the following steps S201 and S202:

[0070] Step S201: Fit the measured phase and the real phase to obtain the harmonic error.

[0071] Step S201: Calculate a fixed template noise phase according to the harmonic error, the measured phase and the true phase.

[0072] In applications, the relationship between the measured phase and the true phase can be fitted by mathematical modeling techniques, such as the least squares method or other regression analysis methods. Assuming that there is a certain periodic error pattern between the measured phase and the true phase, one or more harmonic functions can usually be used to approximate this periodic error to obtain a mathematical expression that can describe the difference between the measured phase and the true phase, that is, the harmonic error. For example, a fitting model can be constructed using polynomial fitting or other nonlinear fitting methods. The fitting model contains undetermined coefficients. Through multiple measured phases and true phase data, the optimal parameters are determined using fitting tools (such as MATLAB or Python's scipy.optimize) or other methods, so that the undetermined coefficients in the fitting model can be determined, that is, the harmonic error is obtained.

[0073] In application, the fixed template noise phase is calculated as follows:

[0074]

[0075] in, is the true phase, To measure the phase, is a fixed template noise phase, is the harmonic error.

[0076] In one embodiment, Figure 3 As shown, before step S101, the following steps S301 to S303 are included:

[0077] Step S301: Control the time-of-flight camera to turn on, and preheat the time-of-flight camera for a specified time.

[0078] In the application, the time-of-flight camera can be controlled to turn on and preheat to a specified time, such as 5 minutes, 10 minutes, etc. This is to ensure that the internal components of the camera reach a stable operating temperature, thereby improving the accuracy and consistency of the measurement.

[0079] Step S302: setting the exposure time of the time-of-flight camera.

[0080] In the application, according to the calibration environment and the characteristics of the diffuse reflection target, a suitable exposure time can be selected to obtain the best image quality and measurement effect, such as 6us to 4000us.

[0081] Step S303: Control the time-of-flight camera to move to a plurality of positions set at equal intervals in sequence.

[0082] In application, the time-of-flight camera can be fixed on a movable carrier, and the movable carrier can be provided with components such as rollers or sliders that can cooperate with the slide rails, and the rollers or sliders can be driven by a motor. The encoder is installed on the motor shaft or on the roller to accurately measure the moving distance and speed. By outputting different control signals to the control signal input terminal of the motor, the number of rotations of the motor is controlled, so that the time-of-flight camera can be moved to multiple positions set at equal intervals in sequence. A track can also be set, and multiple proximity switches or photoelectric sensors can be set at preset equal intervals on the track. When the output signal of the proximity switch or photoelectric sensor is received, the movable carrier is stopped, so that the time-of-flight camera can be moved to multiple positions set at equal intervals in sequence.

[0083] In one embodiment, before step S101, the method further includes:

[0084] Determine the focus position and control the lens of the time-of-flight camera to move to the focus position;

[0085] The focus position is the lens position where the time-of-flight camera can obtain the highest definition image.

[0086] In application, the lens drive motor can be stopped to stop the lens at the focus position. To further ensure the accuracy of the focus position, industrial glue can be used to fix the movable position of the lens.

[0087] In the application, the focus position can be determined by analyzing the image clarity, which can be used as the lens position for depth calibration. Before depth calibration, the position with the highest image clarity is selected as the lens position for depth calibration to ensure that the camera can obtain the most accurate image details during depth calibration. The clearest image means that the camera can capture the finest features on the focus target. By performing depth calibration at the best focus position, it can be ensured that the camera can more accurately capture the details of the target object in subsequent depth measurements, thereby improving the overall measurement accuracy and reliability.

[0088] In one embodiment, Figure 4 As shown, controlling the lens of the time-of-flight camera to move to a focus position includes the following steps S401 to S405:

[0089] Step S401: Control the time-of-flight camera to move until the distance between the camera and the focus target is equal to the hyperfocal distance of the time-of-flight camera.

[0090] In the application, According to the calculation formula, combined with the parameters of the time-of-flight camera, such as focal length, diffuse aperture diameter, and aperture value, the hyperfocal distance can be calculated in advance. In conjunction with the ranging device and the movable carrier, the time-of-flight camera can be moved to a position where the distance between the time-of-flight camera and the focusing target is equal to the hyperfocal distance of the time-of-flight camera.

[0091] In applications, the focus target can adopt any one of a checkerboard pattern, a Butler pattern, a Siemens star, a USAF 1951 resolution card, a random pattern, etc. The focus target is a self-luminous target that emits light in the 940nm band.

[0092] Step S402: Control the lens of the time-of-flight camera to move to different lens adjustment positions.

[0093] In application, the lens of the time-of-flight camera can move smoothly back and forth along the axis driven by the lens drive motor. The lens drive motor can receive a control signal to realize the lens drive in a specified direction and a specified distance.

[0094] Step S403: Obtain the focus target image acquired by the time-of-flight camera at different lens adjustment positions, such as Figure 5 shown.

[0095] Step S404: Analyze the clarity of the focused target image to obtain the focused target image with the highest clarity, such as Figure 6 shown.

[0096] In the application, the clarity of the focused target image can be analyzed by any method including Laplace operator, image entropy, structural similarity index, spectrum analysis, gradient intensity, variance after Gaussian filtering, etc.

[0097] Step S405: taking the lens adjustment position corresponding to the focus target image with the highest definition as the focus position, and stopping the lens at the focus position.

[0098] In one embodiment, step S404 is followed by steps S501 and S502:

[0099] Step S501, calculating the sum of squares of grayscale gradients of the focused target image;

[0100] Step S502: taking the focused target image with the largest grayscale gradient square sum as the focused target image with the highest definition.

[0101] In applications, the grayscale gradient of an image refers to the rate of change of the grayscale value of pixels in the image. The grayscale gradient is larger in areas with edges and rich details. High-definition images usually have a larger sum of squared grayscale gradients because they contain more details and edges. This step calculates the grayscale gradient of each pixel in the image and sums the squares of these gradient values ​​to obtain the sum of squared grayscale gradients, which can determine the in-focus target image with the highest definition.

[0102] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0103] The embodiment of the present application also provides a calibration device for a time-of-flight camera, which is used to execute the steps in the embodiment of the calibration method for a time-of-flight camera.

[0104] like Figure 7 As shown, the calibration device 100 of a time-of-flight camera provided in an embodiment of the present application includes:

[0105] The communication module 101 is used to obtain a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions set at equal intervals; and

[0106] The control module 102 is connected to the communication module and is used to calculate the measured phase based on the multiple diffuse reflection target images; obtain the distance between the time-of-flight camera and the diffuse reflection target at each position; calculate the real phase according to the distance; calculate the fixed template noise phase according to the measured phase and the real phase; and generate a depth calibration file according to the fixed template noise phase;

[0107] The communication module is further used to write the depth calibration file into the time-of-flight camera to complete the calibration when communicating with the time-of-flight camera.

[0108] In applications, the communication connection can be a wired connection or a wireless connection.

[0109] In one embodiment, the control module is further configured to:

[0110] Controlling the time-of-flight camera to turn on and preheating the time-of-flight camera for a specified period of time;

[0111] Setting the exposure time of the time-of-flight camera;

[0112] The time-of-flight camera is controlled to move to a plurality of positions arranged at equal intervals in sequence.

[0113] In one embodiment, the control module is further configured to:

[0114] Control the lens of the time-of-flight camera to move to the focus position;

[0115] The focus position is the lens position where the time-of-flight camera can obtain the highest definition image.

[0116] In applications, each module in the calibration device of the time-of-flight camera may be a software program module, or may be implemented by different logic circuits integrated in a processor, or may be implemented by multiple distributed processors.

[0117] The embodiment of the present application also provides a calibration system for a time-of-flight camera, including a calibration device 100, a camera fixing and adjusting device, a focus target, and a diffuse reflection target;

[0118] The time-of-flight camera is fixed on the camera fixing and adjusting device, the focus target or the diffuse reflection target is arranged toward the lens of the time-of-flight camera, and the optical axis of the time-of-flight camera passes through the center of the focus target or the diffuse reflection target, and the calibration device is connected to the time-of-flight camera and the camera fixing and adjusting device;

[0119] The calibration device is used to control the camera fixing and adjusting device to move to a plurality of positions set at equal intervals;

[0120] The time-of-flight camera is used to collect image data of the focus target or the diffuse reflection target and output it to the calibration device;

[0121] The calibration device is used to execute the calibration method of any of the above embodiments.

[0122] In the application, the calibration device is the control center of the system, communicating with the time-of-flight camera to control the entire calibration process. In addition, it also communicates with the controlled components such as the time-of-flight camera and the camera fixing and adjusting device to control the controlled components to perform corresponding actions. It can interact with the user through the software interface and communicate with the camera fixing and adjusting device through the interface to accurately control the position of the time-of-flight camera.

[0123] In the application, the camera fixing and adjusting device is used to fix the time-of-flight camera and can accurately adjust the position of the camera, so as to collect images at different distances by moving the camera to different positions for depth calibration. The time-of-flight camera is fixed on the camera fixing and adjusting device and can be accurately moved to different positions. The camera fixing and adjusting device can rotate along the horizontal plane and / or rotate up and down (i.e., pitch angle adjustment) to adjust the angle of the camera.

[0124] In one embodiment, a target switching device is further included, and a control signal output terminal of the calibration device is connected to a control signal input terminal of the target switching device;

[0125] The calibration device is also used to send a focus target switching signal to the target switching device, or send a diffuse reflection target switching signal to the target switching device;

[0126] The target switching device is used to switch to the focus target based on the focus target switching signal, or to switch to the diffuse reflection target based on the diffuse reflection target switching signal.

[0127] In application, the target switching device can be a column with a bottom connected to the drive motor shaft, and the diffuse reflection target and the focus target are fixed on both sides of the column facing each other, so that the switch between the diffuse reflection target and the focus target can be achieved by controlling the rotation of the drive motor. The diffuse reflection target and the focus target can also be fixed by a connecting rod respectively, and the connecting rod is directly distributed at 180 degrees. The end of the connecting rod is connected to the motor drive shaft in the radial direction. When the motor rotates to 0°, it can switch to one target, and when the motor continues to rotate 180 degrees, it can switch to another target.

[0128] In one embodiment, the camera fixing and adjusting device is provided with a sliding assembly and a camera adjusting assembly, the sliding assembly is arranged at the bottom of the camera fixing and adjusting device, the camera adjusting assembly includes a pitch adjusting assembly and a horizontal rotation adjusting assembly, the time of flight camera is fixed to the camera fixing and adjusting device through the camera adjusting assembly, and the calibration device is communicatively connected with the sliding assembly and the camera adjusting assembly. The communication connection may be at least one of a wired connection and a wireless connection.

[0129] In the application, a level meter can also be provided, and the data output end of the level meter is connected to the calibration device, so that the output of the level meter can be used to determine whether the time-of-flight camera is level. Before focusing or calibrating the camera, the calibration device first controls the camera fixing and adjusting device to adjust at least one of the pitch angle, horizontal angle, height, etc., to ensure that the optical axis of the time-of-flight camera is aligned with the center of the target.

[0130] like Figure 8 As shown, in one embodiment, the calibration system of the time-of-flight camera 5 includes: a calibration device 100 (not shown in the figure) and an electric control track 1, a camera fixing and adjusting device 2, a roller 10, a slider 12, a rotating table 3, an angle displacement table 4, a level 6, a focusing target 8, a diffuse reflection target 9, and a column 11.

[0131] The calibration device 100 is connected to the drive motor of the electric control track 1, the lens drive motor of the time-of-flight camera 5, the drive motor of the rotating platform 3, the drive motor of the angle displacement platform 4, the level 6, and the drive motor of the column 11. The time-of-flight camera 5 includes a modulation signal transmitting end (VCSEL laser) and a modulation signal receiving end (sensor chip and lens combination).

[0132] In application, sliders 12 are provided on both sides of the camera fixing and adjusting device 2, which are threadedly matched with the electric control track 1. The electric control track 1 (which can be a lead screw) and the slider 12 are driven by a lead screw, so that when the driving motor of the electric control track 1 drives the electric control track 1 to rotate, the slider 12 can move along the lead screw, that is, the distance between the time-of-flight camera 5 and the target is controlled. The lens driving motor of the time-of-flight camera 5 can realize the smooth reciprocating movement of the lens along the axial direction by forward or reverse rotation under the control of the calibration device 100. The rotating table 3 is fixed to the top of the camera fixing and adjusting device 2. The driving motor of the rotating table 3 can drive the rotating table 3 to rotate along the horizontal plane under the control of the calibration device 100, thereby realizing the horizontal angle adjustment of the time-of-flight camera 5. The angle displacement table 4 is fixed to the top of the rotating table 3. The driving motor of the angle displacement table 4 can realize the upward or downward rotation of the angle displacement table 4 by forward or reverse rotation under the control of the calibration device 100, thereby realizing the pitch angle adjustment of the time-of-flight camera 5. The level 6 is arranged on the top of the time-of-flight camera 5, so that when adjusting the pitch angle, the calibration device 100 can judge whether the optical axis of the time-of-flight camera 5 is horizontal according to the data fed back by the level 6, and then further adjust the angle displacement stage 4 to realize the feedback adjustment of the pitch angle control. The diffuse reflection target 9 and the focus target 8 are fixed on both sides of the column 11 with their backs facing each other, so that the switch between the diffuse reflection target 9 and the focus target 8 can be realized by controlling the rotation of the driving motor.

[0133] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0134] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0135] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0136] An embodiment of the present application provides a computer program product. When the computer program product runs on a calibration device, the calibration device can implement the steps in the above-mentioned method embodiments when executed.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / calibration device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0138] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A calibration method for a time-of-flight camera, characterized in that: include: Acquire multiple diffuse reflection target images acquired by a time-of-flight camera at multiple positions set at equal intervals; Calculating a measurement phase based on the multiple diffuse reflection target images; Obtaining the distance between the time-of-flight camera and the diffuse reflection target at each of the positions; Calculate the real phase according to the distance; Calculate a fixed template noise phase according to the measured phase and the true phase; Generate a depth calibration file according to the fixed template noise phase; The depth calibration file is written into the time-of-flight camera to complete the calibration.

2. The calibration method of a time-of-flight camera according to claim 1, characterized in that: The step of calculating the measurement phase based on the plurality of diffuse reflection target images comprises: Based on the multiple diffuse reflection target images, the measurement phase is calculated using an inverse tangent calculation formula.

3. The calibration method of a time-of-flight camera according to claim 1, characterized in that: The step of calculating the fixed template noise phase according to the measured phase and the true phase includes: Fitting the measured phase and the true phase to obtain a harmonic error; The fixed template noise phase is calculated based on the harmonic error, the measured phase and the true phase.

4. The calibration method of a time-of-flight camera according to claim 1, characterized in that: Before acquiring a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions arranged at equal intervals, the method includes: Controlling the time-of-flight camera to turn on and preheating the time-of-flight camera for a specified period of time; Setting the exposure time of the time-of-flight camera; The time-of-flight camera is controlled to move to a plurality of positions arranged at equal intervals in sequence.

5. The method for calibrating a time-of-flight camera according to any one of claims 1 to 4, characterized in that: Before acquiring a plurality of diffuse reflection target images acquired by the time-of-flight camera at a plurality of positions arranged at equal intervals, the method further includes: Determine the focus position and control the lens of the time-of-flight camera to move to the focus position; The focus position is the lens position at which the time-of-flight camera can obtain an image with the highest definition.

6. The calibration method of a time-of-flight camera according to claim 5, characterized in that: The determining of the focus position and controlling the lens of the time-of-flight camera to move to the focus position includes: Controlling the time-of-flight camera to move to a distance from the focus target that is equal to the hyperfocal distance of the time-of-flight camera; Controlling the lens of the time-of-flight camera to move to different lens adjustment positions; Acquire a focused target image acquired by the time-of-flight camera at different lens adjustment positions; Analyzing the clarity of the focused target image to obtain a focused target image with the highest clarity; The lens adjustment position corresponding to the focus target image with the highest definition is used as the focus position, and the lens is stopped at the focus position.

7. The calibration method of a time-of-flight camera according to claim 6, characterized in that: The analyzing the clarity of the focused target image to obtain the focused target image with the highest clarity includes: Calculating the sum of squares of grayscale gradients of the focused target image; The focused target image with the largest grayscale gradient square sum is used as the focused target image with the highest definition.

8. A calibration device for a time-of-flight camera, characterized in that: include: A communication module, used for acquiring a plurality of diffuse reflection target images acquired by a time-of-flight camera at a plurality of positions set at equal intervals; as well as A control module connected to the communication module, configured to calculate a measurement phase based on the plurality of diffuse reflection target images; and obtain a distance between the time-of-flight camera and the diffuse reflection target at each position; Calculate the real phase according to the distance; calculate the fixed template noise phase according to the measured phase and the real phase; generate a depth calibration file according to the fixed template noise phase; The communication module is further used to write the depth calibration file into the time-of-flight camera to complete the calibration when communicating with the time-of-flight camera.

9. A calibration system for a time-of-flight camera, characterized in that: It includes a calibration device, a camera fixing and adjusting device, a focusing target and a diffuse reflection target; The time-of-flight camera is fixed on the camera fixing and adjusting device, the focus target or diffuse reflection target is arranged toward the lens of the time-of-flight camera, and the optical axis of the time-of-flight camera passes through the center of the focus target or diffuse reflection target, and the calibration device is connected to the time-of-flight camera and the camera fixing and adjusting device; The calibration device is used to control the camera fixing and adjusting device to move to a plurality of positions set at equal intervals; The time-of-flight camera is used to collect image data of the focus target or the diffuse reflection target and output it to the calibration device; The calibration device is used to execute the calibration method as described in any one of claims 1 to 7.

10. The time-of-flight camera calibration system according to claim 9, characterized in that: It also includes a target switching device, wherein the control signal output end of the calibration device is connected to the control signal input end of the target switching device; The calibration device is further used to send a focus target switching signal to the target switching device, or send a diffuse reflection target switching signal to the target switching device; The target switching device is used to switch to the focus target based on the focus target switching signal, or to switch to the diffuse reflection target based on the diffuse reflection target switching signal.

11. The time-of-flight camera calibration system according to claim 9, characterized in that: The camera fixing and adjusting device is provided with a sliding assembly and a camera adjusting assembly, wherein the sliding assembly is arranged at the bottom of the camera fixing and adjusting device, the camera adjusting assembly comprises a pitch adjusting assembly and a horizontal rotation adjusting assembly, and the time-of-flight camera is fixed to the camera fixing and adjusting device through the camera adjusting assembly; The calibration device is communicatively connected with the sliding assembly and the camera adjustment assembly.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the calibration method according to any one of claims 1 to 7 are implemented.

Images