Temperature compensation model method, determination method, equipment and camera

Abstract

The embodiment of the invention provides a temperature compensation model method, a determination method, equipment and a camera, and the method comprises the steps: obtaining the calibration internal reference values of a TOF camera at a plurality of first test temperatures, determining a first temperature compensation model of the TOF camera according to the plurality of first test temperatures and the plurality of calibration internal reference values, the first temperature compensation model is used for determining an internal reference temperature compensation value corresponding to the working temperature according to the working temperature of the TOF camera. According to the method provided by the embodiment of the invention, the calibration internal reference values of the internal reference (focal length, optical center, distortion and the like) of the TOF camera at different temperatures are acquired, the rule of the internal reference changing along with the temperature is analyzed, and the temperature compensation model is generated based on the rule to perform temperature compensation on the internal reference of the camera, so that the precision of the internal reference of the camera is ensured, the temperature adaptation range of the TOF camera is expanded, and the application range of the TOF camera is expanded. The use flexibility is improved, and the problem of low efficiency of recalibration aiming at different working temperatures is solved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of extended reality technology, and in particular to a temperature compensation model method, determination method, device, and camera. Background Art

[0002] In Extended Reality (XR) devices, multiple cameras (such as color cameras, gray-white cameras, and Time Of Flight (TOF) depth cameras) are usually used to acquire data, and the data of multiple cameras are projected and mapped to obtain data of the virtual world. As an active luminous ranging sensor, the accuracy of the camera's intrinsic parameters is an important factor affecting the projection mapping effect.

[0003] In the related art, a special internal calibration is usually performed for a specific operating temperature (eg, 50° C.) of a TOF camera.

[0004] However, the inventors have discovered that there are at least the following technical problems in the related art: the above method has certain limitations, and when the working temperature of the TOF camera changes, it needs to be recalibrated, which has poor flexibility and low efficiency. Summary of the invention

[0005] The embodiments of the present disclosure provide a temperature compensation model method, determination method, device and camera to improve flexibility and efficiency.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for determining a temperature compensation model, comprising:

[0007] Obtaining calibration internal parameter values ​​of the TOF camera at multiple first test temperatures respectively;

[0008] A first temperature compensation model of the TOF camera is determined according to the plurality of first test temperatures and the plurality of calibrated internal parameter values; the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0009] In a second aspect, an embodiment of the present disclosure provides a temperature compensation method, including:

[0010] Acquire a first temperature compensation model of the TOF camera and a current operating temperature; the first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at the plurality of first test temperatures;

[0011] Based on the first temperature compensation model, an internal reference temperature compensation value corresponding to the operating temperature is determined.

[0012] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0013] A first acquisition module is used to obtain calibration internal parameter values ​​of the TOF camera at multiple first test temperatures respectively;

[0014] The first determination module is used to determine the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values; the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0015] In a fourth aspect, an embodiment of the present disclosure provides a TOF camera, including:

[0016] A second acquisition module is used to acquire a first temperature compensation model of the TOF camera and a current operating temperature; the first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at the plurality of first test temperatures;

[0017] The second determination module is used to determine an internal reference temperature compensation value corresponding to the operating temperature based on the first temperature compensation model.

[0018] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: a temperature control component, a processor, and a memory;

[0019] The temperature control components are all connected to the processor;

[0020] The temperature control component is used to load the TOF camera and set a plurality of first test temperatures for the TOF camera;

[0021] The memory stores computer-executable instructions;

[0022] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for determining the temperature compensation model as described in the first aspect and various possible designs of the first aspect.

[0023] In a sixth aspect, an embodiment of the present disclosure provides a TOF camera, including: a lens, a temperature sensor, a processor, and a memory;

[0024] The temperature sensor is connected to the processor and is used to obtain the current operating temperature and send the operating temperature to the processor;

[0025] The memory stores computer-executable instructions;

[0026] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the method for determining the temperature compensation model as described in the second aspect and various possible designs of the second aspect.

[0027] In the seventh aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the method for determining the temperature compensation model as described in the first and second aspects and various possible designs of the first and second aspects is implemented.

[0028] In an eighth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the method for determining the temperature compensation model as described in the first and second aspects and various possible designs of the first and second aspects.

[0029] The temperature compensation model method, determination method, device and camera provided in the present embodiment include obtaining the calibrated internal parameter values ​​of the TOF camera at multiple first test temperatures respectively, and determining the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values, wherein the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the working temperature according to the working temperature of the TOF camera. The method provided in the embodiment of the present disclosure obtains the calibrated internal parameter values ​​of the TOF camera internal parameters (focal length, optical center, distortion, etc.) at different temperatures, analyzes the law of the change of the internal parameters with temperature, and then generates a temperature compensation model based on the law to perform temperature compensation on the camera internal parameters, thereby ensuring the accuracy of the camera internal parameters, improving the temperature adaptability range of the TOF camera, improving the flexibility of use, and solving the problem of low efficiency of recalibration for different working temperatures. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 A schematic diagram of the structure of a TOF camera provided in an embodiment of the present disclosure;

[0032] Figure 2 Schematic diagram of the method for determining the temperature compensation model provided in the embodiment of the present disclosure Figure 1 ;

[0033] Figure 3 A schematic diagram of the principle of a method for determining a temperature compensation model provided in an embodiment of the present disclosure;

[0034] Figure 4 Schematic diagram of the method for determining the temperature compensation model provided in the embodiment of the present disclosure Figure 2 ;

[0035] Figure 5 A schematic diagram of a temperature compensation method provided in an embodiment of the present disclosure;

[0036] Figure 6 A structural block diagram of an electronic device provided by an embodiment of the present disclosure;

[0037] Figure 7 A structural block diagram of a TOF camera provided in an embodiment of the present disclosure;

[0038] Figure 8 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0040] In mobile phones, Internet of Things (IOT) and Extended Reality (XR) devices (such as Augmented Reality (AR), Virtual Reality (VR), Mixed Reality (MR) and other devices), RGB color cameras are usually used to capture detail information and color information in real scenes, and 6-degree of freedom (DOF) gray-white cameras are used to collect 2D information in real scenes. Time of Flight (TOF) depth cameras are used to quickly and accurately measure distance information in real-world environments and map it to the virtual world to create a more realistic and interactive MR experience. Therefore, extremely high requirements (sub-pixel level) are placed on the matching accuracy of feature information from multiple cameras.

[0041] In the related art, a special internal calibration is usually performed for a specific operating temperature (e.g., 50° C.) of a TOF camera. However, the inventors have found that the above method has certain limitations. When the operating temperature of the TOF camera changes, recalibration is required, which has poor flexibility and low efficiency.

[0042] In view of the above technical problems, the inventors of the present application have found that when applying TOF cameras to devices with relatively small size requirements such as XR devices, the size of TOF cameras is limited, and as an active light-emitting ranging sensor, its heat dissipation characteristics are also limited. Compared with TOF cameras applied to scenarios where the size is not limited, the camera parameters of TOF cameras are more susceptible to temperature changes, thereby affecting the matching accuracy of feature information of multiple cameras and reducing the accuracy of mutual projection mapping of multiple cameras. The inventors also found that in order to adapt to the influence of temperature changes on the camera intrinsic parameters of the TOF camera, the calibrated intrinsic parameter values ​​of the TOF camera intrinsic parameters (focal length, optical center, distortion, etc.) at different temperatures can be obtained, and the law of the change of the intrinsic parameters with temperature can be analyzed, and then a temperature compensation model can be generated based on the law to perform temperature compensation on the camera intrinsic parameters, thereby ensuring the accuracy of the camera intrinsic parameters, improving the temperature adaptation range of the TOF camera, improving the flexibility of use, and solving the problem of low efficiency of recalibration for different working temperatures. Based on this, the embodiment of the present disclosure provides a method for determining a temperature compensation model.

[0043] Figure 1 This is a schematic diagram of the structure of a TOF camera provided in an embodiment of the present disclosure. Figure 1 As shown in the figure, taking the indirect time of flight (ITOF) module solution as an example, the TOF camera mainly includes: a transmitter Tx and a receiver Rx. Among them, the transmitter Tx includes: a vertical cavity surface emitting laser (Vertical-Cavity Surface-Emitting Laser, VCSEL) or an edge emitting laser (Edge Emitting Laser, EEL), a collimator, a diffractive optical element (Diffractive Optical Element, DOE) or a scattering element Diffuser, a laser driver Driver IC, a photodiode (Photo-Diode, PD), a base Holder, a flexible printed circuit (Flexible Printed Circuit, FPC), etc. Among them, VCSEL or EEL, the most commonly used wavelengths are 850nm and 940nm. The light source is usually a diffuse light source (flood lighting), emitting a light beam with a certain divergence (i.e., field of illumination or FOI); Collimator+DOE or Diffuser, used for light field pattern shaping, emitting light fields such as speckles, stripes or rectangular uniform light; Driver IC, used to modulate the intensity, pulse frequency, exposure time, etc. of the light emitted by the light source; photodiode PD, used for optical power detection, anomaly detection and other functions.

[0044] The receiving end Rx includes: image sensor Sensor, receiving lens Rx Lens, narrow band pass filter (BandPass Filter, BPF), image signal processing unit (Image Signal Processing, ISP), Holder, FPC, etc. Among them, Sensor is used to collect the return light from the scene and output the value of each pixel, and calculate the depth based on the TOF demodulation principle; Rx Lens is used to focus the return light on the sensor array imaging lens; BPF is used to filter out interference light outside the narrow bandwidth around the wavelength of the light source; ISP is used to convert the output raw data of the sensor into a depth image or point cloud data.

[0045] In the specific implementation process, a temperature sensor can be set at the edge of the Sensor, and the current working temperature can be obtained through the temperature sensor, and based on the predetermined first compensation model of the TOF camera and the working temperature, the internal temperature compensation value corresponding to the working temperature is determined. The first temperature compensation model is determined according to multiple first test temperatures and multiple calibrated internal parameter values; the multiple calibrated internal parameter values ​​are determined when the TOF camera is respectively at multiple first test temperatures. The temperature compensation method for a TOF camera provided in the embodiment of the present disclosure obtains the calibrated internal parameter values ​​of the TOF camera internal parameters (focal length, optical center, distortion, etc.) at different temperatures, analyzes the law of the change of the internal parameters with temperature, and then generates a temperature compensation model based on the law to perform temperature compensation on the camera internal parameters, thereby ensuring the accuracy of the camera internal parameters, improving the temperature adaptation range of the TOF camera, improving the flexibility of use, and solving the problem of low efficiency of recalibration for different working temperatures.

[0046] Figure 2 Schematic diagram of the method for determining the temperature compensation model provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied to electronic devices such as terminal devices or servers. The method for determining the temperature compensation model includes:

[0047] 201. Obtain calibration internal parameter values ​​of a TOF camera at multiple first test temperatures.

[0048] Specifically, the temperature of the environment in which the TOF camera is located may be adjusted, and a plurality of first test temperature values ​​may be set for the TOF camera. When the TOF camera is at the first test temperature respectively, the calibration internal parameter value may be determined.

[0049] The calibration internal parameter value may include the calibration value of at least one parameter of the focal length (fx, fy), optical center (oc-x, oc-y) and distortion parameter of the TOF camera at the corresponding test temperature. The distortion parameter includes radial distortion parameter (k1, k2, k3) and / or tangential distortion parameter (p1, p2).

[0050] For example, Figure 3 As shown, the TOF camera can be set on a temperature control component, such as a semiconductor refrigerator (Thermoelectric Cooler, TEC). TEC includes a TEC controller and a TEC body, and the TEC controller is used to control the TEC body for temperature regulation. A chart card is fixed in front of the TOF camera, such as a backlight source calibration chart card with a fixed band (such as a checkerboard), and then for each first test temperature, the calibration internal parameter value of the TOF camera at the first test temperature can be obtained based on different distance information and the image data output by the TOF camera. Optionally, the electronic device may also include a power component, and the TEC is set on the power component. Under the drive of the power component, the distance between the TOF camera and the chart card can be adjusted to perform batch experiments at different distances.

[0051] In one embodiment of the present disclosure, obtaining the calibration internal parameter values ​​of the TOF camera at multiple first test temperatures respectively may include: setting multiple first test temperatures based on a preset step size; and determining the calibration internal parameter value of the TOF camera at the first test temperature for each of the multiple first test temperatures.

[0052] Specifically, to facilitate data analysis, multiple first test temperatures may be set with a fixed step size.

[0053] For example, Figure 3 As shown, the TOF camera is placed on a temperature-controllable TEC carrier. The TOF camera shoots a picture card at a fixed distance. The temperature sensor inside the TOF camera, such as the negative temperature coefficient (NTC) thermistor, can output temperature data corresponding to the image data with each frame. The first test temperature can be set to room temperature (20°C) through TEC, and then another first test temperature can be set according to a temperature rise step of 5°C, thereby completing data collection at different first test temperatures.

[0054] 202. Determine a first temperature compensation model of the TOF camera according to the plurality of first test temperatures and the plurality of calibrated internal parameter values; the first temperature compensation model is used to determine an internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0055] Specifically, after obtaining the calibration internal parameter values ​​corresponding to the multiple first test temperatures, the correlation between the multiple first test temperatures and the multiple calibration internal parameter values ​​can be analyzed to find out the change rule of the calibration internal parameter value with the change of the first test temperature, and then determine the first temperature compensation model. The internal parameter temperature compensation value of the TOF camera at the current working temperature can be predicted by the first temperature compensation model. The first temperature compensation model can be a data table. When the TOF camera is working, the internal parameter temperature compensation value corresponding to the current working temperature can be obtained by looking up the table. The first temperature compensation model can also be a mathematical model obtained by big data fitting. When the TOF camera is working, the current working temperature can be input into the mathematical model to obtain the corresponding internal parameter temperature compensation value.

[0056] In the embodiment of the present disclosure, the internal parameter temperature compensation value may be the compensated internal parameter value, or may be the difference between the compensated internal parameter value and the calibrated internal parameter value. The specific value may be determined according to actual needs, and the embodiment of the present disclosure is not limited to this.

[0057] In one embodiment of the present disclosure, the independent variable of the first temperature compensation model is temperature, and the dependent variable is the internal parameter of the TOF camera; determining the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values ​​may include: determining the model parameters in the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values.

[0058] In one embodiment of the present disclosure, determining the model parameters in the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values ​​may include: determining the model parameters of the first temperature compensation model based on least squares fitting according to the multiple first test temperatures and the multiple calibrated internal parameter values.

[0059] In one embodiment of the present disclosure, the first temperature compensation model is a polynomial regression model.

[0060] From the above description, it can be seen that this embodiment obtains the calibrated internal parameter values ​​of the TOF camera internal parameters (focal length, optical center, distortion, etc.) at different temperatures, analyzes the law of change of the internal parameters with temperature, and then generates a temperature compensation model based on the law to perform temperature compensation on the camera internal parameters, thereby ensuring the accuracy of the camera internal parameters, improving the temperature adaptability range of the TOF camera, improving the flexibility of use, and solving the problem of low efficiency of recalibration for different working temperatures.

[0061] In one embodiment of the present disclosure, Figure 4 Schematic diagram of the method for determining the temperature compensation model provided in the embodiment of the present disclosure Figure 2 .like Figure 4 As shown in the above Figure 2 Based on the embodiment, in order to improve the ranging accuracy of the TOF camera, after step 202 of the above embodiment, the following may also be included:

[0062] 203. Obtain calibrated depth values ​​obtained by photographing a target object located at a fixed position by the TOF camera at multiple second test temperatures.

[0063] Specifically, the temperature of the environment in which the TOF camera is located can be adjusted, and multiple second test temperature values ​​can be set for the TOF camera. When the TOF camera is at the second test temperatures, a target object (such as a picture card) located at a fixed position is photographed to obtain a depth image, and then the depth value is calibrated based on the depth image.

[0064] 204. Determine a second temperature compensation model of the TOF camera according to the plurality of second test temperatures and the plurality of calibrated depth values; the second temperature compensation model is used to determine a depth temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0065] Specifically, after obtaining the calibrated depth values ​​corresponding to the multiple second test temperatures, the correlation between the multiple second test temperatures and the multiple calibrated depth values ​​can be analyzed to find out the changing rule of the calibrated depth value with the change of the second test temperature, and then determine the second temperature compensation model. The depth temperature compensation value of the TOF camera at the current working temperature can be predicted by the second temperature compensation model. The second temperature compensation model can be a data table. When the TOF camera is working, the depth temperature compensation value corresponding to the current working temperature can be obtained by looking up the table. The second temperature compensation model can also be a mathematical model obtained by big data fitting. When the TOF camera is working, the current working temperature can be input into the mathematical model to obtain the corresponding depth temperature compensation value.

[0066] In the embodiment of the present disclosure, the depth temperature compensation value may be the compensated depth value, or may be the difference between the compensated depth value and the initial calibrated depth value. The specific value may be determined according to actual needs, and the embodiment of the present disclosure is not limited to this.

[0067] In one embodiment of the present disclosure, the independent variable of the second temperature compensation model is temperature, and the dependent variable is the depth of the TOF camera. Determining the second temperature compensation model of the TOF camera according to a plurality of the second test temperatures and a plurality of the calibrated depth values ​​may include: determining model parameters in the second temperature compensation model of the TOF camera according to the plurality of the second test temperatures and the plurality of the calibrated depth values.

[0068] In one embodiment of the present disclosure, determining the model parameters in the second temperature compensation model of the TOF camera according to the plurality of the second test temperatures and the plurality of the calibrated depth values ​​may include: determining the model parameters of the second temperature compensation model based on least squares fitting according to the plurality of the second test temperatures and the plurality of the calibrated depth values.

[0069] In one embodiment of the present disclosure, the second temperature compensation model may be a polynomial regression model.

[0070] For example, Figure 3 As shown, the TOF camera is placed on a temperature-controllable TEC carrier, and the TOF camera shoots the chart in front, such as a fixed reflectivity diffuse reflection plate. The temperature sensor inside the TOF camera, such as a thermistor (Negative Temperature Coefficient, NTC), can output temperature data corresponding to the depth data with the frame. Through TEC, one of the second test temperatures can be set to room temperature (20°C), and then another second test temperature can be set according to a temperature rise step of 5°C, thereby completing data collection at different second test temperatures and determining different depth values ​​corresponding to different second test temperatures. Optionally, the electronic device can also include a power component, and the TEC is set on the power component. Under the drive of the power component, the distance between the TOF camera and the chart can be adjusted to perform batch experiments at different distances. After obtaining the depth data at different second test temperatures, the second test temperature and the depth data can be correlated and analyzed, and then a data model between the temperature change and the depth change data can be obtained by regression analysis. For example, a polynomial compensation function formula can be established and fitted by the least squares method to obtain a second temperature compensation model.

[0071] From the above description, it can be seen that in order to solve the problem of TOF camera ranging accuracy error deviation caused by temperature drift, the second temperature compensation model is determined by measuring the depth data, transmission and reception parameters of the TOF camera at different temperatures, so that after the TOF camera obtains the current operating temperature (for example, obtained by NTC measurement), it can determine the corresponding depth temperature compensation value based on the second temperature compensation model.

[0072] It should be noted that the determination process of the first temperature compensation model and the determination process of the second temperature compensation model may overlap to improve experimental efficiency. For example, the first test temperature and the second test temperature may be the same temperature. When a certain test temperature is set, the image data and depth data at the current test temperature may be obtained by replacing the image card, and then the calibration internal parameter value and the calibration depth value corresponding to the test temperature may be calculated respectively. Thus, after obtaining the calibration internal parameter value and the calibration depth value corresponding to different temperatures, the first temperature compensation model is determined and the second temperature compensation model is determined. For example, as Figure 3 As shown, the TOF camera is placed on a temperature-controllable TEC carrier, the camera module shoots a fixed-distance chart, and the NTC inside the TOF module can output image data, depth data, and temperature data with each frame. At room temperature (20°C), data at different temperatures are collected according to a temperature rise step of 5°C, and the camera internal parameter data and depth data at different temperatures are solved. The correlation analysis of camera temperature and focal length, optical center, distortion, and depth change is performed, and then the data model between temperature change and internal parameter and depth change data is obtained through regression analysis. The polynomial compensation function formula is established and fitted through the least squares method to obtain the first temperature compensation model and the second temperature compensation model, or to obtain the temperature compensation model (the temperature compensation model can simultaneously realize the functions of the first temperature compensation model and the second temperature compensation model).

[0073] Figure 5 The temperature compensation method provided in the embodiment of the present disclosure is a flow chart. The method of this embodiment can be applied in a TOF camera, such as Figure 5 As shown, the method includes:

[0074] 501. Obtain a first temperature compensation model of a TOF camera and a current operating temperature; the first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at a plurality of the first test temperatures.

[0075] Specifically, the TOF camera can obtain the current operating temperature through a temperature sensor disposed inside the TOF camera (eg, an NTC disposed at the edge of the image sensor of the TOF camera). The first temperature compensation model can be pre-built into the processor of the TOF camera.

[0076] The method for determining the first temperature compensation model may refer to the description of the above embodiment and will not be described in detail here.

[0077] 502. Determine an internal reference temperature compensation value corresponding to the operating temperature based on the first temperature compensation model.

[0078] Specifically, after the current operating temperature is obtained, an internal reference temperature compensation value corresponding to the operating temperature may be determined by a first temperature compensation model.

[0079] From the above description, it can be seen that by obtaining the calibrated internal parameter values ​​of the TOF camera internal parameters (focal length, optical center, distortion, etc.) at different temperatures, analyzing the law of how the internal parameters change with temperature, and then generating a temperature compensation model based on the law to perform temperature compensation on the camera internal parameters, the accuracy of the camera internal parameters is ensured, the temperature adaptability range of the TOF camera is improved, the flexibility of use is improved, and the problem of low efficiency of recalibration for different working temperatures is solved.

[0080] In one embodiment of the present disclosure, Figure 5 Based on the embodiment, in order to improve the ranging accuracy of the TOF camera, the method further includes:

[0081] Acquire a second temperature compensation model of the TOF camera; the second temperature compensation model is determined according to a plurality of second test temperatures and a plurality of calibration depth values; the plurality of calibration depth values ​​are determined when the TOF camera is respectively at the plurality of second test temperatures;

[0082] Based on the second temperature compensation model, a depth temperature compensation value corresponding to the operating temperature is determined.

[0083] In one embodiment of the present disclosure, based on the above embodiment, in order to further reduce the influence of temperature on the data accuracy of the TOF camera, that is, to solve the temperature drift problem of the receiving module of the TOF camera, at least one of the multiple lenses in the TOF camera lens can be set as a glass lens in the embodiment of the present disclosure. Other lenses can be made of non-glass materials, such as plastic.

[0084] Specifically, considering that the thermal expansion coefficients of glass lenses and plastic lenses are different: the thermal expansion coefficient of glass materials is usually lower than that of plastic materials. This means that when the temperature rises, the glass lens expands less than the plastic lens, so the temperature drift performance of the glass lens is relatively good. The temperature sensitivity of the two is different: plastic materials are usually more susceptible to temperature than glass materials, so the temperature drift performance of plastic lenses may be worse than that of glass lenses. The substrate properties of the two are different: glass lenses are usually better than plastic lenses in terms of heat resistance, chemical resistance, hardness, etc., and these differences in properties may affect the temperature drift performance of the lenses. In general, glass lenses have better temperature drift performance than plastic lenses.

[0085] In addition, by comparing the temperature drift coefficients of the existing TOF camera including a full plastic lens and the TOF camera including a non-full plastic lens including a glass lens provided in the embodiment of the present disclosure, the experimental conclusion is that the temperature drift coefficient of the full plastic lens is approximately 0.6 to 0.7 um / °C, and the temperature drift coefficient of the improved non-full plastic lens is 0.3 to 0.4 um / °C, which is doubled.

[0086] In one embodiment of the present disclosure, the existing all-plastic lens can be improved in design. Specifically, the curved surface of each lens can be fine-tuned in combination with the change of the refractive index of the glass lens and the plastic lens to obtain the TOF camera provided by the embodiment of the present disclosure. A completely new design can also be performed according to parameter requirements, and at least one of the multiple lenses is designed as a glass lens during the design process.

[0087] For example, a lens including five lenses is originally designed as a full plastic lens, i.e., a 5P structure. In order to solve the temperature drift problem, a 2P+1G+2P lens structure can be designed, where P represents a plastic lens and G represents a glass lens.

[0088] In one embodiment of the present disclosure, the aperture stop that is most sensitive to temperature drift can be set on the first side of the glass lens (the side where the light beam enters, that is, the side away from the image sensor), so as to optimize and solve the temperature drift problem of the receiving module to the greatest extent.

[0089] Specifically, if an existing all-plastic lens is to be improved in design, the plastic lens with the aperture stop can be replaced with a glass lens. This can improve the temperature drift problem to the greatest extent while controlling the cost.

[0090] It can be seen from the above description that by setting at least one lens in the TOF camera as a glass lens, the temperature drift problem of the receiving module of the TOF camera can be further improved.

[0091] Corresponding to the method for determining the temperature compensation model of the above embodiment, Figure 6 This is a structural block diagram of an electronic device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 6 , the device includes: a first acquisition module 601 and a first determination module 602.

[0092] The first acquisition module 601 is used to obtain the calibration internal parameter values ​​of the TOF camera at multiple first test temperatures.

[0093] The first determination module 602 is used to determine the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values; the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0094] In one embodiment of the present disclosure, the first acquisition module 601 is specifically used to: set multiple first test temperatures based on a preset step size; and determine, for each of the multiple first test temperatures, a calibration internal parameter value of the TOF camera at the first test temperature.

[0095] In one embodiment of the present disclosure, the independent variable of the first temperature compensation model is temperature, and the dependent variable is the internal parameter of the TOF camera; the first determination module 602 is specifically used to determine the model parameters in the first temperature compensation model of the TOF camera according to multiple first test temperatures and multiple calibrated internal parameter values.

[0096] In one embodiment of the present disclosure, the first determination module 602 is specifically used to determine the model parameters of the first temperature compensation model based on least squares fitting according to the plurality of the first test temperatures and the plurality of the calibrated internal parameter values.

[0097] In one embodiment of the present disclosure, the first temperature compensation model is a polynomial regression model.

[0098] In one embodiment of the present disclosure, the calibration internal parameter value includes a calibration value of at least one parameter of the focal length, optical center, and distortion parameter of the TOF camera at a corresponding test temperature.

[0099] In one embodiment of the present disclosure, the first determination module 602 is also used to: obtain calibrated depth values ​​obtained by the TOF camera shooting a target located at a fixed position at multiple second test temperatures; determine a second temperature compensation model of the TOF camera based on the multiple second test temperatures and the multiple calibrated depth values; the second temperature compensation model is used to determine the depth temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

[0100] The device provided in this embodiment can be used to execute the technical solution of the above-mentioned temperature compensation model determination method embodiment, and its implementation principle and technical effect are similar, which will not be repeated in this embodiment.

[0101] Corresponding to the temperature compensation method of the above embodiment, Figure 7 The structure block diagram of the TOF camera provided by the embodiment of the present disclosure. For the convenience of explanation, only the part related to the embodiment of the present disclosure is shown. Figure 7, the device includes: a second acquisition module 701 and a second determination module 702.

[0102] The second acquisition module 701 is used to acquire a first temperature compensation model of the TOF camera and a current operating temperature; the first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at the plurality of first test temperatures;

[0103] The second determination module 702 is configured to determine an internal reference temperature compensation value corresponding to the operating temperature based on the first temperature compensation model.

[0104] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0105] refer to Figure 8 , which shows a schematic diagram of the structure of an electronic device 800 suitable for implementing the embodiment of the present disclosure, and the electronic device 800 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0106] like Figure 8 As shown, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0107] Typically, the following devices may be connected to the I / O interface 805: input devices 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 8 The electronic device 800 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0108] The electronic device provided by the embodiment of the present disclosure may include a temperature control component, and the temperature control component is connected to the processor 801; the temperature control component is used to load the TOF camera and set multiple first test temperatures for the TOF camera.

[0109] In one embodiment of the present disclosure, the temperature control device is a semiconductor refrigerator.

[0110] In one embodiment of the present disclosure, the lens of the TOF camera includes a plurality of lenses; the plurality of lenses include at least one glass lens.

[0111] In one embodiment of the present disclosure, a glass lens among the at least one glass lens is provided with an aperture stop.

[0112] An embodiment of the present disclosure also provides a TOF camera, including: a lens, a temperature sensor, a processor and a memory; the temperature sensor is connected to the processor, and is used to obtain the current operating temperature and send the operating temperature to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory, so that the processor executes the temperature compensation method provided in the above embodiment.

[0113] In one embodiment of the present disclosure, the lens includes a plurality of lenses; the plurality of lenses include at least one glass lens.

[0114] In one embodiment of the present disclosure, a glass lens among the at least one glass lens is provided with an aperture stop.

[0115] In one embodiment of the present disclosure, the temperature sensor is a thermistor and is disposed at an edge of an image sensor chip of the TOF camera.

[0116] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0117] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0118] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0119] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0120] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0123] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0126] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0127] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A method for determining a temperature compensation model of a TOF camera, characterized in that: include: Obtaining calibration internal parameter values ​​of the TOF camera at multiple first test temperatures respectively; A first temperature compensation model of the TOF camera is determined according to the plurality of first test temperatures and the plurality of calibrated internal parameter values; the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

2. The method according to claim 1, characterized in that The step of obtaining the calibration internal parameter values ​​of the TOF camera at a plurality of first test temperatures includes: Based on a preset step length, setting a plurality of first test temperatures; For each first test temperature among the plurality of first test temperatures, a calibration internal parameter value of the TOF camera at the first test temperature is determined.

3. The method according to claim 1, characterized in that The independent variable of the first temperature compensation model is temperature, and the dependent variable is the internal parameter of the TOF camera; The step of determining a first temperature compensation model of the TOF camera according to the plurality of first test temperatures and the plurality of calibration internal parameter values ​​comprises: Model parameters in a first temperature compensation model of the TOF camera are determined according to the plurality of first test temperatures and the plurality of calibrated internal parameter values.

4. The method according to claim 3, characterized in that The step of determining the model parameters in the first temperature compensation model of the TOF camera according to the plurality of the first test temperatures and the plurality of the calibrated internal parameter values ​​comprises: According to the plurality of the first test temperatures and the plurality of the calibrated internal parameter values, model parameters of the first temperature compensation model are determined based on least squares fitting.

5. The method according to claim 3, characterized in that: The first temperature compensation model is a polynomial regression model.

6. The method according to any one of claims 1 to 5, characterized in that: The calibration internal parameter value includes a calibration value of at least one parameter of the focal length, optical center, and distortion parameter of the TOF camera at a corresponding test temperature.

7. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining calibrated depth values ​​obtained by photographing a target object located at a fixed position by the TOF camera at multiple second test temperatures; A second temperature compensation model of the TOF camera is determined according to the plurality of second test temperatures and the plurality of calibrated depth values; the second temperature compensation model is used to determine the depth temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

8. A temperature compensation method for a TOF camera, characterized in that: include: Get the first temperature compensation model of the TOF camera and the current operating temperature; The first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at the plurality of first test temperatures; Based on the first temperature compensation model, an internal reference temperature compensation value corresponding to the operating temperature is determined.

9. An electronic device, characterized in that: include: A first acquisition module is used to obtain calibration internal parameter values ​​of the TOF camera at multiple first test temperatures respectively; The first determination module is used to determine the first temperature compensation model of the TOF camera according to the multiple first test temperatures and the multiple calibrated internal parameter values; the first temperature compensation model is used to determine the internal parameter temperature compensation value corresponding to the operating temperature according to the operating temperature of the TOF camera.

10. A TOF camera, characterized in that: include: A second acquisition module is used to acquire a first temperature compensation model of the TOF camera and a current operating temperature; The first temperature compensation model is determined according to a plurality of first test temperatures and a plurality of calibration internal parameter values; the plurality of calibration internal parameter values ​​are determined when the TOF camera is respectively at the plurality of first test temperatures; The second determination module is used to determine an internal reference temperature compensation value corresponding to the operating temperature based on the first temperature compensation model.

11. An electronic device, characterized in that: Including: temperature control components, processors and memory; The temperature control components are all connected to the processor; The temperature control component is used to load the TOF camera and set a plurality of first test temperatures for the TOF camera; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method for determining the temperature compensation model according to any one of claims 1 to 7.

12. The device according to claim 11, characterized in that The temperature control device is a semiconductor refrigerator.

13. The device according to claim 11 or 12, characterized in that The lens of the TOF camera includes a plurality of lenses; the plurality of lenses include at least one glass lens.

14. The device according to claim 13, characterized in that One of the at least one glass lens is provided with an aperture stop.

15. A TOF camera, characterized in that: include: Lens, temperature sensor, processor and memory; The temperature sensor is connected to the processor and is used to obtain the current operating temperature and send the operating temperature to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the temperature compensation method according to claim 8.

16. The camera according to claim 15, characterized in that The lens comprises a plurality of lenses; the plurality of lenses comprises at least one glass lens.

17. The camera according to claim 16, characterized in that One of the at least one glass lens is provided with an aperture stop.

18. The camera according to any one of claims 15 to 17, characterized in that The temperature sensor is a thermistor and is arranged at the edge of the image sensor chip of the TOF camera.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method for determining the temperature compensation model according to any one of claims 1 to 8 is implemented.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for determining a temperature compensation model according to any one of claims 1 to 8 is implemented.

Images