Analog Modulation TOF Camera Calibration Method, Apparatus, System and Storage Medium

Abstract

This invention discloses a method, apparatus, system, and storage medium for simulating modulation TOF camera correction. First, a light intensity threshold is calculated as the depth error classification standard. The light intensity threshold with the largest standard deviation of the depth error is selected as the depth error oscillation critical point. A light intensity-distance lookup table is established based on the light intensity-distance dataset to correct high-intensity depth data. A light intensity-depth random forest nonlinear model is established, and the hyperparameters of the model are optimized using a particle swarm optimization algorithm for low-intensity depth data. Next, a material correction classification model is constructed. For light scattering errors caused by materials in the measurement scene, the LDA algorithm is used to extract effective features and establish a material-depth distortion dataset. Finally, during real-time acquisition, the trained material correction classification model is used to classify the materials of the acquired objects. Based on the material classification results, the depth data is corrected using the material-depth distortion dataset to obtain a high-quality depth image.

Description

Technical Field

[0001] This invention relates to the field of optical metrology and calibration technology, and in particular to a method, apparatus, system and storage medium for calibrating an analog modulation TOF camera. Background Technology

[0002] With the rapid development of artificial intelligence, 3D information acquisition has received widespread attention in fields such as computer vision, robot navigation, human-computer interaction, and autonomous driving. Common methods for acquiring 3D information include stereo vision, structured light, single-pixel 3D imaging, and time-of-flight (TOF). Stereo vision requires advanced matching algorithms to obtain accurate depth information, which is easily affected by ambient light. Structured light requires projection optimization compensation, which places high demands on the performance of the processing system. Compared to the above two methods, TOF depth cameras utilize actively emitted infrared lasers to acquire depth information, offering advantages such as low cost, high frame rate, and high reliability.

[0003] Time-of-Flight (TOF) depth cameras acquire distance information between an infrared laser and an object by calculating the time of flight between the camera and the object, while simultaneously acquiring grayscale information. This makes them a highly efficient 3D imaging instrument. However, the error between the measurement data from a TOF depth camera and the actual data is affected by various sources of error, such as ambient light noise, integration time, temperature drift, and differences in reflectivity caused by different object surface materials. These factors can significantly degrade the performance of a TOF camera. Therefore, proper calibration of the TOF camera is necessary to achieve reliable depth information acquisition.

[0004] There are several methods for depth calibration of Time-of-Flight (TOF) cameras, one common method being calibration using a guide rail. Specifically, the TOF camera is placed facing a white wall at different distances to capture images. By establishing a relationship between the measured distance and the true distance, the measured distance is corrected, thus obtaining a more accurate depth map. Other calibration methods include calibrating the TOF camera's time delay value using time delay deviation, performing geometric correction on the acquired raw phase images, temporal and spatial noise reduction processing, FPPN correction, Wiggling correction, and temperature error compensation, etc., to eliminate errors introduced from multiple aspects.

[0005] In addition, some patents propose new methods and devices for calibrating Time-of-Flight (TOF) cameras. For example, the patent "A Calibration Method and Device for a Time-of-Flight (TOF) Camera" proposes a TOF camera calibration method that simulates different distances between the TOF camera and the calibration plate by adjusting the TOF camera timing sequence. It collects the measured depth values ​​of the TOF camera and the actual depth values ​​of the calibration plate, analyzes the correspondence between them, and derives the compensation correction value corresponding to the depth measurement value of the TOF camera. Finally, it uses the compensation correction value to compensate the depth measurement value, thereby obtaining the calibration result. The patent "A Calibration Device and Method for a TOF Camera" proposes a TOF camera calibration device. This device has multiple calibration plates arranged at intervals, with parallel surfaces. Each calibration plate has at least one calibration surface. In addition, the device includes multiple guide rails and multiple TOF cameras, enabling rapid and accurate calibration of TOF cameras.

[0006] However, due to limitations in the principles and manufacturing processes of TOF cameras, depth distortion exhibits irregular variations when light intensity is low, and the aforementioned methods cannot explicitly model the depth error. The paper "Design of an Improved Algorithm for Harmonic and Intensity Error Correction in TOF Cameras" employs a method of fitting an error surface to the depth error caused by light intensity. Optimizing the control point matrix can better fit the actual error, but its compensation effect on the irregular depth error caused by low light intensity remains limited.

[0007] Furthermore, the different materials used in different measurement scenarios can lead to light scattering errors. Besides light scattering errors, the reflectivity of different materials in different measurement scenarios can also cause depth distortion, making it impossible to correct using a single model. The patent "Depth Compensation Method for Objects with Different Reflectivities and TOF Camera" improves depth measurement accuracy through light intensity ratio compensation, but its correction effect is limited for scenarios with significant differences in light scattering errors and reflectivity. The paper "Object Identification Using Time-of-Flight Depth Camera and Data Fusion" uses data fusion and machine learning methods to identify objects of different materials and uses the KNN (K-Nearest Neighbor) algorithm for classification, but it does not describe how to use the material classification results to correct material-related depth distortion.

[0008] Therefore, to overcome the influence of non-systematic errors such as multipath effect and light scattering caused by the measurement materials, this invention proposes a TOF camera error correction method based on material classification and identification. Machine learning is used to learn and model the frequency-related depth distortion characteristics of different materials, construct a material depth error dataset, and combine the material classification results with the depth distortion database to correct the depth data, thereby improving the imaging quality and accuracy of the TOF camera. Summary of the Invention

[0009] This invention addresses the aforementioned problems in the prior art. Therefore, a method, apparatus, system, and storage medium for simulating and modulating a Time-of-Flight (TOF) camera correction is needed. First, to address the intensity-distance error caused by varying measured light intensities in a TOF camera, a k-means clustering algorithm is used to calculate a light intensity threshold as a depth error classification standard. The light intensity threshold with the largest standard deviation of the depth error is selected as the depth error oscillation critical point. A light intensity-distance lookup table is established based on the light intensity-distance dataset to correct high-intensity depth data. A light intensity-depth random forest nonlinear model is established, and particle swarm optimization is used to optimize the model's hyperparameters for low-intensity depth data. Next, a stacking material correction classification model is constructed. For light scattering errors caused by materials in the measurement scene, the LDA (Linear Discriminant Analysis) algorithm is used to extract effective features and establish a material-depth distortion dataset. Finally, during real-time acquisition, the trained material correction classification model is used to classify the acquired object materials, and the depth data is corrected using the material-depth distortion dataset based on the material classification results, ultimately obtaining a high-quality depth image. The advantages of this invention lie in its ability to effectively reduce depth measurement errors and improve depth measurement accuracy. Furthermore, it can adaptively correct for different materials and lighting environments, making it widely applicable and highly versatile. In addition, this invention achieves real-time acquisition and correction, meeting the needs of real-time applications. This invention is an important component in the engineering application of TOF depth cameras, enabling convenient and efficient correction of TOF depth cameras and effectively avoiding the shortcomings of existing correction methods.

[0010] According to a first aspect of the present invention, an analog modulation TOF camera calibration method is provided, the method comprising:

[0011] Obtaining light intensity threshold As a standard for classifying depth errors in TOF cameras;

[0012] Based on the light intensity and distance data from the TOF camera, a light intensity-distance dataset is generated, and the light intensity threshold A corresponding to the segment with the largest depth error standard deviation is selected. pi V, the critical point for depth error oscillation ep ;

[0013] Based on the light intensity-distance dataset, using the critical point V above the depth error oscillation threshold... ep High intensity and depth data are used to construct an intensity-distance lookup table for correction;

[0014] Based on the light intensity-distance dataset, using the critical point V below the depth error oscillation threshold... epWe used low-intensity depth data to construct a light intensity-depth random forest nonlinear model and optimized the hyperparameters of the random forest.

[0015] Based on the relevant characteristics of material classification, a material-depth distortion dataset is established, and effective features are extracted from the material-depth distortion dataset to train a material correction classification model;

[0016] The material type of the object is determined by using the trained material correction classification model, and the relationship between the material and the depth distortion value is obtained through the material-depth distortion dataset to correct the depth data.

[0017] Furthermore, the light intensity threshold is obtained using the following method.

[0018] When measuring the depth distance of the target's center pixel at a reference distance using a Time-of-Flight (TOF) camera, the integration time of the TOF camera and the reflectivity of the target plate are varied to obtain depth distance measurements under different light intensity values ​​A. These measurements are then compared with the actual accurate depth values ​​to obtain the depth offset value between each set of measurements and the actual value. This depth offset value is used as the distance error d. error The light intensity-error dataset {A,d} is obtained. error};

[0019] K-means clustering analysis was performed on the collected data using an enumeration method. The k-means clustering was repeated once for each k value, and the average silhouette coefficient for that k value was calculated. The k value corresponding to the largest silhouette coefficient was selected as the final cluster number. The optimal classification k value K was obtained based on depth distance measurements under different light intensities A. AP To divide light intensity into K AP Each segment was subjected to light intensity-depth error clustering analysis;

[0020] Based on the obtained optimal k value K AP For the light intensity-error dataset {A,d} error K-means clustering was used to perform light intensity-depth error clustering analysis to obtain K AP -1 light intensity threshold

[0021] Furthermore, the generation of a light intensity-distance dataset based on the light intensity and distance data from the TOF camera specifically includes:

[0022] For the measurement range R min ~R max Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. The measurement materials with different reflectivity and roughness were used as targets and the data acquisition was repeated.

[0023] At each distance d, the precise distance between the TOF camera and the target is measured using a laser rangefinder. The TOF camera continuously measures and obtains n sets of image frames. Depth error analysis is performed on a selected region from the image center, and the distance error d is defined. error for:

[0024]

[0025]

[0026] d error =d measure -d real

[0027] in, is the average measured distance of pixel (i,j), and f is the camera frame number. The distance d is the distance measured at pixel (i,j) in the f-th frame. measure The average measured distance for the central region is given by , where a and b are the number of rows and columns of the selected region, respectively, s is the total number of pixels, and d is the average measured distance for the central region. real It is the actual distance measured by a laser rangefinder, d error For measurement error;

[0028] Based on the obtained light intensity value A and depth measurement value d for each group measure Actual value d real With depth offset value d error Obtain the light intensity-distance dataset {A,d} real ,d measure ,d error};

[0029] With light intensity threshold Using this as a boundary, the light intensity-distance dataset is segmented, and the standard deviation of the depth error is calculated. The light intensity threshold A corresponding to the maximum standard deviation is selected. pi (where i∈[1,…,K) AP -1]) serves as the critical point V for depth error oscillation. ep .

[0030] Furthermore, based on the light intensity-distance dataset, the threshold V above the depth error oscillation point is used. ep High intensity depth data is used to construct an intensity-distance lookup table for correction, specifically including:

[0031] For each preset range, data at the same measurement distance are grouped together. Using a linear polynomial fitting method, sixth-order polynomial functions d for the measurement distance and the measured light intensity are established respectively. measure =f(A), to compensate for the insufficient range of measured light intensity data;

[0032] The light intensity value is expressed in A step Divided into units, the range starts from A min ~A max For each group, a mapping between the measured values ​​and the true values ​​is established. For every two data points in the n groups of measured depth data, a cubic spline is constructed by connecting them using a higher-order polynomial function. The spline function is as follows:

[0033]

[0034] Based on the cubic spline interpolation results, an intensity-distance lookup table AD_LUT(A,d) is established. measure ,d real A represents the light intensity. step The unit is A, and the range is from A. min ~A max d real This is the actual distance, ranging from R. min ~R max .

[0035] Furthermore, based on the light intensity-distance dataset, the method utilizes a value below the depth error oscillation critical point V. ep We used low-intensity depth data to construct a light intensity-depth random forest nonlinear model and optimized the hyperparameters of the random forest, specifically including:

[0036] Based on the aforementioned light intensity-distance dataset, low-intensity depth data below the depth error wobbling threshold are used as the training sample set {A}. i ,d measure,i ,d error,i}, i=1,2,...N, where A i For the i-th measured light intensity value, d measure,i For the i-th measured distance, d error,i Let x be the measurement error of the i-th measurement. i ={A i ,d measure,i}, y i ={d error,i}, establish y i to g*x i The error correction mapping is used to determine the nonlinear model g to correct the measurement error;

[0037] A nonlinear model g is established using the random forest method. The random forest describes the nonlinear mapping M:R M→R, where a new feature vector θ is mapped to a prediction error offset y. The feature vector θ contains light intensity, depth, gradient, and pixel location information calculated from the training sample set data, with different weights. The nonlinear mapping is performed by a binary decision tree. The set of trees is learned, where T is the number of trees, each trained on a subset of the training sample set, and a single decision tree T t The given training data is recursively divided into two partitions to minimize the uncertainty of the target variable in the result subset. The median of the predicted target variable is used as the prediction error offset y to obtain the light intensity-deep random forest nonlinear model.

[0038] The particle swarm optimization algorithm is used to fine-tune the hyperparameters n_estimators and max_depth of the random forest model, and to find a set of optimal parameters for the random forest model to approximate the depth error in the training sample set space, thus obtaining the best set of hyperparameters.

[0039] Furthermore, the step of establishing a material-depth distortion dataset based on material classification-related features, extracting effective features from the material-depth distortion dataset, and training a material correction classification model specifically includes:

[0040] For the measurement range R min ~R max Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. Different common measurement materials were used as targets, and the TOF camera measurement frequency was changed to repeatedly collect data to obtain a material-depth distortion dataset.

[0041] The TOF camera captures the feature vector of a single pixel in the image at each step, which is then used as the feature vector for material classification. This includes light intensity, integration time, measurement depth d, and depth distortion d at each depth d. error Measure the frequency f and the local neighborhood information after filtering and merging the Laplace filter and Gabor filter;

[0042] The Laplace filter used is defined as follows:

[0043]

[0044] Where f(x,y) is the depth value or amplitude intensity value of pixel (x,y) in the image obtained by TOF, and Δ 2 f(x,y) represents the result of the Laplacian operator being applied to the image f(x,y), using Laplacian kernels of different sizes, 3×3 and 5×5, to merge information at different scales;

[0045] The Gabor filter expression used is as follows:

[0046]

[0047] Where x and y represent pixel coordinates, λ represents wavelength, θ represents direction, ψ represents phase offset, σ represents the standard deviation of the Gaussian envelope, and γ represents ellipticity; x' and y' are coordinate axes along the Gabor wavelet direction, calculated as follows:

[0048]

[0049] The Gabor filter parameter combination is: λ = 8, θ = 0°, 45°, 90° and 135°, ψ = 0, σ = 3, γ = 0.5;

[0050] Using Laplacian and Gabor filters to filter depth and intensity images and incorporate them into feature vectors allows the classifier to learn the relationship between error and depth discontinuities, as well as to extract texture information at different orientations and frequencies.

[0051] The linear discriminant analysis (LDA) method was used to reduce the dimensionality of the material-depth distortion dataset, obtaining the optimal feature dimension L. The objective function of LDA is:

[0052]

[0053] The optimal projection vector is obtained by solving the objective function of LDA, so as to distinguish different categories of targets in low-dimensional space.

[0054] A combination of SVM, KNN, and random forest was selected as the base classifier, and XGBoost was used as the meta classifier for training to obtain a material correction classification model.

[0055] Furthermore, the process of determining the material type of the object using the trained material correction classification model, obtaining the relationship between the material and depth distortion values ​​through a material-depth distortion dataset, and correcting the depth data specifically includes:

[0056] The feature vectors of the pixels in the real-time acquired depth image are input into the trained material correction classification model, and the material type of the object is determined based on the material classification results.

[0057] Based on the material-depth distortion dataset, the mapping relationship between the material and the depth distortion value is obtained, and the depth data is corrected.

[0058] According to a second technical solution of the present invention, an analog modulation TOF camera correction device is provided, the device comprising: a data acquisition unit configured to acquire a light intensity threshold. As a depth error classification standard for TOF cameras, the data generation unit is configured to generate a light intensity-distance dataset based on the light intensity and distance data from the TOF camera, and select the light intensity threshold A corresponding to the segment with the largest standard deviation of depth error. pi V, the critical point for depth error oscillation ep The first correction unit is configured to, based on the light intensity-distance dataset, utilize a value above the depth error oscillation threshold V. ep High intensity-depth data is used to construct an intensity-distance lookup table for correction; the model building unit is configured to, based on the intensity-distance dataset, utilize data below the depth error oscillation critical point V. ep The system uses low-intensity depth data to construct an intensity-depth random forest nonlinear model and optimizes the random forest hyperparameters. The model training unit is configured to establish a material-depth distortion dataset based on material classification-related features, extract effective features from the material-depth distortion dataset, and train a material correction classification model. The second correction unit is configured to determine the material type of the object using the trained material correction classification model, obtain the relationship between the material and the depth distortion value through the material-depth distortion dataset, and correct the depth data.

[0059] According to a third technical solution of the present invention, an analog modulation TOF camera correction system is provided, the system comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method described above.

[0060] According to a fourth technical solution of the present invention, a non-transitory computer-readable storage medium storing instructions is provided, which, when executed by a processor, performs the method described above.

[0061] The analog modulation TOF camera calibration method, apparatus, system, and storage medium according to various embodiments of the present invention have at least the following technical effects:

[0062] This invention offers several advantages because it employs secondary modeling based on the results of various anomaly distributions. First, it considers multiple statistical assumptions, making it adaptable to different data distributions in practical applications. Second, the method uses a deep learning model to describe a complex mathematical relationship, which is difficult for enterprises to counteract through data generation methods. Furthermore, this method can be deployed on a server and automatically run by connecting to a database during the inference phase, requiring only minimal human and material resources. Attached Figure Description

[0063] In drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with or without letter suffixes may indicate different instances of similar parts. The drawings generally illustrate various embodiments by way of example rather than limitation and, together with the description and claims, serve to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and not intended to be exhaustive or exclusive embodiments of the apparatus or method.

[0064] Figure 1 A flowchart of an analog modulation TOF camera calibration method according to an embodiment of the present invention is shown.

[0065] Figure 2 A schematic diagram of the structure of a classification model according to an embodiment of the present invention is shown.

[0066] Figure 3 A flowchart illustrating the classification model establishment process of an analog modulation TOF camera correction method according to an embodiment of the present invention is shown.

[0067] Figure 4 A flowchart illustrating the forgery detection model establishment of an analog modulation TOF camera correction method according to an embodiment of the present invention is shown.

[0068] Figure 5 A structural diagram of an analog modulation TOF camera correction device according to an embodiment of the present invention is shown. Detailed Implementation

[0069] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted, as long as it does not disrupt the logical consistency between them and render the entire process impossible.

[0070] This invention provides a method for calibrating a simulated modulation TOF camera, specifically a method for calibrating a simulated modulation TOF camera based on multiple anomaly distributions in online monitoring data and deep learning.

[0071] like Figure 1 As shown, the method specifically includes the following steps:

[0072] Step S1: Calculate the light intensity threshold using the k-means clustering algorithm, which serves as the depth error classification standard for the TOF camera.

[0073] In some embodiments, step S1 specifically includes the following steps:

[0074] S11: Use a TOF camera to measure the depth distance of the target's center pixel at a distance of 150cm. Change the integration time of the TOF camera and the reflectivity of the target plate to obtain depth distance measurements under different light intensities. Compare these measurements with the actual accurate depth values ​​to obtain the depth offset value {A,d} between each set of measurements and the actual values. error}

[0075] S12: Perform k-means clustering analysis on the collected data using an enumeration method. Repeat the k-means clustering for each k value (2-12), calculate the average silhouette coefficient for that k value, and finally select the k value corresponding to the largest silhouette coefficient as the final cluster number. The optimal k value K for classification is obtained based on the data collected in S11. AP =2, such as Figure 2 This allows the light intensity to be divided into two stages for light intensity depth error clustering analysis.

[0076] S13: Based on the optimal k value obtained in S12, perform the analysis on the light intensity-error dataset {A,d} collected in S21. error Perform k-means clustering analysis to obtain the light intensity threshold A. P =500 LSB, such as Figure 3 The optimal value of k, K, is obtained based on S12. AP The light intensity-error data {A,d} collected in S11 error K-means clustering was used to perform light intensity depth error clustering analysis, resulting in a light intensity threshold A. p =500LSB

[0077] Step S2: Conduct a material depth distortion data acquisition experiment using a TOF camera, with a light intensity threshold A. p Using the boundary as a reference, generate TOF camera light intensity-error depth distortion datasets separately, and extract the light intensity threshold with the maximum standard deviation of the error as the depth error swing critical point V. ep .

[0078] In some embodiments, step S2 specifically includes the following steps:

[0079] S21: Within a measurement range of 50–240 cm, 20 sets of depth distances were measured in 10 cm increments. The TOF camera was moved at each measurement distance d, and the integration time of the TOF camera was changed to obtain the measurement depth values ​​from low to high light intensity. Standard diffuse reflective plates with reflectivities of 18%, 50%, and 90% were used as targets, and the data acquisition was repeated.

[0080] S22: At each distance d, the precise distance between the TOF camera and the target is measured using a laser rangefinder. 30 frames are continuously measured using the TOF camera. Depth error analysis is performed on the central 20×20 pixel region of the image, and the distance error is defined as d. error .

[0081]

[0082]

[0083] d error =d measure -d real

[0084] Where, d mi,j is the average measured distance of pixel (i,j), and f is the number of frames per second of the camera. It is the distance measured at pixel (i,j) in the f-th frame, where n = 30, d measure The average measured distance for the central region is given by , where a and b are the number of rows and columns of the selected region, respectively, s is the total number of pixels, and d is the average measured distance for the central region. real It is the actual distance measured by a laser rangefinder, d error This is for measurement error.

[0085] S23: Thus, the light intensity value A and depth measurement value d for each group are obtained. measure Actual value d real With depth offset value d error Thus, the light intensity-depth dataset {A,d} of the TOF camera is obtained. real ,d measure ,d error}

[0086] S24: Using light intensity threshold A p Using 500 LSB as the boundary, the standard deviation of depth error is calculated for segmented TOF light intensity-distance datasets. At this point, the critical point of depth error oscillation is V. ep =500LSB.

[0087] S25: The specific correction process for light intensity error is as follows: (1) The light intensity value A is obtained by measuring the light intensity value using the TOF camera. measure With depth value d measure (2) If A measure The depth error swing critical point V ep Then the light intensity value A can be found. measure The light intensity range AD_LUT(A) is used to correct for depth offset to obtain d. measure Corresponding to d real (3) If A measure Less than the depth error oscillation critical point Vep Then, inputting the trained light intensity-deep random forest nonlinear model yields the error prediction value e. predict The corrected depth value is then calculated.

[0088] Step S3: Based on the depth error oscillation critical point V obtained in S2, calculate the depth error oscillation critical point V from the TOF camera light intensity-distance dataset. ep =500LSB is used to divide the light intensity into three parts. For the high light intensity stage above the depth error swing threshold, cubic B-spline interpolation is performed to establish a light intensity-distance lookup table as a high light intensity depth correction model.

[0089] In some embodiments, step S3 specifically includes the following steps:

[0090] S31: For data at the same measurement distance within every 10cm range, a linear polynomial fitting method is used to establish sixth-order polynomial functions for the measurement distance and the measured light intensity, i.e., d measure =f(A), in order to compensate for the lack of range of measured light intensity data.

[0091] S32: Convert the light intensity value to A step The data was divided into units of 50 LSB, ranging from 500 LSB to 2000 LSB. For each group, a mapping between measured and true values ​​was established. A relatively smooth cubic spline interpolation method was used for interpolation fitting. Specifically, a cubic spline was constructed by connecting every two data points in the 20 groups of measured depth data using a high-order polynomial function. The spline function is as follows:

[0092]

[0093] S33: Based on the cubic spline interpolation results obtained in S31, to facilitate the use of the TOF camera in different depth ranges, a suitable true depth offset value can be quickly found for compensation. A light intensity-distance lookup table AD_LUT(A,d) is established. measure ,d real A represents light intensity in units of 50 LSB, ranging from 500 LSB to 2000 LSB. real This refers to the actual distance, ranging from 50 to 240 cm.

[0094] Step S4: Based on the depth error oscillation critical point V obtained in S2 from the TOF camera intensity-distance dataset. ep The data is divided into segments. For low-intensity depth data below the depth error swing threshold, a light intensity-depth random forest nonlinear model is established as a low-intensity depth correction model. The particle swarm optimization algorithm is used to optimize the hyperparameters n_estimators and max_depth of the random forest.

[0095] In some embodiments, step S4 specifically includes the following steps:

[0096] S41: Construct a sample dataset. Using the light intensity-depth dataset from the TOF camera obtained in S3 above, use the low-intensity depth data below the depth error wobbling threshold as the training sample set {A}. i ,d measure,i ,d error,i}, i=1,2,...N, where A i To measure the light intensity value, d measure,i To measure the distance to the camera, d error,i Let x be the camera measurement error. i ={A i ,d measure,i}, y i ={d error,i}, establish y i to f*x i The error correction mapping is used to determine the nonlinear model f and to correct the measurement error.

[0097] S42: Construct a regression prediction model. A random forest model is used to build the regression prediction model. The random forest describes the nonlinear mapping M:R. M →R, where a new feature vector θ is mapped to a prediction offset y. This mapping is done by a binary decision tree. The decision tree is learned from a set of trees (T is the number of trees), each trained on a subset of the training sample set. A single decision tree T t The given training data is recursively split into two partitions to minimize the uncertainty of the target variable in the resulting subset, and the final prediction offset y′ is calculated as the median of the predicted target variable.

[0098] S43: The particle swarm optimization algorithm was used to fine-tune the hyperparameters n_estimators and max_depth of the RF model. An optimal set of parameters for the random forest model was found to approximate the depth error in the training sample space. It was found that when n_estimators = 20 and max_depth = 8, the convergence speed and prediction performance of the RF model were the best.

[0099] Step S5: Collect and extract material classification-related features, establish a material-depth distortion dataset, use the LDA algorithm to extract effective features, and train a material correction classification model based on the stacking strategy.

[0100] In some embodiments, step S5 specifically includes the following steps:

[0101] S51: Within a range of 60–500 cm, 45 depth distance measurements were performed in 10 cm increments. The TOF camera was repeatedly moved back and forth at each distance d, and the precise distance between the TOF camera and the target was measured using a laser rangefinder. 30 frames were continuously measured using the TOF camera. The integration time of the TOF camera was varied to obtain depth values ​​under low and high light intensities. Twenty different types of materials were used as samples, including commonly used metals, wood, plastics, and fabrics, as well as a standard diffuse reflectance plate used for testing as the target. The TOF camera measurement frequency was varied to repeatedly acquire data, resulting in a material-depth distortion dataset.

[0102] S52: The feature vector of a single pixel in each image acquired by a TOF camera consists of light intensity, integration time, measurement depth d, and depth distortion d at each distance d. error The measurement frequency f and the local neighborhood information after filtering and merging the Laplace filter and Gabor filter constitute the feature vector for material classification.

[0103] The Laplace filter used can be defined as follows:

[0104]

[0105] Where f(x,y) is the depth value or amplitude intensity value of pixel (x,y) in the image obtained by TOF, and Δ 2 f(x,y) represents the result of applying the Laplacian operator to the image f(x,y). Laplacian kernels of different sizes, 3×3 and 5×5, are used to combine information at different scales.

[0106] The Gabor filter expression used is as follows:

[0107]

[0108] Where x and y represent pixel coordinates, λ represents wavelength, θ represents direction, ψ represents phase offset, σ represents the standard deviation of the Gaussian envelope, and γ represents ellipticity. x' and y' are coordinate axes along the Gabor wavelet direction, calculated using the following formula:

[0109]

[0110] The Gabor filter parameter combination is: λ = 8, θ = 0°, 45°, 90° and 135°, ψ = 0, σ = 3, γ = 0.5.

[0111] Using Laplacian and Gabor filters to filter depth and intensity images and incorporate them into feature vectors allows the classifier to learn the relationship between error and depth discontinuities, as well as extract texture information at different orientations and frequencies.

[0112] S53: Linear Discriminant Analysis (LDA) is used to reduce the dimensionality of the material-depth distortion dataset, yielding an optimal feature dimension L=4, which saves storage space and reduces computational overhead. The objective function of LDA is:

[0113]

[0114] Solving the above equation yields the optimal projection vector, enabling better differentiation of different target categories in a low-dimensional space.

[0115] S54: Construct a classifier based on the stacking concept for material classification to improve the model's generalization ability and accuracy. Three machine learning methods—SVM, KNN, and a combination of RF—are selected as base classifiers, and XGBoost is used as a meta-classifier for training to obtain a material correction classification model.

[0116] Step S6: During real-time acquisition, the material type of the object is determined using a material correction classification model. The relationship between the material and the depth distortion value is obtained through the material-depth distortion dataset, and the depth data is corrected.

[0117] In some embodiments, step S6 specifically includes the following steps:

[0118] S61: Input the feature vector of the depth image pixels acquired in real time into the trained material correction classification model, and determine the material type of the object based on the material classification result.

[0119] S62: Using the material-depth distortion dataset acquired by S51, obtain the mapping relationship between this material and the depth distortion value, and correct the depth data.

[0120] In summary, this invention proposes a TOF camera correction method based on analog modulation. First, addressing the intensity-distance error caused by varying measured light intensities and distances in TOF cameras, a k-means clustering algorithm is used to calculate a light intensity threshold as a depth error classification standard. The light intensity threshold with the largest standard deviation of the depth error is selected as the depth error oscillation critical point. A light intensity-distance lookup table is established based on the light intensity-distance dataset to correct high-intensity depth data. A light intensity-depth random forest nonlinear model is established, and particle swarm optimization is used to optimize the model's hyperparameters to correct low-intensity depth data. Then, an optimized stacking material correction classification model is constructed. For light scattering errors caused by materials in the measurement scene, the LDA algorithm is used to extract effective features and establish a material-depth distortion dataset. Finally, during real-time acquisition, the trained material correction classification model is used to classify the acquired object materials, and the depth data is corrected using the material-depth distortion dataset based on the material classification results, ultimately obtaining a high-quality depth image. The advantages of this invention are that it effectively reduces depth measurement errors and improves depth measurement accuracy; it can also adaptively correct for different materials and lighting environments, has a wide range of applications, and good versatility. Furthermore, this invention enables real-time acquisition and calibration, meeting the needs of real-time applications. This invention is a crucial component in the engineering application of TOF depth cameras, providing convenient and efficient calibration for TOF depth cameras and effectively addressing the shortcomings of existing calibration methods.

[0121] This invention also provides an analog modulation TOF camera calibration device, such as... Figure 5 As shown, the device 500 includes:

[0122] Data acquisition unit 501 is configured to acquire light intensity threshold. As a standard for classifying depth errors in TOF cameras;

[0123] Data generation unit 502 is configured to generate a light intensity-distance dataset based on the light intensity and distance data from the TOF camera, and select the light intensity threshold A corresponding to the segment with the largest standard deviation of depth error. pi V, the critical point for depth error oscillation ep ;

[0124] The first correction unit 503 is configured to, based on the light intensity-distance dataset, utilize a value above the depth error oscillation threshold V. ep High intensity and depth data are used to construct an intensity-distance lookup table for correction;

[0125] Model building unit 504 is configured to, based on the light intensity-distance dataset, utilize a depth error oscillation threshold V. epWe used low-intensity depth data to construct a light intensity-depth random forest nonlinear model and optimized the hyperparameters of the random forest.

[0126] The model training unit 505 is configured to establish a material-depth distortion dataset based on material classification-related features, extract effective features from the material-depth distortion dataset, and train a material correction classification model.

[0127] The second correction unit 506 is configured to determine the material type of the object using the trained material correction classification model, obtain the relationship between the material and the depth distortion value through the material-depth distortion dataset, and correct the depth data.

[0128] In some embodiments, the data acquisition unit is further configured to acquire the light intensity threshold by means of the following method.

[0129] When measuring the depth distance of the target's center pixel at a reference distance using a Time-of-Flight (TOF) camera, the integration time of the TOF camera and the reflectivity of the target plate are varied to obtain depth distance measurements under different light intensity values ​​A. These measurements are then compared with the actual accurate depth values ​​to obtain the depth offset value between each set of measurements and the actual value. This depth offset value is used as the distance error d. error The light intensity-error dataset {A,d} is obtained. error};

[0130] K-means clustering analysis was performed on the collected data using an enumeration method. The k-means clustering was repeated once for each k value, and the average silhouette coefficient for that k value was calculated. The k value corresponding to the largest silhouette coefficient was selected as the final cluster number. The optimal classification k value K was obtained based on depth distance measurements under different light intensities A. AP To divide light intensity into K AP Each segment was subjected to light intensity-depth error clustering analysis;

[0131] Based on the obtained optimal k value K AP For the light intensity-error dataset {A,d} error K-means clustering was used to perform light intensity-depth error clustering analysis to obtain K AP -1 light intensity threshold

[0132] In some embodiments, the data generation unit is further configured to:

[0133] For the measurement range R min ~R max Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. The measurement materials with different reflectivity and roughness were used as targets and the data acquisition was repeated.

[0134] At each distance d, the precise distance between the TOF camera and the target is measured using a laser rangefinder. The TOF camera continuously measures and obtains n sets of image frames. Depth error analysis is performed on a selected region from the image center, and the distance error d is defined. error for:

[0135]

[0136]

[0137] d error =d measure -d real

[0138] Where, d mi,j is the average measured distance of pixel (i,j), and f is the camera frame number. The distance d is the distance measured at pixel (i,j) in the f-th frame. measure The average measured distance for the central region is given by , where a and b are the number of rows and columns of the selected region, respectively, s is the total number of pixels, and d is the average measured distance for the central region. real It is the actual distance measured by a laser rangefinder, d error For measurement error;

[0139] Based on the obtained light intensity value A and depth measurement value d for each group measure Actual value d real With depth offset value d error Obtain the light intensity-distance dataset {A,d} real ,d measure ,d error}

[0140] In some embodiments, the first correction unit is further configured to:

[0141] For each preset range, data at the same measurement distance are grouped together. Using a linear polynomial fitting method, sixth-order polynomial functions d for the measurement distance and the measured light intensity are established respectively. measure =f(A), to compensate for the insufficient range of measured light intensity data;

[0142] The light intensity value is expressed in A stepThe data is divided into units of 50 LSB, ranging from 500 LSB to 2000 LSB. A mapping between measured and true values ​​is established for each group. For every two data points in the n groups of depth measurements, a cubic spline is constructed using a higher-order polynomial function. The spline function is as follows:

[0143]

[0144] Based on the cubic spline interpolation results, an intensity-distance lookup table AD_LUT(A,d) is established. measure ,d real A represents light intensity in units of 50 LSB, ranging from 500 LSB to 2000 LSB, and d represents light intensity. real This refers to the actual distance, ranging from 50 to 240 cm.

[0145] In some embodiments, the model building unit is further configured as follows:

[0146] Based on the aforementioned light intensity-distance dataset, low-intensity depth data below the depth error wobbling threshold are used as the training sample set {A}. i ,d measure,i ,d error,i}, i=1,2,...N, where A i For the i-th measured light intensity value, d measure,i For the i-th measured distance, d error,i Let x be the measurement error of the i-th measurement. i ={A i ,d measure,i}, y i ={d error,i}, establish y i to g*x i The error correction mapping is used to determine the nonlinear model g to correct the measurement error;

[0147] A nonlinear model g is established using the random forest method. The random forest describes the nonlinear mapping M:R M →R, where a new feature vector θ is mapped to a prediction error offset y. The feature vector θ contains light intensity, depth, gradient, and pixel location information calculated from the training sample set data, with different weights. The nonlinear mapping is performed by a binary decision tree. The set of trees is learned, where T is the number of trees, each trained on a subset of the training sample set, and a single decision tree T t The given training data is recursively divided into two partitions to minimize the uncertainty of the target variable in the result subset. The median of the predicted target variable is used as the prediction error offset y to obtain the light intensity-deep random forest nonlinear model.

[0148] The particle swarm optimization algorithm is used to fine-tune the hyperparameters n_estimators and max_depth of the random forest model, and to find a set of optimal parameters for the random forest model to approximate the depth error in the training sample set space, thus obtaining the best set of hyperparameters.

[0149] In some embodiments, the model training unit is further configured to:

[0150] For the measurement range R min ~R max Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. Different common measurement materials were used as targets, and the TOF camera measurement frequency was changed to repeatedly collect data to obtain a material-depth distortion dataset.

[0151] The TOF camera captures the feature vector of a single pixel in the image at each step, which is then used as the feature vector for material classification. This includes light intensity, integration time, measurement depth d, and depth distortion d at each depth d. error Measure the frequency f and the local neighborhood information after filtering and merging the Laplace filter and Gabor filter;

[0152] The Laplace filter used is defined as follows:

[0153]

[0154] Where f(x,y) is the depth value or amplitude intensity value of pixel (x,y) in the image obtained by TOF, and Δ 2 f(x,y) represents the result of the Laplacian operator being applied to the image f(x,y), using Laplacian kernels of different sizes, 3×3 and 5×5, to merge information at different scales;

[0155] The Gabor filter expression used is as follows:

[0156]

[0157] Where x and y represent pixel coordinates, λ represents wavelength, θ represents direction, ψ represents phase offset, σ represents the standard deviation of the Gaussian envelope, and γ represents ellipticity; x' and y' are coordinate axes along the Gabor wavelet direction, calculated as follows:

[0158]

[0159] The Gabor filter parameter combination is: λ = 8, θ = 0°, 45°, 90° and 135°, ψ = 0, σ = 3, γ = 0.5;

[0160] Using Laplacian and Gabor filters to filter depth and intensity images and incorporate them into feature vectors allows the classifier to learn the relationship between error and depth discontinuities, as well as to extract texture information at different orientations and frequencies.

[0161] The linear discriminant analysis (LDA) method was used to reduce the dimensionality of the material-depth distortion dataset, obtaining the optimal feature dimension L. The objective function of LDA is:

[0162]

[0163] The optimal projection vector is obtained by solving the objective function of LDA, so as to distinguish different categories of targets in low-dimensional space.

[0164] A combination of SVM, KNN, and random forest was selected as the base classifier, and XGBoost was used as the meta classifier for training to obtain a material correction classification model.

[0165] In some embodiments, the second correction unit is further configured to:

[0166] The feature vectors of the pixels in the real-time acquired depth image are input into the trained material correction classification model, and the material type of the object is determined based on the material classification results.

[0167] Based on the material-depth distortion dataset, the mapping relationship between the material and the depth distortion value is obtained, and the depth data is corrected.

[0168] It should be noted that this analog modulation TOF camera correction device is based on the same technical concept as the previously described method and can achieve the same beneficial effect, which will not be elaborated here.

[0169] This invention provides an analog modulation TOF camera calibration system, the system comprising:

[0170] Memory, used to store computer programs;

[0171] A processor for executing the computer program to implement the method as described in any of the above embodiments.

[0172] This invention provides a non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform the methods described in any of the above embodiments.

[0173] Furthermore, although exemplary embodiments have been described herein, their scope includes any and all embodiments based on the invention that have equivalent elements, modifications, omissions, combinations (e.g., schemes involving intersections of various embodiments), adaptations, or alterations. Elements in the claims will be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or during the implementation of this application, and such examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered illustrative only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0174] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments can be used by those skilled in the art when reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the invention. This should not be construed as an intention that a feature of an unclaimed invention is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular embodiment of the invention. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated to be combined with each other in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A method of correcting an analog modulated TOF camera, characterized by, The method includes: Acquiring light intensity threshold as a depth error classification criterion for a TOF camera; According to the light intensity and distance data of the TOF camera, a light intensity-distance data set is generated, and a light intensity threshold corresponding to a section with the largest depth error standard deviation is selected as a depth error swing critical point ; According to the light intensity-distance data set, a light intensity-distance lookup table is constructed using high light intensity depth data above a depth error swing threshold for correction; According to the light intensity-distance data set, a light intensity-depth random forest nonlinear model is constructed with low light intensity depth data below a depth error swing critical point and random forest hyperparameters are optimized; Based on the relevant characteristics of material classification, a material-depth distortion dataset is established, and effective features are extracted from the material-depth distortion dataset to train a material correction classification model; The material type of the object is determined by using the trained material correction classification model, and the relationship between the material and the depth distortion value is obtained through the material-depth distortion dataset to correct the depth data.

2. The method according to claim 1, characterized in that, The light intensity threshold is obtained using the following method. : When measuring the depth distance of the target's center pixel at a reference distance using a Time-of-Flight (TOF) camera, the integration time of the TOF camera and the reflectivity of the target plate are varied to obtain depth distance measurements under different light intensity values ​​A. These measurements are then compared with the actual accurate depth values ​​to obtain the depth offset value between each set of measurements and the actual value. This depth offset value is used as the distance error. The light intensity-error dataset was obtained. ; K-means clustering analysis was performed on the collected data using an enumeration method. The k-means clustering was repeated once for each k value, and the average silhouette coefficient for that k value was calculated. The k value corresponding to the largest silhouette coefficient was selected as the final cluster number. The optimal k value for classification was obtained based on depth distance measurements under different light intensities A. To divide light intensity into Each segment was subjected to light intensity-depth error clustering analysis; Based on the obtained optimal k value For the light intensity-error dataset K-means clustering was used to perform light intensity depth error clustering analysis, and the results were obtained. individual light intensity thresholds .

3. The method according to claim 1, characterized in that, The generation of a light intensity-distance dataset based on the light intensity and distance data from the TOF camera specifically includes: For measurement range Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. The measurement materials with different reflectivity and roughness were used as targets and the data acquisition was repeated. At each distance d, the precise distance between the TOF camera and the target is measured using a laser rangefinder. The TOF camera continuously measures and obtains n sets of image frames. Depth error analysis is performed on a selected region from the image center, and the distance error is defined. for: ； ； ； in, It is a pixel Average measurement distance, f It is the camera's frame number. It is in the f Pixels in a frame The measured distance The average measured distance for the central region is given by , where a and b are the number of rows and columns of the selected region, respectively, and s is the total number of pixels. It is the actual distance measured by a laser rangefinder. For measurement error; Based on the light intensity values ​​obtained for each group Depth measurement value actual value With depth offset value Obtain the light intensity-distance dataset .

4. The method according to claim 1, characterized in that, Based on the light intensity-distance dataset, the threshold above the depth error oscillation is used. High intensity depth data is used to construct an intensity-distance lookup table for correction, specifically including: For each preset range, data at the same measurement distance are grouped together, and a linear polynomial fitting method is used to establish sixth-order polynomial functions for the measurement distance and the measured light intensity. This is to compensate for the insufficient range of measured light intensity data; light intensity value Divided by unit, the range is from For each group, a mapping between the measured values ​​and the true values ​​is established. For every two data points in the n groups of measured depth data, a cubic spline is constructed using a higher-order polynomial function. The spline function is as follows: ； Based on the cubic spline interpolation results, a light intensity-distance lookup table is established. , For light intensity As a unit, the range is from , This is the actual distance, and the range is... .

5. The method according to claim 1, characterized in that, Based on the light intensity-distance dataset, the method utilizes a depth error oscillation threshold. We used low-intensity depth data to construct a light intensity-depth random forest nonlinear model and optimized the hyperparameters of the random forest, specifically including: Based on the aforementioned light intensity-distance dataset, low-intensity depth data below the depth error wobbling threshold are used as the training sample set. ,in For the i-th measured light intensity value, For the i-th measured distance, Let the i-th measurement error be... , ,Establish The error correction mapping is used to determine the nonlinear model g to correct the measurement error; A nonlinear model g is constructed using the random forest method. The random forest describes the nonlinear mapping. A new feature vector θ is mapped to a prediction error offset y. The feature vector θ contains light intensity, depth, gradient, and pixel location information calculated from the training sample set data, with different weights. The nonlinear mapping is performed by a binary decision tree. The set of trees is learned, where T is the number of trees, each trained on a subset of the training sample set, for a single decision tree. The given training data is recursively split into two partitions to minimize the uncertainty of the target variable in the resulting subset, and the median of the predicted target variable is used as the prediction error offset. We obtained the light intensity-deep random forest nonlinear model; The particle swarm optimization algorithm is used to fine-tune the hyperparameters n_estimators and max_depth of the random forest model, and to find a set of optimal parameters for the random forest model to approximate the depth error in the training sample set space, thus obtaining the best set of hyperparameters.

6. The method according to claim 1, characterized in that, The process of establishing a material-depth distortion dataset based on material classification-related features, extracting effective features from the material-depth distortion dataset, and training a material correction classification model specifically includes: For measurement range Within, in units of step size S, a total of The depth distance was measured by moving the TOF camera at each measurement distance d and changing the integration time of the TOF camera to obtain the measurement depth values ​​from low light intensity to high light intensity. Different common measurement materials were used as targets, and the TOF camera measurement frequency was changed to repeatedly collect data to obtain a material-depth distortion dataset. The TOF camera captures the feature vector of a single pixel in the image at each step, which is then used as the feature vector for material classification. This includes light intensity, integration time, and measurement depth. d Each depth d Depth distortion on Measurement frequency f And the local neighborhood information after filtering and merging the Laplace filter and the Gabor filter; The Laplace filter used is defined as follows: ； in, It is a Time-of-Flight (TOF) method that obtains pixels in an image. Depth value or amplitude intensity value, This indicates that the Laplacian operator is applied to the image. The results above use Laplace kernels of different sizes, 3×3 and 5×5, to merge information at different scales; The Gabor filter expression used is as follows: ； Where x and y represent pixel coordinates, λ represents wavelength, θ represents direction, ψ represents phase offset, σ represents the standard deviation of the Gaussian envelope, and γ represents ellipticity; x' and y' are coordinate axes along the Gabor wavelet direction, calculated as follows: ； The Gabor filter parameter combination is: λ=8, θ=0°, 45°, 90° and 135°, ψ=0, σ=3, γ=0.5; Using Laplacian and Gabor filters to filter depth and intensity images and include them in the feature vector allows the classifier to learn the relationship between error and depth discontinuities, as well as extract texture information at different orientations and frequencies. The linear discriminant analysis (LDA) method was used to reduce the dimensionality of the material-depth distortion dataset, obtaining the optimal feature dimension L. The objective function of LDA is: ； The optimal projection vector is obtained by solving the objective function of LDA, so as to distinguish different categories of targets in low-dimensional space; A combination of SVM, KNN, and random forest was selected as the base classifier, and XGBoost was used as the meta classifier for training to obtain a material correction classification model.

7. The method according to claim 1, characterized in that, The process involves determining the material type of the object using a trained material correction classification model, obtaining the relationship between the material and depth distortion values ​​through a material-depth distortion dataset, and correcting the depth data. Specifically, this includes: The feature vectors of the pixels in the real-time acquired depth image are input into the trained material correction classification model, and the material type of the object is determined based on the material classification results. Based on the material-depth distortion dataset, the mapping relationship between the material and the depth distortion value is obtained, and the depth data is corrected.

8. A calibration device for an analog modulation TOF camera, characterized in that, The device includes: The data acquisition unit is configured to acquire the light intensity threshold. , as a depth error classification standard for TOF cameras; The data generation unit is configured to generate a light intensity-distance dataset based on the light intensity and distance data from the TOF camera, and select the light intensity threshold corresponding to the segment with the largest standard deviation of depth error. As the critical point of depth error swing ; The first correction unit is configured to, based on the light intensity-distance dataset, utilize a oscillation threshold above the depth error threshold. High intensity and depth data are used to construct an intensity-distance lookup table for correction; The model building unit is configured to, based on the light intensity-distance dataset, utilize a depth error oscillation threshold. We used low-intensity depth data to construct a light intensity-depth random forest nonlinear model and optimized the hyperparameters of the random forest. The model training unit is configured to build a material-depth distortion dataset based on material classification-related features, extract effective features from the material-depth distortion dataset, and train a material correction classification model. The second correction unit is configured to determine the material type of the object using a trained material correction classification model, obtain the relationship between the material and the depth distortion value through the material-depth distortion dataset, and correct the depth data.

9. An analog modulation TOF camera calibration system, characterized in that: The system includes: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform the method according to any one of claims 1 to 7.

Images