A method and system for dynamic compensation of TOF ranging errors

Abstract

The application relates to the field of electric digital data processing, in particular to a TOF ranging error dynamic compensation method and system. The method comprises the following steps: collecting multi-phase original integral charge of a sensor, calculating original infrared amplitude and original ranging depth; a first local window is constructed, energy weight is calculated based on the infrared amplitude, weighted variance is calculated by using the energy weight, and a depth fluctuation characteristic value is obtained; the time difference of the depth fluctuation characteristic value and the original ranging depth is calculated, an asymmetric direction activation factor and an auxiliary gating factor are constructed and time sequence verification is performed, a fusion gating factor and a time sequence asymmetric direction covariance characteristic are obtained; a second and a third local window are constructed respectively, and weighted fusion is completed based on a continuous error suppression operator and an edge perception weight, and compensated depth data is obtained. The application has the effect of improving the TOF ranging precision in a dynamic scene.

Description

Technical Field

[0001] This application relates to the field of electronic digital data processing, and in particular to a method and system for dynamic compensation of TOF ranging error. Background Technology

[0002] With the rapid development of 3D machine vision technology, Time-of-Flight (TOF) depth sensing technology has been widely applied in various fields due to its advantages such as fast response speed and high resolution. However, in practical applications, the surface of the target object is often composed of different materials. When the target object moves quickly, the same pixel of the TOF sensor will receive reflected light signals from different spatial locations and different material surfaces in adjacent time frames. Due to the slight asynchrony between the abrupt changes in depth and the abrupt changes in the amplitude of the infrared signal in terms of physical acquisition on the time scale, existing TOF depth data processing algorithms based on continuous frame data will misjudge the "amplitude change boundary caused by material reflectivity" as the "physical depth jump boundary" when processing such continuous frame data, thus generating a large number of flying points at the edge of the moving target. At the same time, because the multi-frame fusion network in the time dimension cannot distinguish between "real physical motion" and "temporal flicker caused by noise", it leads to severe depth trailing in the direction of motion.

[0003] In existing technologies, when facing dynamic scenes, the continuous exposure of the sensor in a very short time will mix the afterimage of the moving object with the real background to form a trailing area, which is prone to serious ranging errors. Current deep learning networks often cannot distinguish between the leading edge of the real object that is approaching and the trailing area of ​​the object leaving, resulting in blurred edges in the final output 3D depth map. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a method and system for dynamic compensation of TOF ranging errors.

[0005] Firstly, this application provides a dynamic compensation method for TOF ranging errors, including: The original integrated charge of the sensor in multiple phases is collected; the original infrared amplitude and the original ranging depth are calculated based on the original integrated charge. A first local window is constructed with the current pixel as the center. The energy weight of each pixel is calculated based on the original infrared amplitude. The weighted variance of the original ranging depth within the first local window is calculated using the energy weight to obtain the depth fluctuation feature value. Calculate the depth fluctuation feature value of each pixel and the time difference of the original ranging depth. Based on the time difference of the depth fluctuation feature value and the time difference of the original ranging depth, construct an asymmetric orientation activation factor. In view of the case that the background depth fluctuation feature value increases after the object leaves, construct an auxiliary gating factor based on the change of the original signal. Fuse the asymmetric orientation activation factor and the auxiliary gating factor to obtain a fused gating factor. Calculate the temporal asymmetric directional covariance feature based on the temporal variance of the fusion gate factor and the depth fluctuation feature value; A second local window and a third local window, each larger than the first local window, are constructed with the current pixel as the center. A continuous error suppression operator is calculated based on the temporal asymmetric directional covariance features within the second local window. An edge-aware weight is obtained based on the continuous error suppression operator and the original ranging depth of the pixels within the third local window. The original ranging depth of each pixel within the third local window is weighted and fused using the edge-aware weight to obtain the compensated depth data. The compensated depth data is used as the final output TOF ranging result.

[0006] Optionally, the energy weight of each pixel is calculated based on the original infrared amplitude, including: Calculate the sum of squares of the original infrared amplitude of each pixel within the first local window; The ratio of the square of the original infrared amplitude of each pixel to the sum of the squares is used as the energy weight of each pixel.

[0007] Optionally, based on the time difference of the depth fluctuation characteristic value and the time difference of the original ranging depth, an asymmetric direction activation factor is constructed, including: The first criterion is that the decrease in the time difference of the depth fluctuation feature value exceeds a preset decrease threshold. The second criterion is that the time difference of the original ranging depth is greater than the negative of the preset depth descent tolerance threshold. The standard deviation of the time difference of the depth fluctuation feature value within the first local window is calculated as the local noise baseline; the absolute value of the time difference of the depth fluctuation feature value being greater than the local noise baseline is used as the third judgment condition. If all three conditions above are met, the output asymmetric direction activation factor is 1. If any of the decision conditions is not met, the output asymmetric direction activation factor is 0.

[0008] Optionally, an auxiliary gating factor can be constructed based on the change in the original signal, including: The fourth criterion is that the time difference of the original ranging depth is greater than the preset threshold for a significant increase in depth. Calculate the time difference of the orthogonal components of the original integrated charge; use the absolute value of the time difference of the orthogonal components of the original integrated charge being greater than a preset charge difference abrupt change threshold as the fifth judgment condition; If both of the above conditions are met, the output auxiliary gating factor is 1. If any judgment condition is not met, the output auxiliary gating factor is 0.

[0009] Optionally, calculating the temporal asymmetric directional covariance feature based on the temporal variance of the fusion gating factor and the depth fluctuation feature value includes: Calculate the time mean of depth fluctuation feature values ​​within a preset number of frames; Calculate the square of the difference between the depth fluctuation feature value and the time mean within a preset number of frames, sum the squares of the differences and take the mean, and use the mean as the time variance; Multiplying the fusion gating factor by the temporal variance yields the temporal asymmetric directional covariance feature.

[0010] Optionally, the continuous error suppression operator is calculated as follows: Calculate the mean and standard deviation of the temporal asymmetric directional covariance features of each pixel within the second local window; A dynamic extreme value penalty upper limit is constructed based on the mean and the standard deviation; Calculate the ratio of the temporal asymmetric directional covariance feature to the upper limit of the dynamic extremum penalty; The ratio is normalized to obtain the continuous error suppression operator.

[0011] Optionally, the calculation method for the edge-aware weights includes: Calculate the reverse reconstruction weights based on the continuous error suppression operator; Calculate the depth gradient magnitude based on the original ranging depth; Substituting the depth gradient magnitude into the exponential decay function yields the edge preservation factor; The product of the reverse reconstruction weight and the edge preservation factor is used as the edge-aware weight.

[0012] Secondly, this application provides a TOF ranging error dynamic compensation system, including: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, a TOF ranging error dynamic compensation method according to the above is implemented.

[0013] This application has the following technical effects: 1. This application uses the infrared brightness information of the image as a standard to measure the reliability of the distance measurement. It determines the energy weight based on the original infrared amplitude of each pixel in the local window, suppresses the weight contribution of low reflectivity pixels in the spatial heterogeneity assessment, and filters out static false edges caused by material shot noise. It introduces directional features to calculate the temporal asymmetric directional covariance feature, identifies the false trailing regions generated after the object leaves, and avoids the real motion front being misjudged as the trailing region. For the identified trailing regions, it calculates the continuous error suppression operator based on the temporal asymmetric directional covariance feature, and determines the edge perception weight by combining the depth gradient magnitude. This enables untrusted pixels to fuse the original ranging depth of stable pixels in the neighborhood, suppressing the impact of dynamic target motion trailing on the accuracy of TOF ranging. Attached Figure Description

[0014] Figure 1 This is a flowchart of steps S1-S4 in a TOF ranging error dynamic compensation method according to an embodiment of this application. Detailed Implementation

[0015] This application discloses a dynamic compensation method for TOF ranging error, referring to... Figure 1 This includes steps S1-S4.

[0016] S1: Acquire the original integrated charge of the sensor's multi-phase components and calculate the original infrared amplitude and original ranging depth.

[0017] In one embodiment, the vertical-cavity surface-emitting laser of the TOF sensor employs... A continuous infrared light wave, modulated at a frequency of MHz, is emitted, illuminating the surface of the target object and reflecting back to the sensor. Within an extremely short global exposure time, the sensor's pixel array integrates and collects photon-excited charges at four different modulation phase delay windows: 0°, 90°, 180°, and 270°. These four phase-level charge integration data are directly packaged into complete data frames without any spatial smoothing filtering, outputting the raw integrated charge values. .

[0018] Furthermore, differential subtraction operations are performed using the four-phase integral data at the underlying quadrature demodulation hardware level (e.g., This eliminates common-mode light interference from the environment. The true AC amplitude of the signal, i.e., the original infrared amplitude, is calculated using the in-phase and quadrature components and the Euclidean norm.

[0019] Specifically, the formula for calculating the original infrared amplitude can be: In the formula, These are the horizontal and vertical coordinates of the pixel in the image matrix, respectively; These represent the original integrated charge collected by the TOF sensor under different modulation phase delays. The original infrared amplitude characterizes the intensity of infrared light reflectivity on the target surface and is the core basis for evaluating the signal-to-noise ratio; the larger the amplitude, the more reliable the signal and the higher the accuracy of depth measurement.

[0020] Furthermore, based on the above differential calculation results, the phase delay of the modulated optical signal can be obtained through the arctangent function, and then converted into the initial physical distance, i.e., the original ranging depth. This depth value is the reference data source for all subsequent time-series analysis and spatial compensation, but it contains shot noise and flying-spot errors, which need to be compensated for in subsequent steps.

[0021] Specifically, the formula for calculating the original ranging depth can be: In the formula, This is the original ranging depth, in units of... It represents the initial physical distance from the target to the sensor, but it includes shot noise and flypoint error; Let be the speed of light constant, which is taken in this embodiment. The specific value can be adjusted according to the ranging accuracy required for the application scenario. Make equivalent corrections; The modulation frequency of the active lighting source is taken in this embodiment. MHz.

[0022] In summary, this step synchronously acquired the original infrared amplitude through orthogonal demodulation. Compared with the original ranging depth The former serves as the basis for signal-to-noise ratio assessment and energy weighting, while the latter serves as the depth benchmark to be compensated. Together, they constitute an indispensable input data source for the subsequent S2 to S4 steps, ensuring that the entire compensation process is based on real and complete physical signals.

[0023] S2: Construct the first local window centered on the current pixel, calculate the energy weight of each pixel based on the original infrared amplitude, and use the energy weight to calculate the weighted variance to obtain the depth fluctuation feature value.

[0024] This step aims to address the misjudgment of static false edges caused by shot noise in low-reflectivity regions. In the underlying physical mechanism of Time-of-Flight (TOF) ranging, the variance of shot noise in depth data is inversely proportional to the square of the infrared amplitude data. In low-reflectivity regions (such as black materials), due to the extremely small number of received photons, the original ranging depth undergoes drastic random jumps. Existing algorithms often misjudge these jumps caused by material shot noise as true physical geometric boundaries (i.e., flying points). Therefore, this step introduces amplitude energy normalization weights to suppress such false edges, improving the purity of feature values ​​from the source.

[0025] Specifically, with the current pixel Build a first local window of size 5×5 centered on the window. The energy weight of each pixel is calculated based on the original infrared amplitude. The formula for calculating the energy weight is as follows: In the formula, In pixels The first local window centered on Any pixel within The original infrared amplitude; It is the sum of squares of the original infrared amplitudes of all pixels within the first local window; It is a minimal constant to prevent the denominator from being zero. In this embodiment, it is taken as... ; The normalized energy weights are positively correlated with the pixel signal-to-noise ratio. Larger pixels (high reflectivity pixels) dominate the weighted average; For pixels with low reflectivity (e.g., those in low-reflectivity pixels), their contribution is adaptively suppressed. This self-normalization mechanism ensures that the eigenvalues ​​reflect only changes in the actual geometry, rather than differences in material reflectivity.

[0026] Furthermore, the local weighted depth expectation is calculated using this energy weight. This represents the weighted average depth after amplitude energy normalization. It eliminates random jump interference caused by shot noise in low reflectivity regions, providing a stable reference benchmark for subsequent determination of the true geometric boundary.

[0027] Specifically, the formula for calculating the locally weighted depth expectation can be: In the formula, In pixels The first local window centered on Any pixel within The original ranging depth; For normalized energy weights; The expected depth of the local weighted depth is the weighted average depth after amplitude energy normalization.

[0028] Finally, based on the aforementioned energy weights and weighted depth expectation, the depth fluctuation characteristic value is calculated. This feature value is the basic input for subsequent timing gating decisions, and its unit is m², reflecting the first local window. The degree of fluctuation in the depth of an intrapixel. When When the value is small, it indicates that the local area has a uniform depth, or that low-reflectivity noise has been suppressed; when When the value is large, it indicates that there is a real depth jump in the local area (such as the boundary of an object), and that the jump is supported by high reflectivity pixels, rather than simply noise jitter.

[0029] Specifically, the formula for calculating the depth fluctuation characteristic value can be: In the formula, The depth fluctuation characteristic value (weighted variance) is expressed in m². For the current pixel The first local window, centered on the element, has a size of 5×5. For normalized energy weights; In pixels The first local window centered on Any pixel within The original ranging depth; For locally weighted depth expectation; For the first local window Any pixel within The depth deviation from the local mean is used to measure the degree of deviation.

[0030] In summary, this step effectively filters out shot noise interference in low reflectivity regions through amplitude energy normalization weighting, resulting in depth fluctuation characteristic values. It can accurately distinguish between real geometric boundaries and material pseudo edges, providing a high signal-to-noise ratio feature input for the accurate triggering of asymmetric direction activation factors in step S3, and laying the foundation for subsequent capture of dynamic motion blur.

[0031] S3: Calculate the time difference of depth fluctuation characteristic value and original ranging depth, construct asymmetric directional activation factor and auxiliary gating factor and perform time series verification to obtain the fusion gating factor and time series asymmetric directional covariance feature.

[0032] This step aims to accurately identify the false trailing shadow region created after an object leaves, while protecting the real motion front that is approaching. When an object moves rapidly, traditional symmetric compensation algorithms output equally large feature values ​​for both the motion front (approaching the real physical boundary) and the motion trailing edge (the false trailing shadow departing), easily leading to the collapse of the real contour. If this symmetric feature is used directly for compensation, the real approaching physical contour will be forcibly smoothed, resulting in severe "volume shrinkage" or "front collapse" of the 3D target. Therefore, directional features can be introduced, activating statistical weights only when the feature value decays over time (the edge departs and produces a trailing shadow), and forcibly zeroing out when the feature value increases (the front approaches).

[0033] Specifically, the depth fluctuation feature value of each pixel and the time difference of the original ranging depth are first calculated to capture the trend of physical quantities changing over time. The formula for calculating the time difference is as follows: In the formula, This represents the difference between the depth fluctuation feature values ​​of the current frame and the previous frame; For the current frame The depth fluctuation characteristic value.

[0034] In the formula, This represents the difference between the original ranging depth of the current frame and the previous frame; For the current frame The original ranging depth.

[0035] After calculating the time difference, an asymmetric directional activation factor is constructed based on the time difference. This is used to capture the conventional motion blur process. Specifically, the formula for calculating the asymmetric directional activation factor can be: In the formula, This represents the difference between the depth fluctuation feature values ​​of the current frame and the previous frame; The threshold for the magnitude decrease, where , The feature value of depth fluctuation across the entire map The median, in m²; coefficient The sensitivity used to adjust the judgment is set to 0.05 in this embodiment, meaning that only when the local heterogeneity decreases more than the median level of the entire image will the judgment be made. A change is considered "significant" only when it is doubled. The value range can be [0.02, 0.08], and the specific value can be calibrated according to the sensor model and the specific working illuminance; Will As the first criterion, the depth fluctuation characteristic value must show a significant decrease (i.e., the decrease exceeds the threshold). ), used to eliminate small random fluctuations; This represents the difference between the original ranging depth of the current frame and the previous frame; This is the depth descent tolerance threshold, expressed in meters (m), used to exclude descent caused by changes in reflectivity rather than actual distance. decline. The value range can be [0.01, 0.05]. In this embodiment, we take... The specific value can be calibrated based on the ranging accuracy of the sensor and the speed of the target object. if (A significant decrease in depth) means that the object is rapidly approaching the sensor, at which point even... It is also the forefront of real motion and should not be misjudged as a ghosting effect.

[0036] Will As the second criterion, excluding "objects actually getting closer", the essence of motion blur is "residual old information", and its depth should tend to the background value (usually moving away from or remaining unchanged), rather than suddenly getting closer; For the first local window Inside The standard deviation of the local noise baseline, measured in m², reflects the local uncontrolled noise floor and is used to filter noise fluctuations. Will As the third criterion, this means that the changing signal exceeds the local noise by at least one standard deviation, thus avoiding false triggers in noise-dominated regions and ensuring... The changes are not caused by random fluctuations in local noise, but are statistically significant.

[0037] Based on the above three criteria, the following condition applies: local heterogeneity is significantly decreasing (mog dissipates); depth is not significantly approaching (excluding the true front); and the change exceeds the local noise floor (statistically reliable). .

[0038] However, relying solely on asymmetric directional activators There is still a risk of missed detections. In some complex scenarios, when an object moves away quickly, as pixels transition from the "object surface" to the "background area," if the texture complexity or depth heterogeneity of the background itself is higher than that of the object surface (e.g., a textured background wall is revealed after a smooth object moves away), or if multipath interference causes a drastic jump in the depth data of the trailing area, the local window depth fluctuation feature value may be affected. It may rise instead of fall (i.e.) This results in the gating conditions not being met, leading to missed detections.

[0039] To compensate for this deficiency, an auxiliary gating factor is introduced. It directly captures the essential characteristics of "object leaving" at the physical level: a significant increase in depth (the target moves away from the sensor) and a sudden change in the original charge signal (the source of reflected light received by the pixel changes). Through these two independent criteria, It can effectively identify special motion blur scenes where "the background becomes more complex after the object leaves" or "multipath interference leads to increased heterogeneity in the motion blur region," and... This complements each other, ensuring comprehensiveness in motion blur detection.

[0040] Specifically, the formula for calculating the auxiliary gating factor can be: In the formula, The threshold for significantly increasing depth is measured in meters (m) and can range from [0.05, 0.2]. In this embodiment, we take... The specific value can be calibrated based on the sensor's maximum effective range and the scene's depth dynamic range; When the object leaves, the center pixel of the local window changes from "object surface" to "background," and the depth fluctuation feature value increases significantly. If the window size just covers the object's edge, the background depth may be mixed into the window after the object leaves, leading to an increase in local heterogeneity. rise, ), In The conditions may miss this scenario, and According to the fourth judgment condition This method directly detects these missed detections by significantly increasing the detection depth (physical distance); This is the charge differential abrupt change threshold, measured in LSBs, and can range from 50 to 150. In this embodiment, it is set to... The specific value can be calibrated according to the bit depth of the sensor's analog-to-digital converter; Represents the orthogonal components (sine terms) of the current frame; Will As the fifth criterion, which includes both amplitude and phase information, it indicates that the original charge difference has abruptly changed (exceeding 80 LSB), meaning that the light signal received by the pixel has changed drastically (from object reflection to background reflection or multipath interference, not only the amplitude changes but also the phase jumps). This is an indicator function that takes the value 1 if the condition in parentheses is met, and 0 otherwise.

[0041] Based on the above two criteria, the object is deemed to have moved away from or away from the target if and only if: the depth increases significantly (the object moves away from or away from the target); or a sudden change in charge level occurs in the orthogonal components. . It captured a special scenario where depth abruptly changes after an object leaves, but heterogeneity may increase instead of decrease, and... They complement each other.

[0042] After obtaining the asymmetric directional activation factor and auxiliary gating factor, fusion and timing verification are performed. As a fusion gating factor. Only when the asymmetric direction activation factor... With auxiliary gating factor The fusion gating factor is set to 1 only when both are simultaneously identified as motion blur regions; if any factor is identified as non-motion blur (value 0), the fusion gating factor is set to 0. This operation effectively filters out false triggering signals that may be generated by a single gating, ensuring high confidence in motion blur recognition and laying the foundation for subsequent accurate compensation. To avoid false triggering in a single frame due to noise, the above fusion gating factor is required to be continuously... Intra-frame activation (i.e.) In continuous Only when all values ​​within the frame are 1 are the pixels considered to have truly entered the motion blur state. The value ranges from 2 to 5 frames. In this embodiment, 3 is used. The specific value can be calibrated based on the sensor's data output frame rate and the object's maximum expected speed.

[0043] After obtaining the fusion gating factor, the temporal asymmetric directional covariance feature is calculated based on the temporal variance of the fusion gating factor and the depth fluctuation feature value.

[0044] Specifically, the formula for calculating the temporal asymmetric directional covariance characteristics can be: In the formula, This is the fusion gate factor. A value of 1 indicates that the current pixel is a motion blur region, and a value of 0 indicates that the current pixel is not a motion blur region. The time average represents the most recent value. The mean of local fluctuation characteristics of the frames is used as the temporal baseline. For the current frame, The length of the sliding time window can be 3 to 5. In this embodiment, it is 3. The specific value can be determined according to the actual working conditions. Represents pixels In recent The temporal variance of a frame reflects the stability of the historical fluctuations of that pixel. As a temporal asymmetric directional covariance feature, the higher the value, the more likely the pixel is to be trapped in the false multipath trailing and flying point disaster area of ​​the motion trailing edge. It accurately locates the "unreliable area left after the object leaves" while protecting the "real contour that is approaching".

[0045] S4: Construct the second and third local windows respectively, and perform weighted fusion based on the continuous error suppression operator and edge-aware weights to obtain the compensated depth data.

[0046] After detecting a ghosting region, for unreliable pixels, depth values ​​need to be "borrowed" from surrounding reliable pixels for repair. To avoid denominator collapse caused by edge erosion and large-area ghosting, a dual-window collaborative strategy is adopted to solve the two major challenges of "statistical baseline stability" and "edge fidelity repair".

[0047] Specifically, firstly, a second local window with a size larger than the first local window is constructed. (In this embodiment, a size of 15×15 is used) to calculate the local statistical baseline. Using a large local window balances statistical robustness and local adaptability; even with large-area motion in the image, the statistics within the local window can still accurately identify local anomalies. The specific size range can be [11×11, 21×21], which can be calibrated according to the sensor resolution and the spatial scale of the dynamic target. A continuous error suppression operator is calculated based on the temporal asymmetric directional covariance characteristics within the second local window.

[0048] Specifically, the calculation formula for the continuous error suppression operator can be: In the formula, For the second local window Any pixel within The continuous error suppression operator represents the probability that the depth data of a certain pixel is "unreliable", and its value is between 0 and 1. For the second local window Any pixel within Temporal asymmetric directional covariance characteristics; These are the second local windows. The mean and standard deviation of the temporal asymmetric directional covariance features (size 15×15) are used to construct the upper limit of dynamic extremum penalty.

[0049] Specifically, with the current pixel Build a third local window at the center for deep reconstruction. (In this embodiment, 5×5 is used), and the specific size range can be [3×3, 7×7]. It can be determined by combining the typical diffusion width of the trail in space (determined by the object's movement speed and exposure time) and the size of small geometric structures. The edge perception weight is obtained based on the continuous error suppression operator and the original ranging depth of the pixels in the third local window.

[0050] Specifically, the formula for calculating edge-aware weights can be: In the formula, This is for reverse reconstruction of weights (credibility weights). The larger (and less trustworthy) the value, the smaller the weight. This is the edge preservation factor; if the gradient is large (meaning the pixel...), it indicates that the edge is not preserved. (Located at the edge of a real object), the exponent term approaches 0, and the weight... A sharp decline; To determine the depth gradient magnitude, the Sobel algorithm was used for gradient calculation. This is the edge scale parameter, in meters (m), and its value ranges from [0.02, 0.1]. In this embodiment, we take... The value can be selected according to the actual situation; The edge-aware weights characterize the third local window. Any pixel within The value range is [0, 1]. When it approaches 1, it indicates that the pixel is in a flat and uniform area without any depth jump; when it approaches 0, it indicates that the pixel is a ghost or is at a real edge.

[0051] After obtaining the edge-aware weights, apply the edge-aware weights to the third local window. The original ranging depths of each pixel are weighted and fused to obtain the compensated depth data.

[0052] Specifically, the formula for calculating the compensated depth data can be: In the formula, In this embodiment, the size of the spatial local neighborhood (i.e., the third local window) used to perform the final depth reconstruction is 5×5. For the third local window Any pixel within The original ranging depth; To prevent the use of a zero constant, this embodiment uses an empirical value of 10. -6 To prevent the denominator from being zero; It is the optimal output depth after compensation, for pixels deeply trapped in motion blur ( Approaching 1), stripping it of its own value, and forcibly absorbing surrounding safe neighbors through a formula ( A stable depth approaching 1); for pixels at true edges, even if there are reliable neighboring pixels, if they are on surfaces of different depths, the edge factor suppresses their weights to avoid pixel blurring; for fully reliable pixels ( (approaching 0 and with a flat surrounding area) Approaching 1, Approaching Preserve the original data.

[0053] The above calculations complete the high-fidelity dynamic edge repair, and the compensated depth data is used as the final output TOF ranging result.

[0054] This application also discloses a TOF ranging error dynamic compensation system, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, a TOF ranging error dynamic compensation method according to this application is implemented.

[0055] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0056] In this application, the aforementioned memory can be any tangible medium that contains or stores a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (DRAM), dynamic random access memory (DRAM), static random access memory (SRAM), etc., or any other medium that can be used to store desired information and can be accessed by an application program, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device.

[0057] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A dynamic compensation method for TOF ranging error, characterized in that, include: Acquire the raw integrated charge quantity of the sensor across multiple phases; The original infrared amplitude and original ranging depth are calculated based on the original integrated charge. A first local window is constructed with the current pixel as the center. The energy weight of each pixel is calculated based on the original infrared amplitude. The weighted variance of the original ranging depth within the first local window is calculated using the energy weight to obtain the depth fluctuation feature value. Calculate the depth fluctuation feature value and the time difference of the original ranging depth for each pixel. Based on the time difference of the depth fluctuation feature value and the time difference of the original ranging depth, construct an asymmetric direction activation factor. Furthermore, to address the issue of increased background depth fluctuation characteristics after an object leaves, an auxiliary gating factor is constructed based on the change in the original signal. A fused gating factor is obtained by fusing asymmetric directional activation factors with auxiliary gating factors. Calculate the temporal asymmetric directional covariance feature based on the temporal variance of the fusion gate factor and the depth fluctuation feature value; A second local window and a third local window, each larger than the first local window, are constructed with the current pixel as the center. A continuous error suppression operator is calculated based on the temporal asymmetric directional covariance feature within the second local window. The edge perception weight is obtained based on the continuous error suppression operator and the original ranging depth of the pixels in the third local window; the original ranging depth of each pixel in the third local window is weighted and fused using the edge perception weight to obtain the compensated depth data. The compensated depth data is used as the final output TOF ranging result.

2. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, The energy weight of each pixel is calculated based on the original infrared amplitude, including: Calculate the sum of squares of the original infrared amplitude of each pixel within the first local window; The ratio of the square of the original infrared amplitude of each pixel to the sum of the squares is used as the energy weight of each pixel.

3. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, Based on the time difference of the depth fluctuation characteristic value and the time difference of the original ranging depth, an asymmetric direction activation factor is constructed, including: The first criterion is that the decrease in the time difference of the depth fluctuation feature value exceeds a preset decrease threshold. The second criterion is that the time difference of the original ranging depth is greater than the negative of the preset depth descent tolerance threshold. The standard deviation of the time difference of the depth fluctuation feature value within the first local window is calculated as the local noise baseline; the absolute value of the time difference of the depth fluctuation feature value being greater than the local noise baseline is used as the third judgment condition. If all three conditions above are met, the output asymmetric direction activation factor is 1. If any of the decision conditions is not met, the output asymmetric direction activation factor is 0.

4. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, An auxiliary gating factor is constructed based on the change in the original signal, including: The fourth criterion is that the time difference of the original ranging depth is greater than the preset threshold for a significant increase in depth. Calculate the time difference of the orthogonal components of the original integrated charge; use the absolute value of the time difference of the orthogonal components of the original integrated charge being greater than a preset charge difference abrupt change threshold as the fifth judgment condition; If both of the above conditions are met, the output auxiliary gating factor is 1. If any judgment condition is not met, the output auxiliary gating factor is 0.

5. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, The temporal asymmetric directional covariance feature is calculated based on the temporal variance of the fusion gating factor and the depth fluctuation feature value, including: Calculate the time mean of depth fluctuation feature values ​​within a preset number of frames; Calculate the square of the difference between the depth fluctuation feature value and the time mean within a preset number of frames, sum the squares of the differences and take the mean, and use the mean as the time variance; Multiplying the fusion gating factor by the temporal variance yields the temporal asymmetric directional covariance feature.

6. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, The calculation method for the continuous error suppression operator is as follows: Calculate the mean and standard deviation of the temporal asymmetric directional covariance features of each pixel within the second local window; A dynamic extreme value penalty upper limit is constructed based on the mean and the standard deviation; Calculate the ratio of the temporal asymmetric directional covariance feature to the upper limit of the dynamic extremum penalty; The ratio is normalized to obtain the continuous error suppression operator.

7. The TOF ranging error dynamic compensation method according to claim 1, characterized in that, The calculation method for the edge-aware weights includes: Calculate the reverse reconstruction weights based on the continuous error suppression operator; Calculate the depth gradient magnitude based on the original ranging depth; Substituting the depth gradient magnitude into the exponential decay function yields the edge preservation factor; The product of the reverse reconstruction weight and the edge preservation factor is used as the edge-aware weight.

8. A dynamic compensation system for TOF ranging error, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement a dynamic compensation method for TOF ranging error according to any one of claims 1-7.

Images