ToF three-dimensional imaging simulation method based on light propagation modeling

Abstract

The application discloses a ToF three-dimensional imaging simulation method based on light propagation modeling, comprising the following steps: constructing light emission models and light receiving models divided according to light path propagation; decomposing a ToF imaging basic model into a plurality of imaging processing units with a sequence according to physical light paths and signal processing procedures; allocating associated error types to each unit and establishing a mapping; dividing errors into systematic errors and non-systematic errors, respectively establishing mathematical models and configuring the corresponding units to form a coupled error simulation model chain; in response to a simulation instruction, driving the model chain to execute in sequence, superimposing the error influence of each unit, and generating a simulation depth image containing the coupling effect of multi-source errors. Through the structured decomposition and sequential coupling simulation of the full link, the application realizes high-fidelity simulation of multi-source errors, improves the authenticity and engineering practicability of ToF imaging simulation, and can be used for ToF system design, error evaluation and algorithm verification.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging simulation technology, and in particular to a Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling. Background Technology

[0002] Time-of-flight (ToF) 3D imaging technology calculates the distance to a target object by measuring the time or phase difference required for an actively emitted light pulse to travel from emission to reflection back to the sensor. It boasts advantages such as relatively simple structure, long detection range, and high frame rate, and is widely used in fields such as space rendezvous and docking, industrial precision measurement, robot environmental perception and navigation, and autonomous driving. However, in practical applications, the final depth measurement results of a ToF imaging system are susceptible to interference from various error sources and noise from multiple components, including the light source, optical path, sensor circuitry, and signal processing algorithms. This leads to decreased measurement accuracy and various artifacts in the depth image, severely limiting its performance in complex, high-precision scenarios.

[0003] To better understand the working principle of Time-of-Flight (ToF) systems and assess the combined impact of various errors on the final measurement results, high-fidelity computer simulation of the entire physical process of ToF 3D imaging is of great significance. By simulating the imaging process, system designers can predict its performance before manufacturing the prototype, optimize key parameters, and provide a controllable verification platform for the subsequent development of error compensation algorithms.

[0004] Existing ToF imaging simulation methods have the following main shortcomings:

[0005] (1) The error modeling granularity is relatively coarse: Most existing methods treat various errors in the ToF imaging process as a whole and use a simple noise superposition model for processing. They fail to effectively distinguish the specific sources and physical links of different errors, making it difficult for the simulation results to truly reflect the actual impact of each error source on the imaging results.

[0006] (2) Insufficient simulation capability for multiple error coupling: In actual ToF imaging, multiple errors (such as light source non-uniformity, lens distortion, shot noise, quantization noise, etc.) often coexist and may affect each other. Existing simulation methods can usually only simulate a single type or a few types of errors, making it difficult to reproduce the complex imaging effect under the coupling of multiple error sources.

[0007] (3) Insufficient integration with physical processes: Existing simulation methods mostly adopt a simplified superposition mode of "ideal imaging model" plus "empirical noise model", which fails to closely integrate the mathematical modeling of errors with the physical process of the ToF system from light emission, light propagation, photoelectric conversion to signal solution, resulting in limited physical accuracy and reliability of the simulation.

[0008] International patent application WO2016171913A1 discloses a time-of-flight (TOF) simulation method for multipath optical phenomena. This method uses a graphics tool to record light path samples arriving at virtual camera pixels. Each sample contains intensity weights and path information (such as path length and path segment count) to simulate multipath effects. However, this technical solution focuses on simulating the propagation path of light in the geometry of a 3D scene (ray tracing) and does not address the complex signal processing links within the ToF sensor (such as modulation signal generation, photocharge accumulation, analog-to-digital conversion, etc.) and the various errors generated in these links.

[0009] Chinese invention patent CN106066987A discloses a method for online parameter calibration and compensation in Time-of-Flight (TOF) imaging, specifically involving modeling and compensating for wobbling errors. It employs a complex plane phasor-based method to model wobbling errors, but this approach only focuses on wobbling errors and lacks a holistic consideration and simulation of various other error sources throughout the entire ToF imaging system.

[0010] Chinese invention patent CN111260579A discloses a method for denoising and enhancing low-light images based on a physical noise generation model. This method breaks down the imaging process into three stages: photon to electron, electron to voltage, and voltage to digital signal, and models the noise (such as photon shot noise, readout noise, line noise, and quantization noise) at each stage. While this approach provides a staged link modeling concept, its application is limited to low-light RGB image sensors and does not address the key aspects unique to ToF 3D imaging, such as active light sources (e.g., VCSELs), modulation signal generation, and time-of-flight distance calculation, as well as their corresponding unique errors (e.g., oscillation error, temperature drift error).

[0011] Therefore, there is an urgent need for a method that can perform refined full-link modeling of the entire ToF 3D imaging process, systematically decompose and map multi-source errors according to their generation stages, and support multi-error coupling simulation according to real physical and signal processing flows, so as to improve the realism, accuracy and engineering practicality of the simulation. Summary of the Invention

[0012] To address the problems existing in the prior art, the present invention aims to provide a Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling. By performing structured modeling of the entire imaging process and refined decomposition and independent modeling of error components, the method achieves coupled simulation of multi-source errors, providing strong support for the design, error assessment, and compensation of ToF systems.

[0013] To achieve the above-mentioned objectives, this invention provides a ToF three-dimensional imaging simulation method based on light propagation modeling, comprising the following steps:

[0014] Step S1: Establish a basic model for ToF imaging based on the light propagation model, and divide the ToF imaging process into a light emission process model and a light reception process model according to the light propagation path;

[0015] Step S2: The ToF imaging basic model is arranged into a simulation model chain according to the physical optical path and signal processing flow, consisting of multiple imaging processing units with a sequential order;

[0016] Step S3: Assign an associated error type to each imaging processing unit in the simulation model chain, and establish a mapping relationship between the error and the corresponding imaging processing unit;

[0017] Step S4: Construct an error mathematical model for each error type associated with the imaging processing unit, and configure the error mathematical model in the corresponding imaging processing unit to form a coupled error simulation model chain;

[0018] Step S5: In response to the simulation command for the target scene, drive the imaging processing units in the coupling error simulation model chain to execute sequentially in the order specified, so as to superimpose the error effects of each unit during the simulation process and generate a simulated depth image containing multi-source error coupling effects.

[0019] According to one technical solution of the present invention, the plurality of imaging processing units decomposed in step S2 includes:

[0020] The system includes a modulation signal generation unit, a VCSEL transmission unit, a transmission optical path propagation unit, an object surface reflection unit, a receiving optical path propagation unit, a lens imaging unit, a ToF chip receiving unit, a photocharge accumulation unit, an analog-to-digital conversion unit, and a distance calculation unit.

[0021] According to one technical solution of the present invention, in step S3, assigning an associated error type to each imaging processing unit specifically includes:

[0022] The swing error is allocated to the modulation signal generation unit;

[0023] The non-uniformity error of the light source is allocated to the VCSEL emitting unit;

[0024] The ambient light noise is distributed to the transmitting optical path propagation unit and the receiving optical path propagation unit;

[0025] The lens distortion error is allocated to the lens imaging unit;

[0026] The sensor non-uniformity error and sensor mode noise are distributed to the ToF chip receiving unit;

[0027] Shot noise and temperature drift noise are distributed to the photocharge accumulation unit;

[0028] The quantization error is allocated to the analog-to-digital conversion unit;

[0029] The temperature drift error is allocated to the distance calculation unit.

[0030] According to a technical solution of the present invention, step S4, which involves constructing an error mathematical model for the error type, further includes:

[0031] The error types are divided into systematic errors and non-systematic errors, and different modeling strategies are adopted for each; among them,

[0032] The modeling strategies for system errors include: modeling the wobbling error based on the high harmonic expansion of the transmitted signal; modeling the light source non-uniformity error based on the light source intensity distribution curve; modeling the lens distortion error based on radial and tangential distortion models; and modeling the temperature drift error based on the temperature-time-error empirical formula.

[0033] Modeling strategies for non-systematic errors include: modeling ambient light noise based on blackbody radiation and sensor response; modeling shot noise based on Poisson distribution; modeling sensor mode noise based on proportional models of light response non-uniformity, dark signal non-uniformity, and dark current non-uniformity; and modeling temperature drift noise based on hot electron generation models.

[0034] According to one technical solution of the present invention, the mathematical modeling of sensor non-uniformity error allocated to the ToF chip receiving unit further includes:

[0035] An ADC sampling deviation model, a row address sampling deviation model, and a fixed pattern error model are established to simulate the effects of vertical stripes, row direction gradients, and fixed noise on depth images, respectively.

[0036] According to one technical solution of the present invention, the light emission model constructed in step S1 further includes:

[0037] Treating the VCSEL laser as a point source, the irradiated area of ​​the target region is calculated based on its half-divergence angle and the distance between the system and the target region.

[0038] Calculate the total radiation intensity and total peak power based on the operating parameters and number of lasers;

[0039] Calculate the irradiance of the target area based on the total peak power, irradiated area, and energy loss coefficient;

[0040] The optical receiving model constructed in step S1 further includes:

[0041] Based on the ToF chip parameters and receiving lens parameters, assuming the target surface is a Lambertian surface, calculate the radiation intensity, optical power received by each pixel, and the total received power of the chip.

[0042] Based on the chip's total received power and integration time, expressions for the transmitted signal and the echo signal are established;

[0043] The distance value is calculated by cross-correlation of differential sampled values ​​from different phases.

[0044] According to one technical solution of the present invention, the modeling of the swing error based on the higher harmonic expansion of the transmitted signal specifically includes: defining a transmitted signal expression containing fundamental and higher harmonic components; and correspondingly defining an echo signal expression containing fundamental and higher harmonic components.

[0045] The lens distortion error is modeled based on radial and tangential distortion models, specifically including: using a Taylor series expansion around the principal point to describe radial distortion, the degree of which is related to the distance from the image center point; and using a tangential distortion formula to describe the distortion of the image in the horizontal and vertical directions.

[0046] The modeling of temperature drift error based on the empirical formula of temperature-time-error specifically includes: obtaining the working environment temperature measured by the system's built-in sensor; and calculating the temperature drift error at a specific moment based on the working environment temperature, the preset ambient temperature coefficient, the sensor temperature coefficient, and the light source temperature coefficient using the empirical formula.

[0047] According to one technical solution of the present invention, the modeling of ambient light noise based on blackbody radiation and sensor response specifically includes:

[0048] Calculate the radiation intensity of sunlight at a specific wavelength and receiving bandwidth based on the blackbody radiation law;

[0049] By combining the light intensity response curves of each pixel of the sensor, the number of photons generated by sunlight noise in each pixel is calculated.

[0050] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory; when the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the ToF three-dimensional imaging simulation method based on light propagation modeling as described in any of the above technical solutions.

[0051] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the ToF three-dimensional imaging simulation method based on light propagation modeling as described in any of the above technical solutions.

[0052] Compared with existing technologies, the Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling provided by this invention has the following significant technical advantages:

[0053] (1) Refined modeling: The entire ToF imaging process is decomposed into multiple continuous units according to the physical optical path and signal processing flow, making the modeling closer to the actual physical process and the simulation accuracy higher;

[0054] (2) Clear sources of error: Establish the mapping relationship between error and imaging process unit, clarify the generation link of each error, and provide a basis for error tracing and targeted compensation;

[0055] (3) Strong multi-error coupling simulation capability: Errors are superimposed according to the real sequence of the imaging process, which can realistically simulate the complex imaging effect under the simultaneous action of multiple errors;

[0056] (4) Classification and processing of systematic and non-systematic errors: Modeling is performed separately for the different characteristics of the two types of errors, which retains the predictability of systematic errors and reflects the randomness of non-systematic errors;

[0057] (5) High practicality: It can be used for engineering applications such as performance evaluation, key parameter optimization, and error compensation algorithm verification in the design phase of ToF system, which significantly reduces the iteration cost of physical prototype. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0059] Figure 1 This diagram illustrates a basic model for ToF imaging based on a light propagation model according to one embodiment of the present invention.

[0060] Figure 2 This diagram illustrates the error modeling and sequential superposition based on a ToF imaging process model according to an embodiment of the present invention.

[0061] Figure 3 A schematic diagram illustrating the simulation results of the oscillation error according to one embodiment of the present invention;

[0062] Figure 4 This diagram illustrates the simulation results of temperature drift-related errors according to one embodiment of the present invention.

[0063] Figure 5 The diagram illustrates a flowchart of a ToF three-dimensional imaging simulation method based on light propagation modeling according to an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of this application can be combined with each other. The following embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the patent application. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application.

[0066] Example 1

[0067] like Figure 1 and Figure 5 As shown, the present invention provides a Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling, which can be used for the design, error assessment, performance optimization, and algorithm verification of ToF imaging systems. This method can be executed by a computer system. This embodiment corresponds to the process of constructing the coupled error simulation model chain of the present invention.

[0068] Step S1: Construct the basic model for ToF imaging.

[0069] A basic model for Time-of-Flight (ToF) imaging is established based on the light propagation model. The ToF imaging process is divided into a light emission process model and a light reception process model according to the light propagation path.

[0070] 1) Modeling of the light emission process:

[0071] In the simulation, the VCSEL laser is treated as a point source with a half-divergence angle of . Let the distance between the ToF 3D imaging system and the target region be... The area of ​​the laser irradiation zone in the target region is then... Represented as:

[0072] (1)

[0073] For each laser, its nominal radiation intensity is The current intensity under the nominal radiation intensity is In actual operation, the current intensity is When using When multiple lasers are used for illumination, the total radiation intensity is:

[0074] (2)

[0075] The total peak power of the laser in actual operation is:

[0076] (3)

[0077] In the formula, This refers to the voltage used in actual laser operation. This is the power conversion factor.

[0078] irradiance of the target area It can be calculated using the following formula:

[0079] (4)

[0080] In the formula, This is the energy loss coefficient during the light emission process.

[0081] 2) Modeling of the optical receiving process

[0082] The laser emits infrared radiation to the surface of the object and reflects it back, forming an image on the ToF chip via a receiving lens. If the length and width of the ToF chip are... and The diagonal length of the irradiated area can be calculated using trigonometric functions. With length and width , If the ToF sensor has M pixels in the horizontal direction and N pixels in the vertical direction, then the area of ​​the illumination region corresponding to each pixel is...

[0083] (5)

[0084] Power of each pixel corresponding to the illuminated area for

[0085] (6)

[0086] Let the diameter of the receiving lens be... The field of view corresponding to each pixel for:

[0087] (7)

[0088] Assuming the test target surface is a Lambertian surface, meaning that light is diffusely reflected when reflected from its surface, its reflectivity is... Then the radiation intensity received by each pixel on the ToF sensor The calculation is as follows:

[0089] (8)

[0090] In the formula, The area of ​​one pixel in a ToF sensor. This represents the energy loss coefficient during the light reception process.

[0091] The optical power received by this pixel for:

[0092] (9)

[0093] Total power received by the ToF chip for:

[0094] (10)

[0095] After receiving the optical signal, the sensor performs data acquisition during the integration time T. The maximum amplitude of the transmitted signal is equal to the actual transmitted power of the laser. The transmitted signal can then be expressed as The echo signal received by the ToF chip can be represented as For each pixel, the received echo signal is represented as .

[0096] The differential sampling signal can be obtained by cross-correlation calculation. (i=0, 1, 2, 3)

[0097] (11)

[0098] In the formula, These are the differential sampled values ​​under different phases, i.e. The distance value can be obtained using the four-phase sampling formula.

[0099] (12)

[0100] The above flowchart is shown below. Figure 1As shown in the figure. For ease of representation, the optical paths of the light emission and reception processes are symmetrically represented in the figure. The left half of the figure represents the emission optical path, and the right half represents the reception optical path. The emission optical path mainly involves the process of the VCSEL laser emitting a surface array laser, the laser propagating along the optical path to illuminate the surface of the object, and the reception optical path mainly involves the process of the laser being reflected from the object surface, propagating along the optical path to the receiving lens and being received by the ToF sensor, the photoelectric signal conversion, and finally calculating the distance value.

[0101] Step S2: Decompose the ToF imaging basic model into multiple imaging processing units with a sequential order.

[0102] Based on the physical optical path and signal processing flow, the entire ToF imaging process model is decomposed into the following sequential imaging processing units: modulation signal generation unit, VCSEL transmission unit, transmitted light path propagation unit, object surface reflection unit, received light path propagation unit, lens imaging unit, ToF chip receiving unit, photocharge accumulation unit, analog-to-digital conversion unit, and DCS and distance value calculation unit. These ten units together constitute a sequentially executed simulation model chain.

[0103] Step S3: Assign an associated error type to each imaging processing unit in the simulation model chain.

[0104] For each imaging processing unit, the error forms present in actual operation are identified, and a mapping relationship between errors and imaging processing units is established, including: oscillation error corresponding to the modulation signal generation unit; light source non-uniformity error corresponding to the VCSEL emission unit; ambient light noise corresponding to the emission and reception optical path propagation units; lens distortion error corresponding to the lens imaging unit; sensor non-uniformity error and sensor mode noise corresponding to the ToF chip receiving unit; shot noise and temperature drift noise corresponding to the photocharge accumulation unit; quantization error corresponding to the analog-to-digital conversion unit; and temperature drift error corresponding to the DCS and distance value calculation units.

[0105] Step S4: Construct an error mathematical model for the error type associated with each unit.

[0106] The aforementioned error forms are classified into systematic errors and non-systematic errors according to their nature, and mathematical models are established for each, which are then configured in the corresponding imaging processing units to form a chain of coupled error simulation models.

[0107] Modeling methods for systematic errors include:

[0108] To address the swing error of the modulation signal generation unit, a model is constructed based on the high-order harmonic expansion of the transmitted signal.

[0109] To address the non-uniformity error of the light source in the VCSEL emitting unit, a model is built based on the light source intensity distribution curve.

[0110] To address the lens distortion error of the lens imaging unit, a model is constructed based on radial and tangential distortion models.

[0111] To address the temperature drift error between the DCS and the distance value calculation unit, a model is built based on the empirical formula of temperature-time-error.

[0112] Modeling methods for non-systematic errors include:

[0113] The ambient light noise of the transmitting and receiving optical path propagation units is modeled based on blackbody radiation and sensor response.

[0114] For shot noise in photocharge accumulation units, a model based on Poisson distribution is used;

[0115] To address the sensor mode noise of the ToF chip receiver unit, a proportional model based on the non-uniformity of light response, non-uniformity of dark signal, and non-uniformity of dark current is used for modeling.

[0116] To address the temperature drift noise of the photocharge accumulation unit, a hot electron generation model is used for modeling.

[0117] Specifically, for the sensor non-uniformity error corresponding to the ToF chip receiving unit, the modeling method includes: establishing an ADC sampling deviation model, a row address sampling deviation model, and a fixed pattern error model, and simulating their effects on vertical stripes, row direction gradients, and fixed noise points generated in the depth image, respectively.

[0118] In this embodiment, by constructing a coupled error simulation model chain, the Time-of-Flight (ToF) system is transformed into a computer simulation model consisting of multiple functional units connected in an orderly manner, each unit encapsulating a specific error mathematical model. This model chain not only accurately describes the physical process of ToF imaging under ideal conditions but also systematically and structurally incorporates the multi-source, heterogeneous errors commonly found in real-world systems into the simulation framework. Subsequently, when specific target scene parameters are input, this model chain can be driven to run, thereby obtaining high-fidelity simulation results.

[0119] Example 2

[0120] In this embodiment, the mathematical model of the error is further described in detail based on the constructed coupled error simulation model chain.

[0121] 1) System error modeling

[0122] Oscillation error: In practical circuits, the transmitted signal typically contains higher harmonics, with a waveform between a sine wave and a square wave. Let the transmitted signal be:

[0123] (13)

[0124] in, The order of harmonics, The fundamental amplitude of the transmitted signal. Let be the amplitude of the nth harmonic of the transmitted signal. Correspondingly, the received signal is rewritten as:

[0125] (14)

[0126] in, The fundamental amplitude of the received signal. To receive the amplitude of the nth harmonic of the signal, This is the DC bias component.

[0127] Lens distortion error: This error includes radial distortion and tangential distortion.

[0128] For radial distortion, the degree of distortion increases with the distance from the image center point, and is described by the Taylor series expansion around the principal point:

[0129] (15)

[0130] in, and These represent the original position and the distorted position of the same point, respectively. and The radial distortion coefficient is... This represents the radial distance from the pixel to the principal point of the image.

[0131] Tangential distortion refers to the bending or twisting of an image along a straight line in both the horizontal and vertical directions, which can be expressed by the following formula:

[0132] (16)

[0133] in, and denoted as the tangential distortion coefficient.

[0134] Light source non-uniformity error: The non-uniformity of the light source is determined by its intensity distribution curve. Assume the light source distribution curve is... Then at the angle of illumination is At that time, its radiation intensity is:

[0135] (17)

[0136] Temperature drift error: Temperature drift error at time t Calculate using the following formula:

[0137] (18)

[0138] in, The ambient temperature is obtained from the temperature sensor built into the ToF 3D imaging system. The operating environment temperature coefficient; and These are the sensor temperature coefficient and the VCSEL temperature coefficient, respectively, which are determined by the system's own characteristics and can be measured experimentally.

[0139] 2) Non-systematic errors

[0140] Shot noise: Shot noise originates from the discrete nature of photons and follows a Poisson distribution. The distance error caused by shot noise is calculated as follows:

[0141] (19)

[0142] In the formula, To receive the signal components in photons, The total number of photons received. This is the signal modulation frequency.

[0143] Ambient light noise: Among ambient light noise, solar noise is the main source of noise, and its radiation intensity is calculated using the following formula:

[0144] (20)

[0145] In the formula, For wavelength, and These represent the first and second radiation constants, respectively. The surface temperature of the sun, The diameter of the sun, The distance between the Earth and the Sun. The receiving bandwidths are 840nm and 860nm, respectively.

[0146] The light power received by each pixel on the ToF sensor is:

[0147] (twenty one)

[0148] The response curve of the same pixel to different light intensities is consistent, assuming that the first... The light intensity response curve of each pixel is The number of photons produced by sunlight in this pixel is:

[0149] (twenty two)

[0150] in, For the first Light intensity response curve of each pixel.

[0151] Temperature drift noise: the number of photons generated by this noise. It can be calculated using the following formula:

[0152] (twenty three)

[0153] In the formula, For temperature drift The resulting voltage change To output the amplification factor, This is for converting the capacitance value.

[0154] Sensor mode noise: Sensor mode noise follows a Poisson distribution and is proportional to shot noise.

[0155] (twenty four)

[0156] In the formula, This is the proportionality coefficient. This represents the equivalent photon number for sensor mode noise.

[0157] By configuring the aforementioned error models into the corresponding units of the simulation model chain, the entire simulation process gains a solid physical foundation. When the simulation commands drive the model chain to run, each unit can calculate an output signal with the error characteristics of that stage based on its input signal and the internally configured error model, laying the foundation for ultimately generating a simulated depth image that highly approximates the real situation.

[0158] In some embodiments of the present invention, the ToF three-dimensional imaging simulation method based on light propagation modeling further includes:

[0159] Step S5: In response to the simulation command for the target scene, drive the imaging processing units in the coupling error simulation model chain to execute sequentially in the order specified, so as to superimpose the error effects of each unit during the simulation process and generate a simulation depth image containing multi-source error coupling effects.

[0160] When the simulation system receives a simulation instruction for a specific target scene (such as setting the target distance, surface reflectivity, ambient light intensity, etc.), the system generates a virtual light pulse as the initial excitation. This excitation signal first enters the modulation signal generation unit of the simulation model chain. The modulation signal generation unit generates a transmission signal containing high-order harmonics according to formula (13) in specific embodiment 2. This signal contains the root cause of the swing error.

[0161] Subsequently, the signal flows sequentially through the “VCSEL transmitting unit”, “transmitting light path propagation unit”, “object surface reflection unit”, “receiving light path propagation unit”, and “lens imaging unit”. While transmitting the signal, each unit corrects and degrades the signal’s intensity, direction, spatial distribution, etc., according to its internally configured error mathematical model (such as formulas (17), (20), (21), (15), (16)).

[0162] The light signal, after being focused by the lens, enters the "ToF chip receiving unit". In this unit, sensor mode noise is added according to formula (24), and the response of each pixel is adjusted differently according to the sensor non-uniformity model. Then, the signal enters the "photocharge accumulation unit". In this unit, Poisson shot noise and temperature drift noise generated by hot electrons are superimposed according to formulas (19) and (23), respectively.

[0163] Subsequently, the noisy analog charge signal enters the "analog-to-digital conversion unit" and is quantized into a digital signal, during which quantization error is introduced according to the uniform distribution model. Finally, the digital signal enters the "distance calculation unit", which calculates the distance according to formulas (11) and (12) and superimposes the temperature drift error on the calculated distance value according to formula (18).

[0164] The error superposition diagram of the entire process is as follows: Figure 2 As shown. Because the simulation strictly follows the actual sequence of physical optical path and signal processing, it can realistically reproduce the complex coupling effects of multiple errors acting simultaneously and being interconnected. For example, the non-uniformity of the light source affects the magnitude of subsequent shot noise (the weaker the signal, the greater the relative impact of shot noise), while temperature changes simultaneously affect the VCSEL's transmit power and the sensor's dark current. These interactive effects can all be reflected in the sequential coupling simulation of this invention.

[0165] Step S6: Generate and output a simulated depth image containing multi-source error coupling effects.

[0166] After the above-described full-process simulation, the computer system finally generates and outputs a simulated depth image. Each pixel value in this simulated depth image represents the simulated measurement distance of the corresponding target point. Because ten major errors from the light source to the solution are systematically superimposed during the simulation process, the output simulated depth image can highly realistically simulate the image that a real ToF camera might acquire under the same conditions, including various typical artifacts and error features such as vertical stripes, gradual changes in line direction, fixed noise, edge distortion, and overall distance drift.

[0167] Figure 3 The simulation results of the oscillation error are presented. Among them, Figure 3The left figure is a comparison between the simulated distance and the actual distance, which includes oscillation error. It can be seen that the simulated distance fluctuates periodically around the actual distance, which is consistent with the theoretical characteristics of oscillation error. Figure 3 The right figure shows the extracted swing error variation, visually illustrating the amplitude and period of the error. This demonstrates that the modeling and simulation of swing error in this invention is effective.

[0168] Figure 4 Simulation results related to temperature drift errors are presented. Among them, Figure 4 The left figure shows the combined error simulation results, including temperature drift error (systematic error) and temperature drift noise (non-systematic error). Figure 4 The right figure shows the individual variations of temperature drift error and temperature drift noise. It can be seen that temperature drift error exhibits a deterministic shift over time and temperature, while temperature drift noise manifests as random fluctuations around this shift. The simulation of this invention can clearly distinguish and simulate these two different types of temperature-related errors, providing a refined tool for evaluating the system's performance at different operating temperatures.

[0169] The output simulated depth images can be used for subsequent system performance evaluation, error analysis, and algorithm verification. For example, they can be used as training data to train a deep neural network to complete depth image denoising or error compensation tasks; they can also be used as a benchmark to quantitatively evaluate the merits of different error compensation algorithms. This implementation achieves a complete closed loop from model construction to simulation execution, verifying the engineering practicality of the invention.

[0170] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory; when the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform a ToF three-dimensional imaging simulation method based on light propagation modeling as described in any of the above technical solutions.

[0171] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0172] The memory can be an internal storage unit of the terminal device, such as a hard drive or RAM. Alternatively, it can be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units. The memory is used to store the computer program and other programs and data required by the terminal device. It can also be used to temporarily store data that has been output or will be output.

[0173] According to one aspect of the present invention, a computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement a ToF three-dimensional imaging simulation method based on light propagation modeling as described in any of the above technical solutions.

[0174] For example, computer-readable storage media can be read-only memory (ROM), random access memory (RAM), read-only optical disc (CD-ROM), magnetic tape, floppy disk, and optical data storage devices. They can be implemented using computer-executable program code, thus allowing them to be stored in a storage device for execution by a computing device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, this invention is not limited to any particular hardware and software combination.

[0175] The Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling provided by this invention has the following beneficial effects:

[0176] Refined modeling: The entire ToF imaging process is decomposed into multiple continuous imaging processing units according to the physical optical path and signal processing flow, and a model is built for each unit, making the simulation closer to the actual physical and circuit processes, and significantly improving the physical accuracy of the simulation.

[0177] Clear error source tracing: A clear mapping relationship between errors and imaging processing units has been established, and the generation links and influence paths of each type of error (such as oscillation error, shot noise, quantization noise, etc.) have been clarified, providing a clear basis for tracing system errors and developing error compensation algorithms for specific links.

[0178] Strong multi-source error coupling simulation capability: According to the real physical order and signal flow order of the imaging process, the error models of each unit are superimposed and coupled for simulation in sequence. It can simulate the complex imaging effect and interaction effect under the simultaneous action of multiple errors with high fidelity, which solves the shortcomings of single error or simple superposition simulation in the existing technology.

[0179] Error classification and processing: Errors are classified into systematic errors and non-systematic errors according to their nature, and different modeling strategies are adopted for their deterministic and random characteristics. This not only preserves the predictable and compensable characteristics of systematic errors, but also accurately reflects the random distribution characteristics of non-systematic errors.

[0180] Highly practical: The method of this invention can be directly applied to engineering practices such as early performance evaluation in the design phase of ToF systems, optimization of key parameters (such as light source power, modulation frequency, integration time, etc.), offline training and online verification of error compensation algorithms, etc., which helps to significantly reduce the dependence on iterative testing of physical prototypes, shorten the R&D cycle and reduce costs.

[0181] The above description is merely one embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A Time-of-Flight (ToF) three-dimensional imaging simulation method based on light propagation modeling, characterized in that, Includes the following steps: Step S1: Establish a basic model for ToF imaging based on the light propagation model, and divide the ToF imaging process into a light emission process model and a light reception process model according to the light propagation path; Step S2: The ToF imaging basic model is arranged into a simulation model chain according to the physical optical path and signal processing flow, consisting of multiple imaging processing units with a sequential order; Step S3: Assign an associated error type to each imaging processing unit in the simulation model chain, and establish a mapping relationship between the error and the corresponding imaging processing unit; Step S4: Construct an error mathematical model for each error type associated with the imaging processing unit, and configure the error mathematical model in the corresponding imaging processing unit to form a coupled error simulation model chain; Step S5: In response to the simulation command for the target scene, drive the imaging processing units in the coupling error simulation model chain to execute sequentially in the order specified, so as to superimpose the error effects of each unit during the simulation process and generate a simulated depth image containing multi-source error coupling effects.

2. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 1, characterized in that, The multiple imaging processing units decomposed in step S2 include: The system includes a modulation signal generation unit, a VCSEL transmission unit, a transmission optical path propagation unit, an object surface reflection unit, a receiving optical path propagation unit, a lens imaging unit, a ToF chip receiving unit, a photocharge accumulation unit, an analog-to-digital conversion unit, and a distance calculation unit.

3. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 2, characterized in that, In step S3, each imaging processing unit is assigned an associated error type, specifically including: The swing error is allocated to the modulation signal generation unit; The non-uniformity error of the light source is allocated to the VCSEL emitting unit; The ambient light noise is distributed to the transmitting optical path propagation unit and the receiving optical path propagation unit; The lens distortion error is allocated to the lens imaging unit; The sensor non-uniformity error and sensor mode noise are distributed to the ToF chip receiving unit; Shot noise and temperature drift noise are distributed to the photocharge accumulation unit; The quantization error is allocated to the analog-to-digital conversion unit; The temperature drift error is allocated to the distance calculation unit.

4. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 3, characterized in that, Step S4, which involves constructing an error mathematical model for the error type, further includes: The error types are divided into systematic errors and non-systematic errors, and different modeling strategies are adopted for each; among them, The modeling strategies for system errors include: modeling the wobbling error based on the high harmonic expansion of the transmitted signal; modeling the light source non-uniformity error based on the light source intensity distribution curve; modeling the lens distortion error based on radial and tangential distortion models; and modeling the temperature drift error based on the temperature-time-error empirical formula. Modeling strategies for non-systematic errors include: modeling ambient light noise based on blackbody radiation and sensor response; modeling shot noise based on Poisson distribution; modeling sensor mode noise based on proportional models of light response non-uniformity, dark signal non-uniformity, and dark current non-uniformity; and modeling temperature drift noise based on hot electron generation models.

5. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 4, characterized in that, The mathematical model for constructing the sensor non-uniformity error allocated to the ToF chip receiving unit further includes: An ADC sampling deviation model, a row address sampling deviation model, and a fixed pattern error model are established to simulate the effects of vertical stripes, row direction gradients, and fixed noise on depth images, respectively.

6. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 1, characterized in that, The light emission model constructed in step S1 further includes: Treating the VCSEL laser as a point source, the irradiated area of ​​the target region is calculated based on its half-divergence angle and the distance between the system and the target region. Calculate the total radiation intensity and total peak power based on the operating parameters and number of lasers; Calculate the irradiance of the target area based on the total peak power, irradiated area, and energy loss coefficient; The optical receiving model constructed in step S1 further includes: Based on the ToF chip parameters and receiving lens parameters, assuming the target surface is a Lambertian surface, calculate the radiation intensity, optical power received by each pixel, and the total received power of the chip. Based on the chip's total received power and integration time, expressions for the transmitted signal and the echo signal are established; The distance value is calculated by cross-correlation of differential sampled values ​​from different phases.

7. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 4, characterized in that, The modeling of the swing error based on the higher harmonic expansion of the transmitted signal specifically includes: defining the expression for the transmitted signal containing the fundamental wave and higher harmonic components; and correspondingly defining the expression for the echo signal containing the fundamental wave and higher harmonic components. The lens distortion error is modeled based on radial and tangential distortion models, specifically including: using a Taylor series expansion around the principal point to describe radial distortion, the degree of which is related to the distance from the image center point; and using a tangential distortion formula to describe the distortion of the image in the horizontal and vertical directions. The modeling of temperature drift error based on the empirical formula of temperature-time-error specifically includes: obtaining the working environment temperature measured by the system's built-in sensor; and calculating the temperature drift error at a specific moment based on the working environment temperature, the preset ambient temperature coefficient, the sensor temperature coefficient, and the light source temperature coefficient using the empirical formula.

8. The ToF three-dimensional imaging simulation method based on light propagation modeling according to claim 4, characterized in that, The modeling of ambient light noise based on blackbody radiation and sensor response specifically includes: Calculate the radiation intensity of sunlight at a specific wavelength and receiving bandwidth based on the blackbody radiation law; By combining the light intensity response curves of each pixel of the sensor, the number of photons generated by sunlight noise in each pixel is calculated.

9. An electronic device, characterized in that, include: One or more processors, one or more memories, and one or more computer programs; wherein the processor is connected to the memory, and the one or more computer programs are stored in the memory, and when the electronic device is running, the processor executes the one or more computer programs stored in the memory to cause the electronic device to perform the ToF three-dimensional imaging simulation method based on light propagation modeling as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, implement the ToF three-dimensional imaging simulation method based on light propagation modeling as described in any one of claims 1 to 8.

Images