Adaptive single-photon accumulation imaging method based on cycle uncertainty estimation

Abstract

The present application belongs to the field of laser radar and computing imaging technology, and relates to an adaptive single-photon accumulation imaging method based on cycle uncertainty estimation, comprising: acquiring a current photon flight time histogram generated by internal accumulation of a detector; calculating a histogram increment feature vector of a current detection stage relative to a previous detection stage; performing nonlinear operation on a memory state vector of the previous detection stage and the histogram increment feature vector to update a current memory state vector; performing feature splicing, feature fusion and mapping on the current memory state vector and an index value of the current detection stage to obtain an uncertainty prediction value of current depth estimation; if the uncertainty prediction value is less than or equal to a preset confidence threshold, a collection termination signal is generated; if the uncertainty prediction value is greater than the preset confidence threshold, a continuous collection signal is generated. The method realizes adaptive photon accumulation truncation control, reduces power consumption and improves frame rate while ensuring imaging accuracy.

Description

Technical Field

[0001] This invention belongs to the field of lidar and computational imaging technology, specifically relating to an adaptive single-photon cumulative imaging method based on cyclic uncertainty estimation. Background Technology

[0002] LiDAR, as an active remote sensing technology, measures target distance by emitting laser pulses and receiving the echo signals reflected from targets, based on the Time of Flight (ToF) principle. Among them, lidar technology based on single-photon avalanche diodes (SPADs) utilizes the single-photon detection capability of SPADs in Geiger mode, combined with time-correlated single-photon counting technology, to achieve high-sensitivity detection of weak echo signals and picosecond-level time resolution. This allows for its widespread application in fields such as autonomous driving, 3D sensing in consumer electronics, and extremely low-light imaging.

[0003] Specifically, due to the quantum randomness of single-photon detection and interference from ambient background light (such as sunlight) and device dark counting, the echo of a single laser pulse is often difficult to use directly for accurate depth estimation. Therefore, existing SPAD imaging processing typically employs a multi-cycle cumulative histogram construction strategy: controlling the transmitter to repeatedly emit thousands of laser pulses towards the same point, constructing a statistical histogram along the time axis from the arrival times of the multiple detected photons. In the histogram, true signal photons form statistical peaks within the corresponding target flight time bins, while noise photons are randomly distributed, forming a noise floor. Using peak extraction algorithms, the target's depth information can be calculated from the statistical data.

[0004] However, existing SPAD imaging methods generally employ a fixed-period acquisition strategy, which forces a preset number of acquisition cycles (e.g., a fixed accumulation of 10,000 pulses) regardless of the reflectivity of the target, its distance, or the intensity of the ambient background light to ensure sufficient signal-to-noise ratio even under the worst conditions (Beer M, Haase JF, Ruskowski J, et al. Background light rejection in SPAD-based LiDAR sensors by adaptive photon coincidence detection[J]. Sensors, 2018, 18(12): 4338.). This strategy ignores the dynamic diversity of the imaging scene: in scenes with good signal-to-noise ratio or sufficient photon flux, only a very small number of accumulation cycles are often needed to form a clearly distinguishable signal peak in the histogram. If the remaining acquisition process is continued at this time, it will not significantly improve the depth accuracy, but will instead lead to high-frequency emission of the laser and frequent switching of the readout circuit, resulting in a sharp increase in the overall power consumption of the processing.

[0005] Furthermore, the fixed-period redundant acquisition leads to excessively long acquisition times, directly limiting the imaging frame rate and easily causing motion blur when imaging fast-moving objects. Simultaneously, the massive amount of redundant data increases the occupancy of on-chip memory (SRAM) and the pressure on data transmission bandwidth. Although some existing technologies attempt to terminate acquisition early using simple photon counting thresholds, they fail to fully consider the statistical fluctuation characteristics of background noise (i.e., Poisson-Skellam distribution). Therefore, simple photon counting cannot accurately reflect the reliability of depth estimation and is highly susceptible to misjudgment leading to imaging errors in high-background-noise environments.

[0006] Therefore, how to break through the constraints of traditional fixed-cycle strategies and establish an adaptive control method that can perceive data quality in real time and make intelligent decisions on when to stop acquisition based on inherent statistical uncertainty, so as to significantly reduce power consumption and increase frame rate while ensuring imaging accuracy, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] To address the aforementioned problems in the existing technology, this invention provides an adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation. The technical problem to be solved by this invention is achieved through the following technical solution:

[0008] This invention provides an adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation, comprising the following steps:

[0009] In the current detection phase, obtain the current photon flight time histogram accumulated inside the detector;

[0010] Perform temporal difference processing on the current photon flight time histogram to calculate the histogram increment feature vector of the current detection stage relative to the previous detection stage;

[0011] The trained recurrent feature extraction model is used to perform nonlinear operations on the memory state vector of the previous detection stage and the histogram incremental feature vector to update the current memory state vector; wherein, the memory state vector is configured to implicitly encode the signal-to-noise ratio and ambient background light intensity of the current imaging scene.

[0012] The current memory state vector is concatenated with the current detection stage index value, and the trained uncertainty prediction model is used to perform feature fusion and mapping on the concatenated vector to obtain the uncertainty prediction value of the current depth estimate.

[0013] If the uncertainty prediction value is less than or equal to the preset confidence threshold, a collection termination signal is generated to stop photon collection and output the current photon time-of-flight histogram; if the uncertainty prediction value is greater than the preset confidence threshold, a continue collection signal is generated to enter the next detection stage.

[0014] In one embodiment of the present invention, the histogram incremental feature vector is:

[0015] ;

[0016] in, This is the histogram incremental feature vector. This is the current photon flight time histogram. This is the photon flight time histogram from the previous detection phase.

[0017] In one embodiment of the present invention, the cyclic feature extraction model includes an encoding unit and a cyclic memory unit, wherein,

[0018] The encoding unit is used to perform linear projection and dimensionality reduction on the histogram incremental feature vector to obtain a compressed feature vector.

[0019] The recurrent memory unit is used to concatenate and fuse the compressed feature vector with the memory state vector of the previous detection stage, and to perform nonlinear operations on the concatenated features through a fully connected layer and an activation function to obtain the current memory state vector.

[0020] In one embodiment of the present invention, the formula for calculating the current memory state vector is:

[0021] ;

[0022] in, For the first The memory state vector of each detection phase For the first The memory state vector of each detection phase This is the histogram incremental feature vector. This is the weight matrix of the coding unit. This is the bias vector of the coding unit. This is the weight matrix of the circular memory unit. This is the bias vector for the circular memory unit; This represents a vector concatenation operation. It is a linear rectification activation function.

[0023] In one embodiment of the present invention, the uncertainty prediction model employs a multilayer perceptron network.

[0024] In one embodiment of the present invention, the formula for calculating the uncertainty prediction value is as follows:

[0025] ;

[0026] in, For uncertain predicted values, The range of values ​​is , These are the weights of the hidden layers in a multilayer perceptron. These are the bias parameters of the hidden layers in a multilayer perceptron. The weights of the output layer in a multilayer perceptron. These are the bias parameters of the output layer in a multilayer perceptron. It is the Sigmoid activation function. It is a linear rectified activation function. For the first The memory state vector of each detection phase This is the index value for the detection phase.

[0027] In one embodiment of the present invention, the training method for the cyclic feature extraction model and the uncertainty prediction model includes:

[0028] Obtain a simulation dataset containing various combinations of signal-to-noise ratio, background luminous flux, and pulse width;

[0029] For each scenario combination in the simulation dataset, a photon arrival time series is generated, and a photon time-of-flight histogram is constructed for each detection stage. A histogram increment sequence is generated using the photon time-of-flight histogram for each detection stage.

[0030] For each detection stage of each scenario combination, the true depth estimation error rate at each time point is calculated as an uncertainty label;

[0031] Construct a hybrid loss function that includes a mean squared error term and a mean absolute error term;

[0032] Using the histogram increment sequence as input and the true depth estimation error rate as the uncertainty label, the network parameters of the recurrent feature extraction model and the uncertainty prediction model are iteratively updated using the backpropagation algorithm until the hybrid loss function converges, thus obtaining the trained recurrent feature extraction model and the trained uncertainty prediction model.

[0033] In one embodiment of the present invention, the uncertainty label is:

[0034] ;

[0035] in, To estimate the error rate for true depth, This refers to the number of repetitions required for photon accumulation in the current scene combination and detection phase. This represents the number of times the depth estimation result deviates from the true distance by more than the allowable error.

[0036] In one embodiment of the present invention, the hybrid loss function is:

[0037] ;

[0038] in, For a mixed loss function, To balance the weighting coefficients, Mean square error, The mean absolute error, The uncertainty prediction value output by the uncertainty prediction model. Estimate the error rate for the true depth.

[0039] Another embodiment of the present invention provides an area array lidar imaging system, including multiple pixel units, and performs the imaging method as described in the above embodiments for each pixel unit, controlling the acquisition termination time of each pixel unit to achieve pixel-level adaptive exposure control.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] 1. In the imaging method of the present invention, by introducing the histogram incremental feature vector, the influence of the absolute order of the cumulative photon number is effectively eliminated, so that the cyclic feature extraction model focuses on learning the relative growth dynamics of signal and noise. Compared with the method of directly using the full histogram, there is no need to store the complete historical histogram sequence on the chip, which significantly reduces the occupation of hardware storage resources and improves the generalization ability of the model under different light intensity scenarios.

[0042] 2. In the imaging method of the present invention, the current memory state vector and the current detection stage index value are feature-concatenated. An uncertainty prediction model is constructed by explicitly injecting the detection stage index value. This can effectively distinguish between the two states of "long-term accumulation under low signal-to-noise ratio" and "short-term accumulation under high signal-to-noise ratio", thereby achieving accurate calibration of the uncertainty of depth estimation. In high signal-to-noise ratio scenarios, the acquisition cycle can be greatly reduced, and the overall power consumption of the system can be significantly reduced.

[0043] 3. In the imaging method of the present invention, an uncertainty prediction model is constructed based on the Poisson-Skaelas statistical theory. Compared with the simple photon counting threshold method in the prior art, it can more accurately reflect the reliability of depth estimation under complex lighting conditions, avoid imaging errors or data accuracy reduction caused by misjudgment under strong background noise interference, and improve the effective frame rate of the imaging system. Attached Figure Description

[0044] Figure 1 A schematic flowchart of an adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation provided in an embodiment of the present invention;

[0045] Figure 2 A schematic flowchart of another adaptive single-photon cumulative imaging method based on cyclic uncertainty estimation provided in an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram illustrating the calculation of the incremental eigenvectors of a histogram.

[0047] Figure 4 This is a schematic diagram of the data flow in the cyclic feature extraction model and the uncertainty prediction model of this invention;

[0048] Figure 5 This is a schematic diagram illustrating the impact of different signal-to-noise ratios on acquisition requirements under experimental conditions with constant background photon flux.

[0049] Figure 6 This diagram illustrates the impact of changes in photon flux on acquisition requirements under experimental conditions with a constant signal-to-noise ratio. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0051] Example 1

[0052] This embodiment provides an adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation. This method achieves real-time monitoring of the photon accumulation process by controlling the data closed loop of the transmitter, detector, and signal processing end. It is mainly applied in lidar or 3D imaging equipment that uses SPAD arrays as detectors, and aims to solve the problems of power consumption redundancy and frame rate limitation caused by fixed period acquisition strategy.

[0053] Please see Figure 1 and Figure 2 , Figure 1 This is a flowchart illustrating an adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation, provided in an embodiment of the present invention. Figure 2 This is a flowchart illustrating another adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation provided in an embodiment of the present invention. The method includes the following steps:

[0054] S101. During the current detection phase, obtain the current photon flight time histogram accumulated inside the detector.

[0055] Specifically, at the start of the imaging cycle, the state of the imaging system is first reset, including clearing the histogram storage space and resetting the memory state vector of the recurrent feature extraction model. For all-zero vectors, and reset the initial photon flight time histogram It is a vector of all zeros.

[0056] This embodiment discretizes the accumulation process into multiple detection stages. Each detection stage is a discrete acquisition time divided based on a preset laser pulse cycle interval. The time interval parameter of each detection stage is set. Implementation. Time interval parameter. The granularity of adaptive control is defined, for example, setting each 100 laser pulse cycles as a detection phase.

[0057] Subsequently, the laser emitting unit is controlled to emit a periodic sequence of high-frequency Gaussian laser pulses (typically picosecond to nanosecond pulses) toward the target scene, and the SPAD detector is controlled to be in Geiger mode to respond to echo photons. The time of flight of photons is measured using time-to-digital conversion (TDC) technology, and the detected events are accumulated in real time into the corresponding time bins of the histogram memory.

[0058] After the system finishes executing the first... After the cumulative cycle corresponding to each detection stage (i.e., the cumulative number of pulses reaches...), Parallel reading of the generated current photon flight time histogram .

[0059] Photon flight time histogram It consists of multiple time chambers, with the value in each chamber representing the total number of photons detected within that time interval. Specifically, it is a photon time-of-flight histogram. It is a dimension A one-dimensional vector (e.g.) ), of which The element value represents the time from the start of the current frame's acquisition to the current moment, in the [number]th [frame]. The total number of photons captured within each time chamber. It should be noted that during the reading... During the process, the photon acquisition and TDC counting processes of the physical layer are not interrupted to ensure time utilization.

[0060] S102. Perform time-series difference processing on the current photon flight time histogram and calculate the histogram increment feature vector of the current detection stage relative to the previous detection stage.

[0061] To eliminate the non-stationarity of the input data caused by the linear growth of the cumulative photon number over time, this step does not directly infer the original photon flight time histogram, but instead calculates its temporal dynamic characteristics.

[0062] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating the calculation of the histogram incremental eigenvector. Specifically, it obtains the first... Histogram of photon flight time in each detection phase and the Histogram of photon flight time cached in each detection phase And according to the formula Calculate the incremental eigenvector of the histogram .

[0063] Histogram incremental feature vector Characterized in the most recent detection time interval Within the signal-to-noise ratio (SNR) frame, the dynamic distribution of newly arriving photons along the time axis is observed. This feature can keenly reflect the local growth rate of signal photons relative to background noise photons. In high SNR scenarios, the increment of the signal time frame is significantly higher than that of the noise frame; while in low SNR scenarios, the difference in increments between the two is smaller. This difference is a key basis for subsequent steps to determine the SNR.

[0064] This embodiment effectively eliminates the influence of the absolute magnitude of the accumulated photon count on the model by introducing histogram incremental feature vectors as input to the recurrent feature extraction model, allowing the model to focus on learning the relative growth dynamics of signal and noise. Compared to methods that directly use the full histogram, it eliminates the need to store the complete historical histogram sequence on-chip, significantly reducing the occupation of hardware storage resources (SRAM) and improving the model's generalization ability under different light intensity scenarios.

[0065] S103. Using the trained recurrent feature extraction model, perform nonlinear operations on the memory state vector and histogram incremental feature vector from the previous detection stage to update the current memory state vector; wherein, the memory state vector is configured to implicitly encode the signal-to-noise ratio and ambient background light intensity of the current imaging scene.

[0066] This step utilizes a trained recurrent feature extraction model to process the histogram incremental feature vector. By using a time-series recursive approach, stable scene parameter features are extracted from unstable instantaneous increments.

[0067] Specifically, combining the memory state vector from the previous detection phase The trained recurrent feature extraction model is used to perform nonlinear operations to update the current memory state vector. :

[0068] ;

[0069] in, This indicates the use of a cyclic feature extraction model for nonlinear computation. For the first The memory state vector of each detection phase For the first The memory state vector of each detection phase.

[0070] Memory state vector It is a low-dimensional latent variable that implicitly encodes the signal-to-noise ratio (SBR) of the current imaging scene and the ambient background light intensity. This varies with the detection phase. The increase in memory state The confidence level of the accumulated evidence regarding the existence of the target signal gradually increases.

[0071] S104. The current memory state vector and the current detection stage index value are concatenated for feature extraction. The trained uncertainty prediction model is then used to perform feature fusion and mapping on the concatenated vector to obtain the uncertainty prediction value of the current depth estimate.

[0072] Specifically, obtain the index value of the current detection stage. Since each detection stage corresponds to a fixed number of pulse cycles, the detection stage index value... The detection phase index value is used to characterize the total number of laser pulse cycles that have accumulated so far. This explicitly characterizes the current physical accumulation time (or photon flux scale). The detection phase index value... As explicit prior physical information, and memory state vector Feature concatenation is performed, and then the concatenated vector is jointly encoded and mapped using a trained uncertainty prediction model. Finally, the normalized value output by the Sigmoid activation function is used as the uncertainty prediction value for the current depth estimate. Uncertainty forecast value It is a value in The values ​​in the interval are used to quantify the probability of an error in the depth estimate if data collection stops at the current moment.

[0073] This embodiment introduces physical prior information. This solves the ambiguity problem of histogram shape. Statistically, the peak value of a histogram formed by the accumulation of a weak signal over a long period may be highly similar to the peak value formed by the accumulation of a strong signal over a short period. Neural networks struggle to distinguish between these two states based solely on histogram features. The introduction of an index value during the detection phase addresses this issue. Then, the algorithm can identify whether the current data collection is in the early or late stage, thereby correctly assessing the true level of the current data signal-to-noise ratio.

[0074] In the imaging method of this invention, the current memory state vector and the current detection stage index value are concatenated. By explicitly injecting the detection stage index value, an uncertainty prediction model based on physical priors is constructed. This effectively distinguishes between two states: "long-term accumulation under low signal-to-noise ratio" and "short-term accumulation under high signal-to-noise ratio," thereby achieving accurate calibration of depth estimation uncertainty. Experimental results show that, after introducing the detection stage index value, the acquisition cycle can be significantly reduced in high signal-to-noise ratio scenarios, with an average reduction of approximately 67.3% in the number of laser emissions, significantly lowering the overall power consumption of the system.

[0075] S105. If the uncertainty prediction value is less than or equal to the preset confidence threshold, a collection termination signal is generated to stop photon collection and output the current photon flight time histogram; if the uncertainty prediction value is greater than the preset confidence threshold, a continue collection signal is generated to enter the next detection stage.

[0076] Specifically, the uncertain predicted value Compared with the preset confidence threshold (For example The comparison is performed, and the following control strategy is executed based on the comparison result:

[0077] 1) Termination strategy: If This indicates that the quality of the currently accumulated data has met the accuracy requirements for depth estimation (i.e., the expected error rate is below the threshold). At this point, a data acquisition termination signal is generated to control the laser emission unit to stop emitting and output the current photon flight time histogram. The depth calculation module (such as performing a peak search algorithm or a Gaussian fitting algorithm) is then used to calculate the final target distance.

[0078] Specifically, the execution steps of the depth calculation module include: in response to the acquisition termination signal, filtering the final output photon time-of-flight histogram; determining the peak position of the signal photon in the histogram using the peak search algorithm (Argmax) or Gaussian fitting algorithm; and calculating the distance information of the target object based on the speed of light and the peak position of the signal photon.

[0079] 2) Continue the strategy: If This indicates that the current signal-to-noise ratio is insufficient and the depth estimation is unreliable. At this point, a signal to continue acquisition is generated to maintain the acquisition state, and the index value of the detection phase is updated. Update the current photon flight time histogram to the previous stage cache. ), and return to execute step S101.

[0080] 3) Forced termination strategy: If the index value during the detection phase... Reach the preset maximum number of stages Regardless of the predicted value, the system will forcibly stop data collection and output the results to prevent the system from getting stuck in an infinite loop under extreme conditions (such as when the target is not visible).

[0081] This embodiment also provides an area array lidar imaging system, including multiple pixel units. The above-mentioned adaptive single-photon cumulative imaging method based on cyclic uncertainty estimation is executed for each pixel unit, and the acquisition termination time of each pixel unit is controlled to achieve pixel-level adaptive exposure control.

[0082] This embodiment first introduces the histogram incremental feature vector, then uses a trained cyclic feature extraction model to process the histogram incremental feature vector to update the current memory state vector. Finally, the current memory state vector and the current detection stage index value are concatenated for feature fusion and mapping to obtain the uncertainty prediction value of the current depth estimation. This achieves independent adaptive control for each pixel. For high reflectivity or close-range targets, only a few cycles are needed to terminate the acquisition, significantly reducing power consumption and increasing the instantaneous frame rate. For low reflectivity or distant targets, the acquisition time is automatically extended to ensure the imaging success rate, significantly reducing power consumption and increasing the frame rate while ensuring imaging accuracy.

[0083] The method of this invention establishes a probabilistic statistical model of the photon accumulation process and uses a lightweight neural network to extract scene parameters from the temporal evolution features of the histogram. Under the premise of ensuring the accuracy of depth estimation, it realizes adaptive photon accumulation truncation control of the acquisition process.

[0084] Example 2

[0085] Building upon Example 1, this example provides a detailed description of the cyclic feature extraction model and the uncertainty prediction model. The cyclic feature extraction model and uncertainty prediction model in this example are designed based on the Poisson-Skellam statistical theory of photon counting, and feature small parameter count, high computational efficiency, and suitability for on-chip integration.

[0086] The Poisson-Skellam statistical theory defines the depth estimation error rate. Cumulative number of cycles and scene intrinsic parameters The following approximation relationship is satisfied:

[0087] ;

[0088] in, It is the right-tail function of the standard normal distribution. These parameters depend on the signal photon flux, the background noise photon flux, and the laser pulse width. The cumulative number of cycles. The cyclic feature extraction model is configured to implicitly regress parameters from the histogram incremental feature vector. Index value during the detection phase Used to characterize the number of cumulative periods .

[0089] Please see Figure 4 , Figure 4 This is a schematic diagram of the data flow in the cyclic feature extraction model and the uncertainty prediction model of this invention.

[0090] In one specific embodiment, the recurrent feature extraction model includes an encoder and a recurrent memory cell.

[0091] The encoder unit is used to perform linear projection and dimensionality reduction on the high-dimensional histogram incremental feature vector through a fully connected layer to achieve spatial feature compression and obtain a low-dimensional compressed feature vector.

[0092] Specifically, due to the number of time slots in the photon flight time histogram Typically, large values ​​(e.g., 150) directly input into a recurrent feature extraction model can lead to parameter redundancy. However, the effective information in the histogram incremental feature vector is mainly concentrated within the signal peak and its neighborhood of several time bins. Low-dimensional features are sufficient to capture this sparse distribution information, significantly reducing subsequent computation. Therefore, an encoding unit is used, and the input vector is processed through a fully connected layer. Mapped to low-dimensional compressed feature vectors ,in, Indicates by The original histogram feature space composed of time bins (e.g.) ), This represents the compressed feature space after dimensionality reduction.

[0093] The formula for calculating the encoder unit is:

[0094] ;

[0095] in, This is the weight matrix of the coding unit. This is the bias vector for the coding unit. Preferably, the compression dimension... Set to 3 or 4.

[0096] The Memory Cell is used to receive the compressed feature vector, concatenate and fuse the compressed feature vector with the memory state vector from the previous detection stage, and perform nonlinear operations on the concatenated features through a fully connected layer and activation function to obtain the current memory state vector.

[0097] Specifically, the circular memory unit is configured to achieve the fusion of temporal information and implicit parameter estimation, and its input is the current compressed feature vector. and the memory state vector of the previous detection phase , The dimension is Hidden memory space (e.g.) The current memory state vector is updated using the following formula:

[0098] ;

[0099] in, This represents a vector concatenation operation. This is the weight matrix of the circular memory unit. This is the bias vector for the circular memory unit. It is a linear rectification activation function.

[0100] The circular memory unit uses ReLU (Linear Rectified Activation Function) instead of the traditional Tanh or Sigmoid functions. This is based on the monotonicity of the SPAD photon accumulation physical process—as time progresses, evidence of the signal's existence only increases or remains, without negative forgetting. The non-negativity and linearity of ReLU better align with this physical law, and in digital circuit implementation, only simple comparison logic is required, avoiding complex exponential operations.

[0101] Therefore, the formula for calculating the current memory state vector is expressed as:

[0102] ;

[0103] In one specific embodiment, the uncertainty prediction model employs a multilayer perceptron network.

[0104] Specifically, the uncertainty prediction model is also called an estimation unit. This unit is configured to map the hidden state to uncertainty probability values ​​and fit the model. The physical relationship. This unit adopts a multilayer perceptron (MLP) network structure, and the input is the current memory state vector. And the index value of the detection phase of explicit injection .

[0105] Specifically, a multilayer perceptron includes an input layer, at least one hidden layer, and an output layer.

[0106] The hidden layer operation is as follows: ;

[0107] The operations of the output layer are: ;

[0108] Therefore, the formula for calculating the uncertainty prediction value is:

[0109] ;

[0110] in, For uncertain predicted values, The range of values ​​is , These are the weights of the hidden layers in a multilayer perceptron. These are the bias parameters of the hidden layers in a multilayer perceptron. The weights of the output layer in a multilayer perceptron. These are the bias parameters of the output layer in a multilayer perceptron. It is a linear rectified activation function. Using the Sigmoid activation function ensures the uncertainty of the output prediction value. Strictly constrained Within the interval, it is given a clear probabilistic physical meaning.

[0111] The total number of parameters in the aforementioned multilayer perceptron network structure is only about 1.8k (floating-point numbers), which can be quantized into 8-bit fixed-point numbers in actual deployment, with a model size of less than 2KB. This allows the method to be directly embedded into the readout circuit logic of the sensor front end or into a small FPGA, achieving ultra-low power inference at the microwatt (μW) level.

[0112] Example 3

[0113] Based on Example 2, this example provides a training method for a cyclic feature extraction model and an uncertainty prediction model, so that the network model has the ability to generalize to different scenarios.

[0114] The training method for the cyclic feature extraction model and the uncertainty prediction model in this embodiment includes the following steps:

[0115] S301. Obtain a simulation dataset containing various combinations of signal-to-noise ratio, background luminous flux, and pulse width.

[0116] Since it is difficult to obtain the "real uncertainty" label at every moment in real-world scenarios, this embodiment uses a Monte Carlo simulation method based on a physical model to construct a simulation dataset.

[0117] Specifically, an imaging parameter space is defined, covering a signal-to-noise ratio (SBR) range of [0.01, 10.0], a background luminous flux range of [0.001, 1.0] photons / period / pixel, and different laser pulse width parameters. A large number of scene combinations (e.g., 10,000 sets) are generated by random sampling within the above space.

[0118] S302. Generate photon arrival time series for each scenario combination in the simulation dataset, construct a photon flight time histogram for each detection stage, and generate a histogram increment sequence using the photon flight time histogram for each detection stage.

[0119] Specifically, for each set of scene parameters, a photon arrival time series is generated using the Poisson response statistical model of SPAD. Following the detection stage time intervals in Example 1, a photon time-of-flight histogram for each detection stage is constructed, forming a corresponding histogram evolution sequence sample. Then, an incremental histogram feature vector is generated using the photon time-of-flight histogram for each detection stage.

[0120] S303. For each detection stage of each scenario combination, calculate the ground truth error rate at each time point as an uncertainty label.

[0121] For each detection stage of each sample sequence in the simulation dataset The true depth estimation error rate was calculated through numerous repeated experiments. Specifically, for the current scene parameters and cumulative time, the experiment was repeatedly run... Next (for example) Independent simulation of photon accumulation and depth calculation. Statistics show that the depth estimation results deviate from the true distance by more than the allowable error (e.g., ...). The number of times (cm) .

[0122] The uncertainty label is calculated as follows:

[0123] ;

[0124] in, To estimate the error rate for true depth, This refers to the number of repetitions required for photon accumulation in the current scene combination and detection phase. This represents the number of times the depth estimate deviates from the true distance beyond the allowable error. This uncertainty label accurately reflects the true risk of performing depth estimation under current physical conditions.

[0125] S304. Construct a hybrid loss function that includes a mean squared error term and a mean absolute error term.

[0126] To maintain high sensitivity of the model in the critical region where uncertainty decreases rapidly (i.e., the transition region from unreliable to reliable), this embodiment constructs a hybrid loss function that includes a mean squared error (MSE) term and a mean absolute error (MAE) term:

[0127] ;

[0128] in, It is a mixed loss function; To balance the weighting coefficients, it is preferable to set them to 0.7; The mean squared error is used to guide the model to quickly fit the overall trend. The mean absolute error is used to optimize the prediction accuracy in low uncertainty intervals and prevent premature or late truncation. The uncertainty prediction value output by the uncertainty prediction model. Estimate the error rate for the true depth.

[0129] S305. Using the histogram increment sequence as input, the true depth estimation error rate... As an uncertainty label, the network parameters of the recurrent feature extraction model and the uncertainty prediction model are iteratively updated using the backpropagation over time (BPTT) algorithm until the mixed loss function converges, resulting in the trained recurrent feature extraction model and the trained uncertainty prediction model. The trained parameters can then be deployed in the processing logic of the imaging system.

[0130] Example 4

[0131] Based on Examples 1, 2, and 3, this example illustrates the effectiveness of the adaptive single-photon cumulative imaging method based on cyclic uncertainty estimation through simulation.

[0132] To quantify the technical problem solved by this invention and verify the theoretical basis of adaptive acquisition, this embodiment uses the constructed Adaptive Single-Photon Acquisition Dataset (ADP-SPAD) to conduct experimental statistics on the "necessary acquisition cycles" (i.e. the minimum number of cycles required to achieve depth accuracy) of a large number of samples under different imaging conditions, and compares them with the traditional fixed strategy.

[0133] Please see Figure 5 , Figure 5 To ensure a constant background photon flux ( The diagram illustrates the impact of different signal-to-noise ratios (SBRs) on acquisition requirements under experimental conditions. Experimental data shows that in low SBR scenarios, the necessary number of cycles is close to the upper limit set by the fixed strategy to ensure image quality. However, as the SBR increases (e.g., when the SBR reaches 0.215), the experimentally measured necessary number of cycles decreases significantly, at which point the proportion of redundant cycles generated by the fixed strategy reaches as high as 56.4%.

[0134] Please see Figure 6 , Figure 6 This diagram illustrates the impact of photon flux variation on acquisition requirements under experimental conditions with a constant signal-to-noise ratio (SBR = 0.0464). Experimental data shows that photon flux is a key physical quantity affecting acquisition efficiency. When the photon flux is low, the system requires a longer time to accumulate photons; when the photon flux increases (e.g., reaching 1.16 photons / cycle), the experimentally measured required number of cycles decreases sharply. If a fixed strategy is still implemented at this point, the redundant cycle ratio will further increase to 85.2%.

[0135] The experimental results above confirm that the optimal number of acquisition cycles is not a fixed value, but rather dynamically varies with scene parameters (SBR, photon flux). The "redundant cycles" shown in the figure represent the power consumption and time margin saved by the method of this invention. By predicting uncertainties in real time, this invention can accurately locate the necessary cycle cutoff point in the figure, thereby eliminating the accumulated time redundancy observed in the above experiments and thus improving the imaging frame rate of the SPAD detector.

[0136] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation, characterized in that, Including the following steps: In the current detection phase, obtain the current photon flight time histogram accumulated inside the detector; Perform temporal difference processing on the current photon flight time histogram to calculate the histogram increment feature vector of the current detection stage relative to the previous detection stage; The trained recurrent feature extraction model is used to perform nonlinear operations on the memory state vector of the previous detection stage and the histogram incremental feature vector to update the current memory state vector; wherein, the memory state vector is configured to implicitly encode the signal-to-noise ratio and ambient background light intensity of the current imaging scene. The current memory state vector is concatenated with the current detection stage index value, and the concatenated vector is fused and mapped using a trained uncertainty prediction model to obtain the uncertainty prediction value of the current depth estimate; the detection stage index value is used to characterize the total number of laser pulse cycles that have been accumulated. If the uncertainty prediction value is less than or equal to the preset confidence threshold, a collection termination signal is generated to stop photon collection and output the current photon time-of-flight histogram; if the uncertainty prediction value is greater than the preset confidence threshold, a continue collection signal is generated to enter the next detection stage.

2. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 1, characterized in that, The histogram increment feature vector is: ； in, This is the histogram incremental feature vector. This is the current photon flight time histogram. This is the photon flight time histogram from the previous detection phase.

3. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 1, characterized in that, The recurrent feature extraction model includes an encoding unit and a recurrent memory unit, wherein, The encoding unit is used to perform linear projection and dimensionality reduction on the histogram incremental feature vector to obtain a compressed feature vector. The recurrent memory unit is used to concatenate and fuse the compressed feature vector with the memory state vector of the previous detection stage, and to perform nonlinear operations on the concatenated features through a fully connected layer and an activation function to obtain the current memory state vector.

4. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 3, characterized in that, The formula for calculating the current memory state vector is: ； in, For the first The memory state vector of each detection phase For the first The memory state vector of each detection phase This is the histogram incremental feature vector. This is the weight matrix of the coding unit. This is the bias vector of the coding unit. This is the weight matrix of the circular memory unit. This is the bias vector for the circular memory unit; This represents a vector concatenation operation. It is a linear rectification activation function.

5. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 1, characterized in that, The uncertainty prediction model employs a multilayer perceptron network.

6. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 5, characterized in that, The formula for calculating the uncertainty prediction value is as follows: ； in, For uncertain predicted values, The range of values ​​is , These are the weights of the hidden layers in a multilayer perceptron. These are the bias parameters of the hidden layers in a multilayer perceptron. The weights of the output layer in a multilayer perceptron. These are the bias parameters of the output layer in a multilayer perceptron. It is the Sigmoid activation function. It is a linear rectified activation function. For the first The memory state vector of each detection phase This is the index value for the detection phase. Indicates the first The memory state vector and the index value of each detection phase splicing.

7. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 1, characterized in that, The training methods for the recurrent feature extraction model and the uncertainty prediction model include: Obtain a simulation dataset containing various combinations of signal-to-noise ratio, background luminous flux, and pulse width; For each scenario combination in the simulation dataset, a photon arrival time series is generated, and a photon time-of-flight histogram is constructed for each detection stage. A histogram increment sequence is generated using the photon time-of-flight histogram for each detection stage. For each detection stage of each scenario combination, the true depth estimation error rate at each time point is calculated as an uncertainty label; Construct a hybrid loss function that includes a mean squared error term and a mean absolute error term; Using the histogram increment sequence as input and the true depth estimation error rate as the uncertainty label, the network parameters of the recurrent feature extraction model and the uncertainty prediction model are iteratively updated using the backpropagation algorithm until the hybrid loss function converges, thus obtaining the trained recurrent feature extraction model and the trained uncertainty prediction model.

8. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 7, characterized in that, The uncertainty label is: ； in, To estimate the error rate for true depth, This refers to the number of repetitions required for photon accumulation in the current scene combination and detection phase. This represents the number of times the depth estimation result deviates from the true distance by more than the allowable error.

9. The adaptive single-photon accumulation imaging method based on cyclic uncertainty estimation according to claim 7, characterized in that, The hybrid loss function is: ； in, For a mixed loss function, To balance the weighting coefficients, Mean square error, The mean absolute error, The uncertainty prediction value output by the uncertainty prediction model. Estimate the error rate for the true depth.

10. A planar array lidar imaging system, characterized in that, It includes multiple pixel units, and executes the method as described in any one of claims 1-9 for each pixel unit, thereby controlling the acquisition termination time of each pixel unit to achieve pixel-level adaptive exposure control.

Images