A camera design optimization method based on single-photon density imaging

Abstract

The application discloses a camera design optimization method based on single-photon density imaging, which relies on a single-photon avalanche diode (SPAD) device to capture incident photon information, and then constructs a photon cube for representing the light detection condition at each pixel point and the characteristic information of the photons. The arrival of the photons can be modeled as a Poisson process, the time contrast projection technology is adopted to generate photon events, and the exponential response curve is used to avoid underflow, the sensitivity under low light conditions is improved by optimizing the event triggering mechanism, and the imaging performance is optimized. The application achieves the imaging effect through the photon cube information collected by the SPAD and the corresponding algorithm. And through the optimization of the algorithm, higher video reconstruction quality and lower bandwidth cost are realized, and the application has good practical value.

Description

Technical Field

[0001] This invention relates to a camera design optimization method based on single-photon density imaging. It belongs to the technical field of combining computer vision and computational photography. Background Technology

[0002] The earliest imaging technology originated from the pinhole imaging principle, where light passing through a small hole creates an inverted image on the opposite wall in a dark room. This principle laid the foundation for the development of optical imaging. Throughout the development of imaging technology, image capture and processing technologies have always been intertwined. Thanks to continuous innovation in sensor technology, the resolution and efficiency of information acquisition have been significantly improved. In the era of film photography, the core of image processing lay in precise exposure control and complex development processes. However, with the rapid development of digital photography technology, image processing has reached pixel-level precision, greatly promoting the overall improvement of image quality.

[0003] Light field photography captures information from the entire light field, facilitating granular processing of the data in post-capture stages and enabling digital editing. This technology has spurred exploration of the limits of imaging technology. Precise calculation and processing of individual photons would significantly improve image acquisition speed and resolution.

[0004] Single-photon imaging is an advanced technology that indirectly acquires information about objects based on the interaction of photons with matter. It constructs an image by detecting the number of photons reflected or emitted from the surface of an object. This technology has extremely high sensitivity, enabling it to accurately capture weak light signals and provide detailed imaging.

[0005] Photon density imaging is an innovative photon imaging technique that acquires images by measuring the density distribution of photons per unit volume. Compared to traditional imaging methods, photon density imaging offers significant advantages in spatial resolution and contrast, exhibiting superior imaging capabilities, especially when dealing with targets with limited photon counts or weak signals. This technology has profound implications for improving imaging accuracy and enhancing image quality in weak signal environments, providing a new solution to imaging problems in complex environments.

[0006] In summary, this invention proposes a camera design optimization method based on single-photon density imaging. This method utilizes single-photon devices to capture photon signals, combines temporal contrast projection technology to generate photon events, and avoids underflow by introducing an exponential response curve. By optimizing the event triggering mechanism, it improves light sensitivity under low-light conditions, thereby optimizing imaging performance. Summary of the Invention

[0007] This invention proposes a camera design optimization method based on single-photon density imaging. The method uses a single-photon avalanche diode (SPAD) as the core device to accurately capture incident photon information and constructs a photon cube—a binary frame sequence composed of time series data—to characterize the photon detection event and its related information at each pixel. Within this framework, the arrival of incident photons is represented by a Poisson process, and the state of each pixel in the photon cube is represented as an independent Bernoulli random variable. Among them B t (x) represents photon detection at time t and pixel position x, where 1 indicates a photon detected and 0 indicates no photon detected. Φ(x, t) represents the number of photons received at time t and pixel x, η is the photon detection efficiency, and r q ω is the counting rate of the sensor. exp This refers to the exposure time.

[0008] Projection calculations onto the generated photon cube are the main step in imaging. An adaptive time-integration projection algorithm is employed to accumulate the number of photons within a specific time region. This is done by summing up the photon detection results from multiple time periods. Obtaining a photon count image within a time region allows for the generation of relatively clear images in static scenes. By performing temporal contrast projection on a photon cube, the aim is to extract temporal brightness variation information from photon detection data.

[0009] This invention presents a camera design optimization method based on single-photon density imaging. The main technical solutions adopted are as follows:

[0010] Step 1: Use a single-photon avalanche diode (SPAD) unit array as an imaging device to capture photon data in the cube with high temporal resolution;

[0011] Step 2: Collect single-photon data. Preprocess the photon data captured in the cube. Since the number of photons obtained per unit time is small, a longer period of photon accumulation is required to obtain enough sampled photons to construct the image.

[0012] Step 3: Construct an image based on the detected single photon position and arrival time, and use signal processing techniques to reduce noise and optimize image quality;

[0013] Step 4: Project the data in the collected cubic photons and use the time contrast for event detection to achieve event detection and judgment under sufficient illumination conditions.

[0014] Furthermore, the specific steps of step 1 are as follows:

[0015] Step 1.1: Use SPAD to perform photoelectric conversion on the incident photons, such as... Figure 1As shown, photons incident on SPAD are absorbed by the semiconductor material. The absorbed photons gain energy to generate an excited electron, which converts the photon information into an electrical signal.

[0016] Step 1.2: Set the size, cell spacing, and frame rate (100kHz) of the SPAD array; calibrate the dark current of the SPAD cell array to reduce false triggering under no-light conditions; adjust the SPAD gain and optimize the photon detectivity of the cells;

[0017] Step 1.3: Synchronize the SPAD unit array with the external clock to achieve accurate frame rate control and set the exposure time of each unit to adapt to the lighting environment;

[0018] Step 1.4: During the exposure period, each SPAD unit independently detects the incident photons and converts them into binary values ​​(0 or 1).

[0019] Furthermore, the specific steps of step 2 are as follows:

[0020] Step 2.1: Start the light source and adjust the light intensity to the single-photon level, initialize the data and start data acquisition. The unit detector records the arrival time and location of each photon event.

[0021] Step 2.2: Since the number of single photons captured in a short time is small, long-term accumulation is required to ensure signal quality. The accumulated signal value of each unit detector is represented by the image grayscale value.

[0022] Step 2.3: Record the time it takes for a photon to travel from emission to arrival at the detector, and calculate more accurate information about the position of each pixel by averaging the time distribution of multiple photon events;

[0023] Furthermore, the specific steps of step 3 are as follows:

[0024] Step 3.1: Filter the acquired photon events to remove background noise and abnormal data in order to obtain a clearer initial image;

[0025] Step 3.2: Apply denoising algorithms such as average filtering or Gaussian filtering to the accumulated photon data to reduce noise and improve image clarity;

[0026] Step 3.3: Use image reconstruction algorithms to enhance resolution and detail on the acquired photon data.

[0027] Furthermore, the specific steps of step 4 are as follows:

[0028] Step 4.1: The single-photon camera is based on a novel image sensor array unit. Unlike traditional frame rate cameras, it does not capture continuous image frames at a fixed frame rate, but records brightness changes in the scene asynchronously at pixel positions.

[0029] Step 4.2: Detect brightness changes independently for each pixel. When the change exceeds a preset threshold, a record is generated: |V(x,t)-V ref (x)|>τ, after the SPAD sensor detects a photon, it generates a photosensitive voltage V(x,t). The deviation of this voltage from the reference voltage is V. ref (x), after a single capture, the new voltage value is updated to the reference voltage V. ref (x) provides a benchmark for subsequent detection. τ is the contrast threshold;

[0030] Step 4.3: The photon cube consists of photon detection data captured by SPAD at high frame rates. Each layer represents the photon detection result at a single moment. Due to the large random noise in single-frame data, an exponential moving average (EMA) is used to smooth the data of each time frame to obtain the current photon intensity estimate: μ t (x)=(1-β)B t (x)+βμ t-1 (x). Where β is the smoothing factor, B t (x) represents the photon detection result, and the event is triggered under the following condition: |h(μ(x))-h(μ) ref (x))|>τ, where h is a scalar function applied to the EMA to transform the estimated value of photon intensity. Logarithmic or exponential functions are usually chosen to adapt to the response characteristics of the sensor.

[0031] Step 4.4: The logarithmic function h(x) = log(x) is prone to underflow under low light conditions. Therefore, the exponential response curve h(x) = 1 - e^(-x / x) of a SPAD camera is used. -αΦ(x，t) ,like Figure 2 As shown, α is a constant independent of photon flux, and Φ(x, t) is the actual light intensity of pixel x at time t. This response curve can avoid the underflow problem of the logarithmic function under low light conditions. The smoothing coefficient β and contrast threshold τ are dynamically adjusted according to the pixel position or incident light intensity to enhance the camera's sensitivity and prevent the camera from failing to record due to insufficient brightness changes in low light conditions, thereby achieving the goal of optimizing camera functions.

[0032] The beneficial effects of this invention are that, through single-photon detection, time-contrast projection technology, exponential response curve, and optimized event triggering mechanism, the camera can still obtain high-quality imaging effects under low-light conditions, which has certain practical value. Attached Figure Description

[0033] Figure 1 This is the process of single-photon imaging.

[0034] Figure 2 This is the nonlinear response curve of SPAD. Detailed Implementation

[0035] This invention provides a camera design optimization method based on photon density imaging. The invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] 1. Single-photon imaging

[0037] Single-photon imaging is a sensitive new imaging technique that constructs a mapped image of a scene by detecting a very small number of incident photons and performing photoelectric conversion. It utilizes single-photon detectors (such as single-photon avalanche diodes, SPADs) to capture photon information under low light intensity, and constructs an image through statistical methods and data processing. The specific working principle is as follows:

[0038] Photon detection: Single-photon imaging systems use highly sensitive detectors to convert the incident signal of a single photon into an electrical signal, capturing incident photons with extremely low light intensity.

[0039] Time-resolved measurement: In time-resolved single-photon imaging, the detector records not only the spatial position of each photon but also its arrival time. Based on the photon's time of flight, the path length of the photon reflected from the light source is calculated, thereby determining the depth information of the object's distance.

[0040] Statistical accumulation: Since only a very small number of photons (or even a single photon) can be captured at a time in single-photon imaging, it is necessary to accumulate photons over a time interval. The imaging system collects a flux of photons over time intervals to statistically analyze and reconstruct the scene image. In this process, pixel locations accumulate enough photons over multiple time intervals to form a clear image.

[0041] Post-processing: The initial images generated by single-photon imaging are often sparse and require post-processing. Common post-processing methods include denoising, image reconstruction, and temporal correlation analysis.

[0042] 2. Photon Cube

[0043] A photon cube is a three-dimensional data structure designed to store detection data from a single-photon camera, recording the sensor's photon detection information in time and space. It is a high-dimensional data volume composed of a series of binary image frames captured by a single-photon detector within consecutive exposure cycles. It provides a flexible way to capture and process photon-level imaging data. A photon cube typically has three dimensions:

[0044] Spatial dimension: used to capture the specific location of a photon incident on a two-dimensional plane, corresponding to the two-dimensional pixel array of the imaging detector, where each pixel records the photon event detected at its location.

[0045] Time dimension: Used to record the arrival time of photons, corresponding to continuous time intervals. In each time interval, all pixels will independently record the arrival of photons.

[0046] Detection value (0 or 1): Used to record whether a photon event was detected at a specific time frame and pixel position. 0 indicates that no photon event was detected, and 1 indicates that a photon event was detected.

[0047] Photon cubes, as a technology for multi-dimensional data processing and imaging, provide higher resolution, accuracy, and diverse analytical dimensions for the imaging process by combining temporal, spatial, and detection information.

[0048] In summary, this invention proposes a camera design optimization method based on single-photon density imaging. This method is based on the projection method of photon cubes and uses exponential response curves, exponential moving average smoothing, and adjustment of contrast threshold to alleviate the underflow problem of the camera under low light conditions, improve the sensitivity of event generation, and optimize the camera function.

[0049] The basic principles, main features, and advantages of this invention have been described above. Those skilled in the art should understand that the above embodiments do not limit the scope of protection of this invention in any way, and all technical solutions obtained by equivalent substitution or other means fall within the scope of protection of this invention.

[0050] All parts not covered in this invention are the same as or can be implemented using existing technologies.

Claims

1. A method for camera design optimization based on single-photon density imaging, the method comprising: It comprises the following steps: Step 1: using a single photon avalanche diode (SPAD) cell array as an imaging device to capture photon data in a cube with high time-domain resolution; Step 2: collecting single photon data, pre-processing the captured photon data in the cube, and accumulating photons for a long time to obtain sufficient sampling photons to construct an image due to the small number of photons obtained per unit time; Step 3: constructing an image according to the detected single photon position and arrival time, and using signal processing techniques to reduce noise and optimize image quality; Step 4: projecting the collected cube photon data, and using time contrast for event detection to realize event detection and judgment under sufficient illumination conditions; Step 4.1: Each pixel independently detects the change of brightness, and a record is generated when the change exceeds a preset threshold, |V(x, t) - V ref (x)| > τ, where V(x, t) is the light-sensing voltage generated by the SPAD sensor after detecting photons; the deviation of this voltage from the reference voltage V ref (x) is V(x, t); after a single capture, the new voltage value is updated as the reference voltage V ref (x), providing a basis for subsequent detection, and τ is the contrast threshold. Step 4.2: The photon cube consists of photon detection data captured by SPADs at high frame rate, each layer represents the photon detection result at a single time instant. Due to the large random noise of single frame data, in order to smooth the brightness, the data of time frame is smoothed by using exponential moving average (EMA) to obtain the current photon intensity estimate: μ t (x) = (1 - β)B t (x) + βμ t-1 (x), where β is the smoothing factor, B t (x) represents the photon detection result, the condition of event triggering is as follows: |h(μ(x))-h(μ ref (x))|>τ, wherein h is a scalar function applied to EMA, which is used to transform the estimated value of photon intensity; Step 4.3: The log(x) function is prone to underflow in low light conditions, so the exponential response curve h(x) = 1-e -αΦ(x,t) where a is a constant independent of photon flux, and Φ(x, t) is the actual light intensity of pixel point x at time t. This response curve can avoid the underflow problem of the log function in low light conditions; the smoothing coefficient β and the contrast threshold τ are dynamically adjusted according to the pixel position or the incident light intensity, which enhances the sensitivity of the camera and avoids the camera from failing to record due to insufficient brightness changes in low light, thereby achieving the purpose of optimizing the camera function.

2. The camera design optimization method based on single-photon density imaging according to claim 1, wherein, The specific steps of step 1 are as follows: Step 1.1: photoelectric conversion of incident photons by SPAD, photons incident on SPAD are absorbed by semiconductor material, and an excited electron is generated by absorbing photons to convert photon information into an electrical signal; Step 1.2: setting the size, cell spacing and frame rate of the SPAD array, calibrating the dark current of the SPAD cell array to reduce false triggering under no light conditions; adjusting the SPAD gain to optimize the photon detection rate of the cell; Step 1.3: synchronizing the SPAD cell array with the external clock to achieve accurate frame rate control, and setting the exposure time of each cell to adapt to the lighting environment; Step 1.4: exposure period, SPAD cells independently detect incident photons and convert them to binary values.

3. The camera design optimization method based on single-photon density imaging according to claim 1, wherein, The specific steps of step 2 are as follows: Step 2.1: start the light source and adjust the light intensity to the single photon level, initialize the data and start data collection, and the cell detector records the arrival time and position of each photon event; Step 2.2: due to the small number of single photons captured in a short time, long-time accumulation capture is required to ensure signal quality, and the signal accumulation value of each cell detector is represented by image gray value; Step 2.3: record the time from photon emission to arrival at the detector, and calculate more accurate information for each pixel position by averaging the time distribution of multiple photon events.

4. The camera design optimization method based on single-photon density imaging according to claim 1, wherein, The specific steps of step 3 are as follows: Step 3.1: filtering the collected photon events to remove background noise and abnormal data to obtain a clearer initial image; Step 3.2: applying average filtering or Gaussian filtering to the accumulated photon data to reduce noise and improve image clarity; Step 3.3: using image reconstruction algorithms to enhance resolution and details for the collected photon data.

Images