SPAD-based sparse 3D ranging system and method based on near-pixel sensing

Abstract

This invention discloses a SPAD sparse 3D ranging system and method based on near-pixel end sensing. The system includes: a photon counting circuit for accumulating the number of photon event pulse signals output by SPAD pixels and outputting a two-dimensional intensity image; a ROI extraction circuit for obtaining a binary ROI image based on an adaptive threshold conversion; a morphological filtering circuit for filtering and ROI fusion operations on the binary ROI image to generate an ROI mask; a reconfigurable connection network for dynamically establishing electrical connection paths between SPAD pixels within the ROI mask and the corresponding TOF extraction circuits; and a TOF extraction circuit for acquiring the TOF results of SPAD pixels within the ROI mask and outputting a sparse 3D image. This invention reduces the number of spatial scans required for a single frame image and solves the data redundancy problem caused by traditional SPAD LiDAR outputting full-resolution 3D images.

Description

Technical Field

[0001] This invention belongs to the field of lidar and three-dimensional sensing technology, specifically relating to a SPAD sparse three-dimensional ranging system and method based on near-pixel end sensing. Background Technology

[0002] SPAD (Single Photon Avalanche Diode) is widely used in lidar systems due to its high sensitivity and picosecond-level time resolution. However, the high sensitivity and nonlinear characteristics of SPAD devices result in a low signal-to-noise ratio for their raw output signal. Therefore, time-correlated single-photon counting techniques based on histogram statistics are typically used to extract the time of flight (TOF).

[0003] Currently, most existing technologies employ time-division multiplexing architectures to reduce the circuit area required for histogram storage. However, with the improvement of LiDAR ranging accuracy, measurement dynamic range, and imaging resolution, the storage resource requirements for histograms further increase, limiting further improvements in LiDAR imaging resolution and measurement frame rate. While step-by-step histogram methods reduce storage resource requirements, their measurement efficiency is relatively low. Successive approximation methods support intra-pixel integration with low resource consumption, but this method requires multiple repeated measurements, and the effective signal ratio is low in the initial stage, resulting in insufficient noise suppression capabilities. Indirect time-of-flight (iTOF) technology can effectively compress circuit area, but its measurement range is relatively limited.

[0004] Furthermore, mainstream SPAD LiDAR systems output full-resolution 3D images, leading to redundant output data and increasing the computational load on backend image processing algorithms. Dynamic Vision Sensing (DVS) methods can detect moving targets in a scene and extract Time-of-Flight (TOF) using histogram algorithms only when a moving target is detected, thus reducing redundant data. However, this method is only applicable to dynamic scenes and cannot effectively handle the measurement of static targets. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a SPAD-based sparse 3D ranging system and method based on near-pixel end sensing. The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a SPAD-based sparse 3D ranging system based on near-pixel end sensing, comprising: Multiple photon counting circuits are connected to a SPAD pixel in the SPAD detector array to accumulate the number of photon event pulse signals output by each SPAD pixel within a preset exposure time, obtain light intensity data, and output a two-dimensional intensity image. A ROI extraction circuit is used to generate an adaptive threshold from the two-dimensional intensity image, so as to convert the two-dimensional intensity image into a binary ROI image according to the adaptive threshold. A morphological filtering circuit is used to perform filtering and ROI fusion operations on the binary ROI image to generate an ROI mask. A reconfigurable connection network and a TOF extraction circuit are provided. The reconfigurable connection network is used to dynamically establish electrical connection paths between SPAD pixels in the ROI mask and the corresponding TOF extraction circuit according to the ROI mask. The TOF extraction circuit is used to obtain the TOF results of SPAD pixels in the ROI mask and output a sparsed 3D image.

[0006] In one embodiment of the present invention, the photon counting circuit includes an integration interval control circuit and an accumulator counter; wherein, The integration interval control circuit is used to receive the start integration signal and integration duration register configuration value sent by the controller, and to enable the integration interval signal when the start integration signal is detected and to stop enabling the integration interval signal after the preset exposure time is reached. The accumulator counter includes an N-bit counter. The reset and enable terminals of the counter are both connected to the integration interval control circuit. The accumulator counter is used to accumulate and count the photon event pulse signals output by the corresponding SPAD pixel within a preset exposure time and output light intensity data.

[0007] In one embodiment of the present invention, the ROI extraction circuit includes: an adaptive threshold generation circuit and a binary comparison circuit, wherein the adaptive threshold generation circuit includes: an intensity histogram circuit and a valley detection circuit; wherein, The intensity histogram circuit includes a depth of The RAM storage unit is used to take the intensity value of each pixel in the two-dimensional intensity image as an address input, and by incrementing the stored value at the corresponding address by 1, the frequency of occurrence of each intensity value in the two-dimensional intensity image is obtained to generate an intensity histogram; wherein, ; The valley detection circuit is used to sequentially read the stored values ​​in the RAM storage unit, perform smoothing filtering processing on a group of consecutive preset number of stored values, locate the local minimum value in each group of stored values, and when the first gradient corresponding to the location of the local minimum value is 0, the local minimum value is determined as a valley. At least one adaptive threshold is determined based on the intensity values ​​corresponding to all valley locations in the intensity histogram. The binary comparison circuit is used to input each adaptive threshold and the two-dimensional intensity image into the comparator respectively. By comparing each intensity data with each adaptive threshold, it outputs a 1-bit binary result to obtain multiple ROI binary images.

[0008] In one embodiment of the present invention, the morphological filtering circuit includes: a fill shift circuit, a morphological manipulation circuit, and a ROI fusion circuit; wherein, The fill-shift circuit is used to perform edge filling processing on the ROI binary image according to the type of morphological operation, and then input the filled ROI binary image into the morphological operation circuit in columns. The morphological operation circuit is used to perform multi-level delayed storage on the filled ROI binary image, and to perform morphological operations under the constraints of structuring elements to output the morphologically filtered ROI binary image. The ROI fusion circuit is used to perform three erosion operations on each morphologically filtered ROI binary image, subtract the calculation result from the morphologically filtered ROI binary image to obtain multiple ROI gradient images, and further superimpose all ROI gradient images to generate an ROI mask.

[0009] In one embodiment of the present invention, including A TOF extraction circuit, the reconfigurable connection network includes a connection network and Each traversal unit This indicates the larger of the number of rows and columns of SPAD pixels in the SPAD detector array. The working states of the traversal unit include: traversal scanning, pixel locking, and scanning end. The reconfigurable connection network includes a horizontal boundary TOF extraction stage and a vertical boundary TOF extraction stage. In the horizontal boundary TOF extraction stage, the traversal unit is used to traverse the corresponding row in the ROI mask in the traversal scanning state and search for macro pixels that match the boundary features in the row; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scan end state, it enables its own Flag1 and Flag2 flag signals to indicate that the traversal is complete; after all traversal units have completed the traversal, the vertical boundary TOF extraction stage begins. In the vertical boundary TOF extraction stage, the traversal unit is used to traverse the corresponding column in the ROI mask in the traversal scanning state and search for macro pixels that match the boundary features in the column; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scanning end state, it enables its own Flag1 and Flag2 flag signals to indicate the end of traversal and complete the measurement of the current frame. The connection network is used to establish a connection path between the locked horizontal boundary macropixel and the corresponding TOF extraction circuit when the Flag1 flag signal of all traversal units is enabled during the horizontal boundary TOF extraction stage; and to establish a connection path between the locked vertical boundary macropixel and the corresponding TOF extraction circuit when the Flag1 flag signal of all traversal units is enabled during the vertical boundary TOF extraction stage.

[0010] In one embodiment of the present invention, the TOF extraction circuit includes: a sampling circuit and a histogram circuit; wherein, The sampling circuit is used to synchronously sample the photon event pulse signal output by the SPAD pixel selected by the reconfigurable connection network in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage, respectively, to obtain the quantized timestamp. The histogram circuit is used to accumulate histograms of the photon arrival time distribution of macropixels locked in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage according to the quantization timestamp, obtain the TOF result through peak detection, and further convert the TOF result into a depth value to output a sparse 3D image of ROI mask mapping.

[0011] In one embodiment of the present invention, the connection network is further configured to, during the horizontal boundary TOF extraction stage, reset the Flag1 flag signal of the traversal unit that has not completed row traversal after obtaining the TOF result, so that the traversal unit that has not completed row traversal continues row traversal search until the Flag2 flag signal of all traversal units is enabled, thus completing the horizontal boundary TOF extraction; and during the vertical boundary TOF extraction stage, after obtaining the TOF result, reset the Flag1 flag signal of the traversal unit that has not completed column traversal, so that the traversal unit that has not completed column traversal continues column traversal search until the Flag2 flag signal of all traversal units is enabled, thus completing the vertical boundary TOF extraction.

[0012] Secondly, the present invention also provides a SPAD-based sparsified three-dimensional ranging method based on near-pixel end sensing, which is applied to the system described in the first aspect.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: The SPAD sparse 3D ranging system based on near-pixel end sensing provided by this invention includes: a photon counting circuit for generating a 2D intensity image, a Region of Interest (ROI) extraction circuit for extracting the region of interest from the 2D intensity image, a morphological filtering circuit for generating an ROI mask, a reconfigurable connection network for establishing physical connections between ROI macropixels and the Time-of-Flight (TOF) extraction circuit, and a TOF extraction circuit for calculating the distance to the target object. Compared with existing technologies, this invention proposes an ROI extraction mechanism based on light intensity distribution, based on a "sensing first, measuring later" strategy, and performs TOF extraction only on macropixels within the ROI region, thereby outputting a sparse 3D image of the ROI mapping. This reduces the number of spatial scans required for a single frame image and solves the data redundancy problem caused by the output of full-resolution 3D images in traditional SPAD LiDAR, significantly improving the system's real-time performance. Furthermore, since this invention obtains the ROI by analyzing the 2D intensity image, it can accommodate measurements of both dynamic and static scenes.

[0014] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of a SPAD sparse three-dimensional ranging system based on near-pixel end sensing provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the principle of ROI extraction based on two-dimensional intensity images provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the morphological manipulation circuit provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the reconfigurable connection network provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the reconfigurable connection and TOF extraction process provided in the embodiments of the present invention; Figure 6 This is a flowchart illustrating a SPAD sparse 3D ranging method based on near-pixel end sensing provided in an embodiment of the present invention. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0017] Figure 1 This is a schematic diagram of a SPAD sparse 3D ranging system based on near-pixel end sensing provided in an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the principle of ROI extraction based on two-dimensional intensity images provided in an embodiment of the present invention. Figures 1-2As shown, this embodiment of the invention provides a SPAD sparse 3D ranging system 100 based on near-pixel end sensing, comprising: Multiple photon counting circuits 101 are connected to a SPAD pixel in the SPAD detector array, and are used to accumulate the number of photon event pulse signals output by each SPAD pixel within a preset exposure time to obtain light intensity data and output a two-dimensional intensity image. ROI extraction circuit 102 is used to generate an adaptive threshold from a two-dimensional intensity image to convert the two-dimensional intensity image into a binary ROI image based on the adaptive threshold. The morphological filtering circuit 103 is used to perform filtering and ROI fusion operations on the binary ROI image to generate an ROI mask. The reconfigurable connection network 104 and the TOF extraction circuit 105 are used to dynamically establish electrical connection paths between SPAD pixels in the ROI mask and the corresponding TOF extraction circuit 105 according to the ROI mask. The TOF extraction circuit 105 is used to obtain the TOF results of SPAD pixels in the ROI mask and output a sparse three-dimensional image.

[0018] like Figure 1 As shown, the system 100 includes multiple photon counting circuits 101, a ROI extraction circuit 102, a morphological filtering circuit 103, a reconfigurable connection network 104, and a TOF extraction circuit 105. The multiple photon counting circuits 101 are directly connected to each SPAD pixel in the SPAD detector array, and are used to accumulate and count the photon event pulse signals output by each SPAD pixel within a preset exposure time to obtain the light intensity data of each SPAD pixel. The light intensity data of all SPAD pixels are arranged according to their spatial positions in the SPAD detector array to form a complete two-dimensional intensity image. The ROI extraction circuit 102 receives the two-dimensional intensity image output by the photon counting circuit 101, uses it to generate an adaptive threshold, and converts the two-dimensional intensity image into a binary ROI image through a binary comparison operation. Furthermore, the morphological filtering circuit 103 eliminates noise interference in the ROI binary image through morphological operations such as erosion and dilation, improving the connectivity and boundary quality of the ROI region and generating a high-quality ROI mask. The reconfigurable connection network 104 dynamically establishes connection paths between SPAD pixels and the TOF extraction circuit 105 based on the ROI mask. Specifically, the reconfigurable connection network 104 traverses and scans the effective area in the ROI mask, establishing measurement connections only for SPAD pixels falling within the ROI mask. The TOF extraction circuit 105 then performs TOF measurements on the SPAD pixels selected by the reconfigurable connection network 104 and converts the data into depth information. Finally, the SPAD sparse 3D ranging system 100 outputs only the 3D coordinate information of the SPAD pixels within the ROI region, forming sparse 3D point cloud data, significantly reducing data redundancy.

[0019] For example, if the SPAD detector array is 320×240, then there are 320×240 photon counting circuits 101. After accumulating and counting the photon event pulse signals output by the SPAD pixels, a two-dimensional intensity image with a resolution of 320×240 is obtained. The photon counting circuit 101 includes an integration interval control circuit and an accumulator counter; wherein, The integration interval control circuit is used to receive the start integration signal and integration duration register configuration value sent by the controller, and to enable the integration interval signal when the start integration signal is detected and to stop enabling the integration interval signal after the preset exposure time is reached. The accumulator counter contains an N-bit counter. The reset and enable terminals of the counter are both connected to the integration interval control circuit. The accumulator counter is used to accumulate and count the photon event pulse signals output by the corresponding SPAD pixel within a preset exposure time and output light intensity data.

[0020] Specifically, upon detecting a valid start-integration signal, the integration interval control circuit immediately enables the integration interval signal, initiating the photon counting process. The enabled state of the integration interval signal is maintained until the preset exposure time is reached, at which point it stops being enabled, thus precisely controlling the integration duration for each measurement. This register-based exposure time control method allows the system 100 to flexibly adjust integration parameters according to the lighting conditions and measurement requirements of different application scenarios. The accumulator counter uses an N-bit binary counter to precisely accumulate photon event pulse signals. Both the reset and enable terminals of the counter are connected to the integration interval control circuit, forming a complete control loop. When the integration interval control circuit enables the integration interval signal, the accumulator counter begins counting the photon event pulse signals output by the corresponding SPAD pixel; when the integration interval signal is de-enabled, the counting process terminates synchronously, ensuring that photon counting for each SPAD pixel is completed within a unified exposure time window, guaranteeing the temporal consistency of light intensity data between different pixels.

[0021] ROI extraction circuit 102 includes: an adaptive threshold generation circuit and a binary comparison circuit. The adaptive threshold generation circuit includes: an intensity histogram circuit and a valley detection circuit; wherein, The intensity histogram circuit includes a depth of The RAM storage unit is used to take the intensity value of each pixel in the two-dimensional intensity image as the address input, and by incrementing the stored value at the corresponding address, the frequency of occurrence of each intensity value in the two-dimensional intensity image is obtained to generate an intensity histogram; where, ; The valley detection circuit is used to sequentially read the stored values ​​in the RAM storage cell, perform smoothing filtering on a group of consecutive preset number of stored values, locate the local minimum value in each group of stored values, and when the first gradient corresponding to the location of the local minimum value is 0, the local minimum value is determined as a valley. At least one adaptive threshold is determined based on the intensity value corresponding to all valley locations in the intensity histogram. The binary comparison circuit is used to input each adaptive threshold and the two-dimensional intensity image into the comparator respectively. By comparing each intensity data with each adaptive threshold, it outputs a 1-bit binary result to obtain multiple ROI binary images.

[0022] Specifically, the ROI extraction circuit 102 is used to extract the region of interest (ROI) from the two-dimensional intensity image. The intensity histogram circuit uses a RAM storage unit of depth M, wherein... N represents the bit width of the intensity data. The intensity histogram circuit takes the intensity value of each pixel in the two-dimensional intensity image as an address input, increments the stored value at the corresponding address by 1, and counts the frequency of occurrence of each intensity value to generate an intensity histogram. The trough detection circuit sequentially reads the stored values ​​in the RAM storage unit, performs smoothing filtering on groups of consecutive stored values, detects the local minimum value in each group of stored values, and, combined with the condition that the first-order gradient is zero, locates the trough position in the histogram. The intensity value corresponding to each trough position is the adaptive threshold.

[0023] The binary comparison circuit receives the adaptive thresholds generated by the valley detection circuit and compares each adaptive threshold with the intensity data in the two-dimensional intensity image. In other words, for each adaptive threshold, a corresponding ROI binary image is generated. Pixels with intensity values ​​greater than or equal to the adaptive threshold are marked as logic high, indicating they belong to the region of interest, while pixels with intensity values ​​lower than the threshold are marked as logic low, indicating the background region.

[0024] The morphological filter circuit 103 includes: a fill shift circuit, a morphological manipulation circuit, and a ROI fusion circuit; wherein, The fill-shift circuit is used to fill the edges of the ROI binary image according to the type of morphological operation, and then input the filled ROI binary image into the morphological operation circuit in columns. The morphological operation circuit is used to perform multi-level delayed storage on the filled ROI binary image and perform morphological operations under the constraints of the structuring element to output the morphologically filtered ROI binary image. The ROI fusion circuit performs three erosion operations on each morphologically filtered binary ROI image, subtracts the result from the morphologically filtered binary ROI image to obtain multiple ROI gradient images, and then superimposes all the ROI gradient images to generate an ROI mask.

[0025] In this embodiment, the fill-shift circuit performs edge filling processing on the input ROI binary image according to the morphological operations to be performed by the SPAD sparse 3D ranging system 100. The morphological operations include erosion and dilation. For example, when the SPAD sparse 3D ranging system 100 performs an erosion operation, the fill circuit fills a column of 1s on both sides of the ROI binary image to ensure that the edge pixels on both sides of the image can participate in the morphological operation without losing effective information. When performing a dilation operation, a column of 0s is filled on both sides of the ROI binary image to avoid boundary effects. After the filling is completed, the filled ROI binary image is input into the morphological operation circuit column by column.

[0026] Figure 3 This is a schematic diagram of the morphological manipulation circuit provided in an embodiment of the present invention. Figure 3 As shown, the morphological manipulation circuit includes a flip-flop array and a logic operation network. Taking a 320×240 SPAD detector array as an example, the size of the flip-flop array is 320×3. Driven by the clock, the flip-flop array performs a three-level delay operation on the input column data (320 bits of data) through three levels of flip-flops. Simultaneously, the solidified logic operation completes the morphological operation and outputs the operation result (320 bits of data). The logic operation is performed under the constraint of a structuring element (such as a cross). Specifically, each structuring element includes erosion operations (AND operation), dilation operations (OR operation), and negation operations on the data within the structuring element. The output is achieved by outputting the corresponding result through a selector, thus realizing a specific morphological operation. By sequentially performing erosion, dilation, and other operations on the filled ROI binary image, combined morphological filtering can be achieved, effectively removing noise from the ROI binary image.

[0027] The ROI fusion circuit first performs three consecutive erosion operations on each morphologically filtered ROI binary image, then subtracts it from the original morphologically filtered ROI binary image to obtain multiple corresponding ROI gradient images, highlighting the boundary features of the ROI region. Subsequently, all ROI gradient images are bitwise ORed to fuse the regions of interest extracted based on different adaptive thresholds into a single ROI mask. This morphological gradient-based fusion method effectively preserves the boundary information of all target regions while eliminating unnecessary ranging areas, ensuring the accuracy and integrity of the ROI mask.

[0028] Figure 4 This is a schematic diagram of the reconfigurable connection network provided in an embodiment of the present invention. Figure 5 This is a schematic diagram illustrating the process of reconfigurable connectivity and Time-of-Flight (TOF) extraction provided in an embodiment of the present invention. Optionally, please refer to... Figures 4-5 The SPAD sparse 3D ranging system 100 based on near-pixel end sensing includes A TOF extraction circuit 105 and a reconfigurable connection network 104 include a connection network and Each traversal unit This indicates the larger of the number of rows and columns of SPAD pixels in the SPAD detector array. The working states of the traversal unit include: traversal scanning, pixel locking, and scanning end. The reconfigurable connection network 104 includes a horizontal boundary TOF extraction stage and a vertical boundary TOF extraction stage. In the horizontal boundary TOF extraction stage, the traversal unit is used to traverse the corresponding row in the ROI mask in the traversal scan state and search for macro pixels that match the boundary features in the row; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scan end state, it enables its own Flag1 and Flag2 flag signals to indicate that the traversal is complete; after all traversal units have completed the traversal, the vertical boundary TOF extraction stage begins. In the vertical boundary TOF extraction stage, the traversal unit is used to traverse the corresponding column in the ROI mask in the traversal scan state and search for macro pixels that match the boundary features in the column; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scan end state, it enables its own Flag1 and Flag2 flag signals to indicate the end of traversal and complete the measurement of the current frame. The connection network is used to establish a connection path between the locked horizontal boundary macro pixels and the corresponding TOF extraction circuit 105 when the Flag1 flag signal of all traversal units is enabled during the horizontal boundary TOF extraction stage; and to establish a connection path between the locked vertical boundary macro pixels and the corresponding TOF extraction circuit 105 when the Flag1 flag signal of all traversal units is enabled during the vertical boundary TOF extraction stage.

[0029] In addition, the connection network is also used in the horizontal boundary TOF extraction stage. After obtaining the TOF result, it resets the Flag1 flag signal of the traversal unit that has not completed row traversal, so that the traversal unit that has not completed row traversal continues to search for rows until the Flag2 flag signal of all traversal units is enabled, thus completing the horizontal boundary TOF extraction. In the vertical boundary TOF extraction stage, after obtaining the TOF result, it resets the Flag1 flag signal of the traversal unit that has not completed column traversal, so that the traversal unit that has not completed column traversal continues to search for columns until the Flag2 flag signal of all traversal units is enabled, thus completing the vertical boundary TOF extraction.

[0030] In this embodiment, the reconfigurable connectivity network 104 adopts a phased processing approach, including a horizontal boundary TOF extraction phase and a vertical boundary TOF extraction phase. During the horizontal boundary TOF extraction phase, the SPAD sparse 3D ranging system 100 is activated. Each traversal unit This indicates the row number of SPAD pixels in the SPAD detector array. Each traversal unit is responsible for traversing and retrieving the corresponding row in the ROI mask, and each traversal unit can enable two flag signals, Flag1 and Flag2, to indicate its own working state. The traversal unit has three working states: traversal scanning state, pixel locking state, and scan end state. When in traversal scanning state, both Flag1 and Flag2 flag signals are disabled. The traversal unit scans bit by bit from the corresponding row of the ROI mask data, searching for macropixels in the row that match the boundary features. It should be noted that boundary features are usually defined as specific pixel patterns, such as boundary feature values ​​like 0111 or 1110, used to identify the edge regions of the target object. When the traversal unit detects a macropixel matching the boundary features during the scan, it immediately switches to pixel locking state. In this state, the traversal unit enables its own Flag1 flag signal, locks the currently detected macropixel position information, and stops traversing backward. The connection network dynamically establishes an electrical connection path between the locked macropixel and the corresponding TOF extraction circuit 105 and completes TOF extraction. After the TOF extraction is completed, the enabled Flag1 flag signal returns to the disabled state, the traversal unit switches back to the traversal scanning state and continues to traverse from the locked macro pixel position. When a row of data is traversed to the last bit, the traversal unit enters the scan end state. At this time, both Flag1 and Flag2 flag signals are enabled to indicate that the traversal retrieval work of that row has been completed.

[0031] The connection network is connected to each traversal unit. The operating state of each traversal unit is determined based on a 2-bit flag output from each unit, and these flags are used to control the reconfigurable connection between the macropixel and the TOF extraction circuit 105, as well as the TOF extraction process. For example... Figure 5 As shown, when the connection network detects that Flag1 of all traversal units is enabled, it indicates that all traversal units have detected macropixels that meet the boundary features or have completed the traversal search. At this time, the connection network enables the macropixels locked by all traversal units, and their outputs are transmitted to the TOF extraction circuit 105 through the data bus, completing the establishment of a dynamic connection path from the macropixel to the TOF extraction circuit 105. After the electrical path is established, the TOF extraction process begins and the TOF result is output to the corresponding location in the RAM. Subsequently, the connection network resets the Flag1 flag signal of the traversal unit, allowing it to continue traversing (except for traversal units that have completed traversal) until the Flag1 flag signal is enabled again, and the above process is repeated.

[0032] It should be noted that the scanning ends at different times for different traversal units. Therefore, the vertical boundary TOF extraction stage should only begin after all traversal units have completed their traversal (i.e., all Flag1 and Flag2 flag signals of all traversal units are enabled) and entered the scanning end state.

[0033] The vertical boundary TOF extraction stage is similar to the horizontal boundary TOF extraction stage, but the SPAD sparse 3D ranging system 100 is enabled. Each traversal unit This indicates the column number of SPAD pixels in the SPAD detector array. The traversal direction is changed to column-based, with each traversal unit responsible for data retrieval in its corresponding column. Similarly, the traversal unit completes the search and locking of boundary features in the column direction by switching between three states, and the connection network establishes the connection path between the vertical boundary macropixel and the TOF extraction circuit 105 based on the flag signal state.

[0034] Furthermore, the TOF extraction circuit 105 includes: a sampling circuit and a histogram circuit; wherein, The sampling circuit is used to synchronously sample the photon event pulse signal output by the SPAD pixel selected by the reconfigurable connection network 104 in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage, respectively, to obtain the quantized timestamp. The histogram circuit is used to accumulate histograms of the photon arrival time distribution of macropixels locked in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage according to the quantization timestamp. The TOF result is obtained through peak detection. After the TOF result is converted into a depth value, the sparse 3D image of ROI mask mapping is output.

[0035] In this embodiment, the sampling circuit samples the photon event pulse signals transmitted by the SPAD via the reconfigurable connection network 104, detects valid photon events, outputs quantized timestamps, and then transmits them to the histogram circuit. The histogram circuit constructs a histogram of photon arrival time distribution by statistically counting the timestamps of a large number of photon event pulse signals. It receives quantized timestamp data from the sampling circuit; the quantized timestamps record the precise time delay of each detected photon event relative to the laser pulse emission time. The histogram circuit contains multiple counting channels, each corresponding to a specific time unit. When the received quantized timestamp falls within a certain time unit range, the count value of the corresponding channel increases. Through this statistical method, the SPAD sparse 3D ranging system 100 can obtain the frequency distribution of photon events within different time units. The generated histogram directly reflects the probability distribution characteristics of photon arrival time, and finally, high-precision TOF results are obtained through peak detection.

[0036] Figure 6 This is a schematic flowchart of a SPAD sparse 3D ranging method based on near-pixel end sensing provided in an embodiment of the present invention. Figure 6As shown, this embodiment of the invention also provides a SPAD-based sparsified 3D ranging method based on near-pixel end sensing, applied to the above-mentioned system 100. The method includes: S1. A two-dimensional intensity image is generated using a SPAD detector array.

[0037] In this embodiment, the application scenario involves a highly reflective rectangular object against a relatively dark background, and the SPAD detector array has a size of 320×240. In step S1, the SPAD detector array receives ambient light signals reflected from the scene. The integration interval control circuit in the photon counting circuit 101 generates a precise integration time window. During this period, the accumulator counter corresponding to each SPAD pixel counts the received photon event pulses in real time. After the integration time ends, the values ​​of all 76,800 counters are read out synchronously, forming a complete two-dimensional intensity image in a spatial arrangement of 320×240. In this two-dimensional intensity image, the area where the highly reflective rectangular object is located exhibits a significantly higher intensity characteristic than the background area.

[0038] S2. Generate a binary ROI image based on the two-dimensional intensity image.

[0039] After receiving the two-dimensional intensity image, the ROI extraction circuit 102 first uses an intensity histogram circuit to statistically analyze the intensity distribution of the entire image. In the aforementioned application scenario, this results in a histogram with a distinct bimodal characteristic, where one peak corresponds to a low-intensity background region and the other peak corresponds to a high-intensity rectangular object region. Next, the trough detection circuit locates the lowest point between the two peaks and determines the intensity value corresponding to that point as an adaptive threshold. For example, when the intensity value corresponding to the trough is 150, the binary comparison circuit compares the intensity value of each pixel in the two-dimensional intensity image with the adaptive threshold of 150: pixels with an intensity value greater than or equal to 150 output logic 1 and are identified as object regions; pixels with an intensity value less than the adaptive threshold of 150 output logic 0 and are identified as background regions. This process generates a preliminary binary ROI image, roughly outlining the contour region of the rectangular object. However, due to sensor noise and edge gradation, the contour region may contain jagged edges and discontinuities.

[0040] S3. Perform morphological filtering on the binary image of the ROI to obtain the ROI mask.

[0041] Specifically, the fill-shift circuit performs edge filling processing on the ROI binary image, and then inputs the image data column by column into the morphological operation circuit. The morphological operation circuit first performs an erosion operation, using a cross-shaped structuring element to eliminate isolated noise points and smooth object boundaries, causing the contour to shrink appropriately inward. Then, a dilation operation is performed, using the same structuring element to restore the contour to its original size, while filling in small holes and breaks in the region. After the "erosion-dilation" opening operation, the discrete noise in the background region is effectively suppressed. Subsequently, the "dilation-erosion" closing operation is performed to eliminate discrete noise points in the object region, obtaining a rectangular region with smooth edges and good connectivity. Finally, the ROI fusion circuit calculates the morphological gradient of the morphologically filtered ROI binary image, subtracts the morphologically filtered ROI binary image from the result of the three erosion processes, and generates a precise object boundary line with a width of only three pixels, obtaining a high-quality ROI mask.

[0042] S4. Establish reconstructable connections for SPAD pixels based on the ROI mask, extract TOF from the selected SPAD pixels, further convert the TOF results into depth values, and output the sparsed 3D image mapped by the ROI mask. The reconfigurable connection network 104 performs a traversal scan of the aforementioned ROI mask. During the horizontal boundary TOF extraction stage, each traversal unit operates row by row, identifying the positions of macropixels in the mask that conform to boundary features, such as boundary patterns with values ​​of 0111 and 1110. Once the traversal unit has locked onto the specific position of the ROI macropixel in its row, the connection network dynamically establishes an electrical connection path from the output of the corresponding SPAD pixel to the input of the TOF extraction circuit 105.

[0043] After completing the horizontal boundary TOF extraction, the vertical boundary TOF extraction stage begins. In this stage, each traversal unit operates column-wise, scanning the ROI mask along the column direction. Each traversal unit is responsible for retrieving data for its corresponding column, identifying the macro-pixel positions in the mask that match the column-direction boundary features. Similar to row traversal, column traversal also employs a specific boundary pattern recognition algorithm to detect target boundary features along the column direction. Once the traversal unit has locked onto the specific location of the ROI macro-pixel during column scanning, the connection network dynamically establishes an electrical connection path from the corresponding SPAD pixel to the TOF extraction circuit 105.

[0044] During this process, whether in the horizontal or vertical traversal stage, the pixels that are not locked in the mask remain disconnected from the TOF extraction circuit 105, realizing a fundamental shift from the traditional full array readout mode to the ROI-driven sparse gating mode.

[0045] In the horizontal boundary TOF extraction stage, the sampling circuit synchronously samples the output pulse of each pixel selected through row traversal, generating quantized timestamp data. The histogram circuit independently accumulates histogram statistics of photon arrival times for each selected row-direction macro-pixel. Similarly, in the vertical boundary TOF extraction stage, the sampling circuit synchronously samples the pixels selected through column traversal, and the histogram circuit independently accumulates the photon arrival time distribution for each selected column-direction ROI macro-pixel.

[0046] After accumulating a sufficient number of laser pulses, the peak positions of the histograms of macropixels in the row and column directions are determined using a peak detection algorithm. The accurate Time-of-Flight (TOF) results are then calculated and converted into corresponding depth values. Finally, a sparse 3D image is output. This image is not a complete 320×240 depth matrix, but rather a dataset containing boundary feature point clouds in both row and column directions, clearly outlining the boundary shape and depth information of the rectangular object in 3D space.

[0047] As can be seen from the above embodiments, the beneficial effects of the present invention are as follows: The SPAD sparse 3D ranging system based on near-pixel end sensing provided by this invention includes: a photon counting circuit for generating a 2D intensity image, a Region of Interest (ROI) extraction circuit for extracting the region of interest from the 2D intensity image, a morphological filtering circuit for generating an ROI mask, a reconfigurable connection network for establishing physical connections between ROI macropixels and the Time-of-Flight (TOF) extraction circuit, and a TOF extraction circuit for calculating the distance to the target object. Compared with existing technologies, this invention proposes an ROI extraction mechanism based on light intensity distribution, based on a "sensing first, measuring later" strategy, and performs TOF extraction only on macropixels within the ROI region, thereby outputting a sparse 3D image of the ROI mapping. This reduces the number of spatial scans required for a single frame image and solves the data redundancy problem caused by the output of full-resolution 3D images in traditional SPAD LiDAR, significantly improving the system's real-time performance. Furthermore, since this invention obtains the ROI by analyzing the 2D intensity image, it can accommodate measurements of both dynamic and static scenes.

[0048] In the description of this invention, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0049] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A SPAD-based sparse 3D ranging system based on near-pixel end sensing, characterized in that, include: Multiple photon counting circuits are connected to a SPAD pixel in the SPAD detector array to accumulate the number of photon event pulse signals output by each SPAD pixel within a preset exposure time, obtain light intensity data, and output a two-dimensional intensity image. A ROI extraction circuit is used to generate an adaptive threshold from the two-dimensional intensity image, so as to convert the two-dimensional intensity image into a binary ROI image according to the adaptive threshold. A morphological filtering circuit is used to perform filtering and ROI fusion operations on the binary ROI image to generate an ROI mask. A reconfigurable connection network and a TOF extraction circuit are provided. The reconfigurable connection network is used to dynamically establish electrical connection paths between SPAD pixels in the ROI mask and the corresponding TOF extraction circuit according to the ROI mask. The TOF extraction circuit is used to obtain the TOF results of SPAD pixels in the ROI mask and output a sparsed 3D image.

2. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 1, characterized in that, The photon counting circuit includes an integration interval control circuit and an accumulator counter; wherein... The integration interval control circuit is used to receive the start integration signal and integration duration register configuration value sent by the controller, and to enable the integration interval signal when the start integration signal is detected and to stop enabling the integration interval signal after the preset exposure time is reached. The accumulator counter includes an N-bit counter. The reset and enable terminals of the counter are both connected to the integration interval control circuit. The accumulator counter is used to accumulate and count the photon event pulse signals output by the corresponding SPAD pixel within a preset exposure time and output light intensity data.

3. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 1, characterized in that, The ROI extraction circuit includes an adaptive threshold generation circuit and a binary comparison circuit. The adaptive threshold generation circuit includes an intensity histogram circuit and a valley detection circuit. The intensity histogram circuit includes a depth of The RAM storage unit is used to take the intensity value of each pixel in the two-dimensional intensity image as an address input, and by incrementing the stored value at the corresponding address, the frequency of occurrence of each intensity value in the two-dimensional intensity image is obtained to generate an intensity histogram; wherein, ; The valley detection circuit is used to sequentially read the stored values ​​in the RAM storage unit, perform smoothing filtering processing on a group of consecutive preset number of stored values, locate the local minimum value in each group of stored values, and when the first gradient corresponding to the location of the local minimum value is 0, the local minimum value is determined as a valley. At least one adaptive threshold is determined based on the intensity values ​​corresponding to all valley locations in the intensity histogram. The binary comparison circuit is used to input each adaptive threshold and the two-dimensional intensity image into the comparator respectively. By comparing each intensity data with each adaptive threshold, it outputs a 1-bit binary result to obtain multiple ROI binary images.

4. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 1, characterized in that, The morphological filtering circuit includes: a fill-shift circuit, a morphological manipulation circuit, and a ROI fusion circuit; wherein... The fill-shift circuit is used to perform edge filling processing on the ROI binary image according to the type of morphological operation, and then input the filled ROI binary image into the morphological operation circuit in columns. The morphological operation circuit is used to perform multi-level delayed storage on the filled ROI binary image, and to perform morphological operations under the constraints of structuring elements to output the morphologically filtered ROI binary image. The ROI fusion circuit is used to perform three erosion operations on each morphologically filtered ROI binary image, subtract the calculation result from the morphologically filtered ROI binary image to obtain multiple ROI gradient images, and further superimpose all ROI gradient images to generate an ROI mask.

5. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 1, characterized in that, include A TOF extraction circuit, the reconfigurable connection network includes a connection network and Each traversal unit This indicates the larger of the number of rows and columns of SPAD pixels in the SPAD detector array. The working states of the traversal unit include: traversal scanning, pixel locking, and scanning end. The reconfigurable connection network includes a horizontal boundary TOF extraction stage and a vertical boundary TOF extraction stage. In the horizontal boundary TOF extraction stage, the traversal unit is used to traverse the corresponding row in the ROI mask in the traversal scanning state and search for macro pixels that match the boundary features in the row; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scan end state, it enables its own Flag1 and Flag2 flag signals to indicate that the traversal is complete; after all traversal units have completed the traversal, the vertical boundary TOF extraction stage begins. In the vertical boundary TOF extraction stage, the traversal unit is used to traverse the corresponding column in the ROI mask in the traversal scanning state and search for macro pixels that match the boundary features in the column; in the pixel locking state, it enables its own Flag1 flag signal and locks the current macro pixel position; in the scanning end state, it enables its own Flag1 and Flag2 flag signals to indicate the end of traversal and complete the measurement of the current frame. The connection network is used to establish a connection path between the locked horizontal boundary macropixel and the corresponding TOF extraction circuit when the Flag1 flag signal of all traversal units is enabled during the horizontal boundary TOF extraction stage; and to establish a connection path between the locked vertical boundary macropixel and the corresponding TOF extraction circuit when the Flag1 flag signal of all traversal units is enabled during the vertical boundary TOF extraction stage.

6. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 5, characterized in that, The TOF extraction circuit includes: a sampling circuit and a histogram circuit; wherein... The sampling circuit is used to synchronously sample the photon event pulse signal output by the SPAD pixel selected by the reconfigurable connection network in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage, respectively, to obtain the quantized timestamp. The histogram circuit is used to accumulate histograms of the photon arrival time distribution of macropixels locked in the horizontal boundary TOF extraction stage and the vertical boundary TOF extraction stage according to the quantization timestamp, obtain the TOF result through peak detection, and further convert the TOF result into a depth value to output a sparse 3D image of ROI mask mapping.

7. The SPAD sparse 3D ranging system based on near-pixel end sensing according to claim 6, characterized in that, The connection network is also used in the horizontal boundary TOF extraction stage to reset the Flag1 flag signal of the traversal unit that has not completed row traversal after obtaining the TOF result, so that the traversal unit that has not completed row traversal continues to search for rows until the Flag2 flag signal of all traversal units is enabled, thus completing the horizontal boundary TOF extraction; and in the vertical boundary TOF extraction stage, to reset the Flag1 flag signal of the traversal unit that has not completed column traversal after obtaining the TOF result, so that the traversal unit that has not completed column traversal continues to search for columns until the Flag2 flag signal of all traversal units is enabled, thus completing the vertical boundary TOF extraction.

8. A SPAD-based sparse 3D ranging method based on near-pixel end sensing, characterized in that, Applied to the system described in any one of claims 1 to 7.

Images