SPAD imaging data compression system suitable for extremely low illumination

Abstract

The application discloses a kind of SPAD imaging data compression systems suitable for extremely low illumination, comprising: data rearrangement module, for arranging the one-dimensional discrete data corresponding to a plurality of pixels received by SPAD array as multiple rows of data, and inputting the multiple rows of data corresponding to each pixel to pulsatile calculation module;Pulsatile calculation module includes a plurality of parallel pulsatile arrays and addition unit, addition unit and each pulsatile array are used to carry out wavelet decomposition calculation to the multiple rows of data of corresponding pixel, obtain the wavelet decomposition result of the multiple rows of data of each pixel, and the wavelet decomposition result of all pixels in the fixed neighborhood range of each pixel is summed to obtain the summation result of each pixel;Sliding window intercepting module is used for sliding window intercepting operation to the summation result of each pixel, to obtain the compression data of each pixel.The application can reduce the processing time of SPAD hardware system, improve the processing efficiency of SPAD hardware system.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a SPAD imaging data compression system suitable for extremely low illumination. Background Technology

[0002] Currently, in high-resolution, high-frame-rate applications, the amount of data generated per second by large-scale single-photon avalanche diode (SPAD) arrays is too large, even exceeding the bandwidth of advanced data buses such as Universal Serial Bus (USB) by an order of magnitude. Therefore, in order to solve this bottleneck, wavelet transform is usually used to compress the data collected by the SPAD array.

[0003] However, traditional wavelet transform implementation methods usually rely on multi-stage filter architectures. After the multi-stage filters perform convolution operations, downsampling operations are usually performed, which leads to a large amount of useless computation. These redundant operations not only waste a lot of computing resources of the SPAD hardware system, but also significantly increase the processing time of the SPAD hardware system and reduce the processing efficiency of the SPAD hardware system.

[0004] Therefore, how to reduce the processing time of SPAD hardware systems is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] To address the problem of reducing the processing time of SPAD hardware systems, this invention provides a SPAD imaging data compression system suitable for extremely low-light conditions. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a SPAD imaging data compression system suitable for extremely low illumination, comprising:

[0007] The data rearrangement module is used to arrange the one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multiple rows of data, and input the multiple rows of data corresponding to each pixel into the pulsation calculation module.

[0008] The pulsation calculation module includes multiple parallel pulsation arrays and addition units. The addition units and each pulsation array are used to perform wavelet decomposition calculation on the multi-row data of the corresponding pixel to obtain the wavelet decomposition result of the multi-row data of each pixel, and to sum the wavelet decomposition results of all pixels in the fixed neighborhood of each pixel to obtain the summation result of each pixel.

[0009] The sliding window cropping module is used to perform a sliding window cropping operation on the summation result of each pixel to obtain compressed data for each pixel.

[0010] In one embodiment of the present invention, the data rearrangement module includes a multi-level register and a finite state machine;

[0011] The multi-level register is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, and introduce delays in the order of rows so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the systolic array.

[0012] A finite state machine is used to pad with zeros between the one-dimensional discrete data corresponding to adjacent pixels, so that the output of wavelet decomposition calculation for each systolic array is not affected by adjacent pixels.

[0013] In one embodiment of the present invention, the multi-level register includes a first-level register and a plurality of second-level registers;

[0014] The first-level register is used to sequentially read m discrete data points from the one-dimensional discrete data corresponding to the first pixel in the block random access memory in each clock cycle, and arrange the m discrete data points into m rows. After arranging all the one-dimensional discrete data points corresponding to the first pixel into m rows, it reads m discrete data points from the one-dimensional discrete data corresponding to the next pixel in the block random access memory in each clock cycle, and arranges the m discrete data points into m rows, until all the one-dimensional discrete data points corresponding to each pixel are arranged into m rows, where m is the same as the number of rows in the systolic array.

[0015] Multiple second-level registers are used to delay the data in the i-th row, such that each row of data is delayed by one clock cycle relative to the previous row of data, 1 < i ≤ m.

[0016] In one embodiment of the present invention, the finite state machine includes a first state, a second state, and a third state;

[0017] When the finite state machine is in the first state, it waits for the target signal to begin data arrangement in the first-level register. When the finite state machine receives the target signal, it switches from the first state to the second state. In the second state, it updates the address values ​​of m discrete data in the one-dimensional discrete data of each pixel, so that the first-level register reads discrete data from the block random access memory according to the address values. After each read of discrete data, the address value of the currently read discrete data in the first-level register is incremented by one. When the counter indicates that the number of reads by the first-level register is equal to the preset number, the finite state machine switches from the second state to the third state. In the third state, it performs zero-padding between the one-dimensional discrete data corresponding to adjacent pixels and returns to the second state after completing the zero-padding operation.

[0018] In one embodiment of the present invention, the pulsation calculation module further includes a weight storage unit, which is used to assign corresponding weight parameters to each pulsation array;

[0019] Wavelet decomposition is performed on multiple rows of data for each pixel using an addition unit and each pulsation array to obtain the wavelet decomposition results for each pixel's multiple rows of data, including:

[0020] Based on the addition unit, each pulsation array and the corresponding weight parameters, wavelet decomposition calculation is performed on the multi-row data of each pixel using matrix multiplication and a preset data flow method to obtain the wavelet decomposition result of the multi-row data of each pixel.

[0021] In one embodiment of the present invention, each pulsation array includes multiple array-distributed processing units, and the number of processing units is m×n;

[0022] Based on the addition unit, each pulsation array, and the corresponding weight parameters, wavelet decomposition is performed on multiple rows of data for each pixel using matrix multiplication and a preset data flow method. The wavelet decomposition results for each pixel's multiple rows of data are obtained, including:

[0023] First, based on the wavelet decomposition principle, the wavelet decomposition process is equivalent to multiplying a one-dimensional vector with a sparse matrix. The non-zero values ​​in each column of the sparse matrix are composed of wavelet low-frequency filter coefficients, and each non-zero value is the result of shifting the previous column down by two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to the corresponding systolic array, so that each systolic array adopts a data flow with fixed weights and performs an A×B+C operation once in each clock cycle. Here, A is the input data of each processing unit, and the input data is one row of data in the multi-row data of each pixel. B is the corresponding weight parameter, and C is the first part of the sum passed by the processing unit above each processing unit. The first part of the sum is the sum of the A×B+C operation performed by the processing unit above each processing unit in each clock cycle. Finally, the operation results of the processing unit in the last row of each systolic array are summed by the addition unit to obtain a value of the wavelet decomposition result of the multi-row data of each pixel.

[0024] In one embodiment of the present invention, the data stream method includes:

[0025] The input data for each processing unit is passed from left to right along the row direction and remains unchanged. The first part of each processing unit is passed from top to bottom along the column direction to perform the operation A×B+C.

[0026] In one embodiment of the present invention, each pulsation array includes a plurality of array-distributed processing units;

[0027] The wavelet decomposition results of all pixels within a fixed neighborhood of each pixel are summed using the addition unit and each systolic array to obtain the summation result for each pixel, including:

[0028] Centered on each pixel, a fixed neighborhood range is determined at a fixed distance for each pixel. Through each pulsation array, the wavelet decomposition results of all pixels within the fixed neighborhood range are summed sequentially according to a preset number of channels. Then, the calculation results output by the processing unit in the last row of each pulsation array are accumulated through the addition unit until the summation data of all channels is calculated to obtain the summation result of each pixel.

[0029] In one embodiment of the present invention, the number of processing units is a×a;

[0030] For each systolic array, the wavelet decomposition results of all pixels within a fixed neighborhood are summed according to a preset number of channels. Then, the calculation results output by the processing unit in the last row of each systolic array are accumulated by the addition unit until the summation data of all channels has been calculated to obtain the summation result of each pixel, including:

[0031] Each systolic array uses a fixed data flow method, performing an A+D operation once per clock cycle. A is the target input data of each processing unit in a preset channel, which is the wavelet decomposition result of summing each pixel in the preset number of channels. D is the second part of the sum passed by the processing unit above each processing unit, which is the sum of the A+D operation performed by the processing unit above each processing unit in each clock cycle. Then, the operation results output by the processing unit in the last row of each systolic array are summed by the addition unit until the summed data of all channels are calculated to obtain the summed result of each pixel.

[0032] In one embodiment of the present invention, the sliding window capture module includes a register chain, an adder tree, a comparator, and a counter;

[0033] The sliding window cropping module performs a sliding window cropping operation on the summed data of each pixel to obtain compressed data for each pixel, including:

[0034] The summation result of each pixel is sequentially input into the first register in the register chain. The data of each register in the register chain is passed to the subsequent register in each clock cycle. When all registers in the register chain are filled, the data in each register is input in parallel into the adder tree for summation. Then, the maximum value of the summation operation of each pixel is obtained through the comparator, and the target index of each maximum value is obtained through the counter and the index register in the register chain. Finally, the data in the address range corresponding to each target index is determined as the compressed data of each pixel and output.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention provides a SPAD imaging data compression system suitable for extremely low-light conditions. It includes a data rearrangement module for arranging one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multi-row data, a pulsation calculation module (i.e., a pulsation array and an addition unit) capable of performing wavelet decomposition calculations in parallel, and more. Compared to other hardware systems, the system provided by this invention upgrades the traditional serial processing architecture to a highly efficient parallel architecture, eliminating redundant calculations at the algorithm level and employing pulsation array multiplexing technology at the hardware level. This significantly shortens the computation cycle and complexity of the SPAD hardware system, thereby improving its processing efficiency.

[0037] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the structure of a SPAD imaging data compression system suitable for extremely low illumination provided in an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the state transition of a finite state machine provided in an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of a data flow based on wavelet decomposition using PE units, provided by an embodiment of the present invention. Detailed Implementation

[0041] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail a SPAD imaging data compression system suitable for extremely low illumination, in conjunction with the accompanying drawings and specific embodiments.

[0042] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0044] This invention addresses the problem of reducing the processing time of SPAD hardware systems by proposing a SPAD imaging data compression system suitable for extremely low-light conditions. Please refer to [link to relevant documentation]. Figure 1 The system 100 includes: multiple parallel data rearrangement modules 110, a pulsation calculation module 120, and a sliding window capture module 130. The data rearrangement module 110 is connected to the pulsation array 1201 in the pulsation calculation module 120, and the pulsation calculation module 120 is connected to the sliding window capture module 130.

[0045] The data rearrangement module 110 is used to arrange the one-dimensional discrete data corresponding to each pixel received by the SPAD array into multiple rows of data, and input the multiple rows of data corresponding to each pixel into the pulsating array.

[0046] For example, each pixel corresponds to 1×1024 discrete data. The data rearrangement module 110 can rearrange the 1×1024 discrete data into 8×128 discrete data (i.e., multi-row data). That is, the 1 row and 1024 column discrete data corresponding to each pixel is rearranged into 8 rows and 128 columns, and the 8 rows and 128 columns discrete data corresponding to each pixel are input into the pulsation array.

[0047] The pulsation calculation module 120 includes multiple parallel pulsation arrays 1201 and an addition unit 1202. The addition unit 1202 and each pulsation array 1201 are used to perform wavelet decomposition calculation on the multi-row data of the corresponding pixel to obtain the wavelet decomposition result of the multi-row data of each pixel, and to sum the wavelet decomposition results of all pixels in the fixed neighborhood of each pixel to obtain the summed data of each pixel.

[0048] The sliding window cropping module 130 is used to perform a sliding window cropping operation on the summation result of each pixel to obtain compressed data for each pixel.

[0049] The data rearrangement module 110 will be described in detail below.

[0050] Specifically, such as Figure 1 As shown, the data rearrangement module 110 includes a multi-level register 1101 and a finite state machine 1102.

[0051] Among them, the multi-level register 1101 is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, and introduce delays in the order of rows so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the pulsation array.

[0052] It should be noted that, in one embodiment of the present invention, the multi-level register 1101 includes a first-level register and multiple second-level registers.

[0053] The first-level register is used to sequentially read m discrete data points from the one-dimensional discrete data corresponding to each pixel in each clock cycle from the Block Random Access Memory (BRAM), and arrange the m discrete data points into m rows. After arranging all the one-dimensional discrete data points corresponding to the first pixel into m rows, a zero-reading operation is performed (the read address is changed to the address where zero is stored in the BRAM) for n cycles. Then, in each clock cycle, the m discrete data points from the one-dimensional discrete data corresponding to the next pixel are sequentially read from the Block Random Access Memory, and the m discrete data points are arranged into m rows. Then, the zero-reading operation is performed for n cycles, and then the next pixel is read, until all the one-dimensional discrete data points corresponding to each pixel are arranged into m rows. m is the same as the number of rows in the systolic array.

[0054] For example, when m equals 8, n equals 10, and each pixel corresponds to 1×1024 discrete data, the first-level register sequentially reads the 1st to 8th discrete data from the 1×1024 discrete data corresponding to the first pixel from the block random access memory in each clock cycle, and arranges these 8 discrete data into 8 rows. Then, it sequentially reads the 9th to 16th discrete data from the 1×1024 discrete data corresponding to the first pixel from the block random access memory, and arranges these 8 discrete data into 8 rows. This continues until all the 1×1024 discrete data corresponding to the first pixel are arranged into m rows. Then, it reads zeros from the BRAM for 10 cycles. Then, in each clock cycle, it sequentially reads the 1st to 8th discrete data from the 1×1024 discrete data corresponding to the next pixel from the block random access memory, and arranges these 8 discrete data into 8 rows. This continues until all the 1×1024 discrete data corresponding to each pixel are arranged into m rows.

[0055] Among them, multiple second-level registers are used to delay the data in the i-th row, so that each row of data is delayed by one clock cycle relative to the previous row of data, 1 < i ≤ m.

[0056] For example, there is one second-level register in the second row of data, which is used to delay the second row of data by one clock cycle relative to the first row of data; there are two second-level registers in the third row of data, which are used to delay the third row of data by one clock cycle relative to the second row of data; there are three second-level registers in the fourth row of data, which are used to delay the fourth row of data by one clock cycle relative to the third row of data; and so on. Seven second-level registers are configured in the eighth row of data, which are used to delay the eighth row of data by one clock cycle relative to the seventh row of data, and so on.

[0057] The finite state machine 1102 includes a first state (IDLE state), a second state (RD_DATA state), and a third state (RD_ZERO state), specifically, as follows: Figure 2 As shown, Figure 2This is a schematic diagram of the state transition of a finite state machine provided in an embodiment of the present invention. When the finite state machine is in the first state, it waits to receive a target signal for the first-level register to start data arrangement. When the finite state machine receives the target signal, it switches from the first state to the second state. When the finite state machine is in the second state, it updates the address value of the one-dimensional discrete data corresponding to each pixel, and increments the address value of the currently read discrete data in the first-level register by one after each time the discrete data is read. When the counter indicates that the number of reads of the first-level register is equal to the preset number, the finite state machine switches from the second state to the third state. When the finite state machine is in the third state, it performs zero-padding operation between the one-dimensional discrete data corresponding to adjacent pixels, and returns to the second state after completing the zero-padding operation.

[0058] After introducing the data rearrangement module 110, the pulse calculation module 120 will be introduced in detail.

[0059] Specifically, such as Figure 1 As shown, each systolic array 1201 of the systolic calculation module 120 includes multiple array-distributed processing element (PE) units 1203. The systolic calculation module 120 also includes a weight storage unit 1204, which is used to assign corresponding weight parameters to the PE units 1203 of each systolic array 1201. Specifically, the PE units 1203 in the first row of each systolic array 1201 can be connected to the weight storage unit 120 to vertically transmit the corresponding weight parameters assigned by the weight storage unit 120 to all PE units 1203 in the systolic array 1201. It should be noted that... Figure 1 The PE units 1203 distributed in the pulsating array 1201 are merely an exemplary demonstration, and no specific limit is made on the number of PE units 1204 in the pulsating array 1201.

[0060] In one embodiment of the present invention, wavelet decomposition calculation is performed on multiple rows of data corresponding to a pixel by the addition unit 1202 and each pulse array 1201 to obtain the wavelet decomposition result of the multiple rows of data for each pixel, including:

[0061] Based on the addition unit 1202, each pulsation array 1201 and the corresponding weight parameters, wavelet decomposition calculation is performed on the multi-row data of each pixel using matrix multiplication and a preset data flow method to obtain the wavelet decomposition result of the multi-row data of each pixel.

[0062] Specifically, each systolic array 1201 includes m×n PE units 1203. The weight storage unit 1204 first uses the wavelet decomposition principle to convert the one-dimensional discrete wavelet decomposition process into the multiplication of a one-dimensional vector (i.e., one-dimensional discrete data) with a sparse matrix. Each column of the sparse matrix consists of filter coefficients, and each column of non-zero values ​​is the result of shifting the previous column down by two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to the corresponding PE unit 1203 of the systolic array 1201, so that each PE unit 1203 of the systolic array 1201 adopts a data flow method with fixed weights and performs an A×B+C operation once in each clock cycle. Wherein, A is the input data of each PE unit 1203, and the input data is one row of data in the multi-row data of each pixel; B is the corresponding weight parameter; C is the first part sum passed by the PE unit 1203 above each PE unit 1203, and the first part sum is the sum of the operation A×B+C performed by the PE unit 1203 above each clock cycle; finally, the operation results of the PE unit 1203 in the last row of each pulsating array 1201 are summed by the addition unit 1202 to obtain the wavelet decomposition result of the multi-row data of each pixel.

[0063] For example, such as Figure 3 As shown, Figure 3 This is a schematic diagram of a data flow based on wavelet decomposition using PE units, provided by an embodiment of the present invention. Figure 3 As shown, the data on the left (1, 2...m...m+1) represents the m rows of data for each pixel. Each row of data is delayed by one cycle compared to the previous row. Each pulsation array 1201 includes m×n PE units 1203 (it should be noted that...). Figure 3 This is merely an example of a systolic array 1201 within the systolic calculation module 120. Each systolic array 1201 processes m rows of data corresponding to a pixel. Multiple systolic arrays process m rows of data from multiple pixels in parallel. Each PE unit 1203 performs an A×B+C operation once per clock cycle. Here, W11~Wmn on each PE unit 1203 represent the weight parameters of each PE unit 1203 within each systolic array 1201.

[0064] Specifically, in the aforementioned data flow method, among the multiple array-distributed PE units 1203, the input data of each PE unit 1203 is passed from left to right along the row direction and remains unchanged, while the first part of each PE unit 1203 is passed from top to bottom along the column direction for the operation A×B+C. For example, for any row of any systolic array 1201, the input data of the i-th column in the current period will be moved to the PE unit 1203 of the (i+1)-th column in the next period, and for any column of any systolic array 1201, the first part of the sum of the i-th row in the current period will be moved to the PE unit 1203 of the (i+1)-th row in the next period for the operation A×B+C.

[0065] In one embodiment of the present invention, the wavelet decomposition results of all pixels within a fixed neighborhood range of each pixel are summed by the addition unit 1202 and each pulsation array 1201 to obtain the summed data of each pixel, including:

[0066] Centered on each pixel, a fixed neighborhood range is determined at a fixed distance for each pixel. The wavelet decomposition results of all pixels within the fixed neighborhood range are summed sequentially according to a preset number of channels through the PE unit 1203 of each systolic array 1201. Then, the operation results output by the PE unit 1203 in the last row of each systolic array 1201, i.e., the partial sum, are accumulated through the addition unit 1202 until the summation data of all channels is calculated to obtain the summation data of each pixel.

[0067] Specifically, each pulsation array 1201 includes a×a PE units 1203. Each PE unit 1203 adopts a fixed data flow method, performing an A+D operation once per clock cycle. A is the target input data of each PE unit 1203 in a preset channel. The target input data is the wavelet decomposition result of summing each pixel in the preset channel. This wavelet decomposition result can form a two-dimensional data. The first batch of target input data is the wavelet decomposition result of summing all pixels in the preset channel from row 1 to row a. The second batch of target input data is the wavelet decomposition result of summing all pixels in the preset channel from row 2 to row a+1. The wavelet decomposition results of the summation in the channel are calculated until all the summation data of all channels are calculated. D is the second part of the sum passed to the PE unit 1203 above each PE unit 1203. The second part of the sum is the sum of the A+C operation performed by the PE unit 1203 above each clock cycle. Then, the operation results of the PE unit 1203 in the last row of each pulsating array 1201 (the second part of the sum of the PE unit 1203 in the last row) are summed by the addition unit 1202 to obtain the summation data of each pixel. The summation data of one pixel is output every clock cycle, and finally the summation result of the entire image can be obtained.

[0068] It should be noted that the systolic array 1201 that sums the wavelet decomposition results by channel within a fixed neighborhood of each pixel and the systolic array 1201 that performs wavelet decomposition calculation on multiple rows of data for each pixel are a multiplexed systolic array. In specific implementation, this systolic array contains max{m,a} rows and max{n,a} columns of PE units. That is, the systolic calculation module 120 including multiple systolic arrays 1201 is a multiplexed module. Constructing this multiplexed module can reduce the integration complexity of the SPAD hardware system and improve the processing efficiency of the SPAD hardware system.

[0069] Specifically, in the aforementioned data flow method, among the multiple array-distributed PE units 1203, the input data of each PE unit 1203 is passed from left to right along the row direction and remains unchanged, while the second part of each PE unit 1203 is passed from top to bottom along the column direction for A+C calculation. For example, for any row of any systolic array 1201, the input data of the i-th column in the current period will be moved to the PE unit 1203 of the (i+1)-th column in the next period, and for any column of any systolic array 1201, the second part of the sum of the i-th row in the current period will be moved to the PE unit 1203 of the (i+1)-th row in the next period for A+C calculation.

[0070] After introducing the pulse calculation module 120, the sliding window capture module 130 will be introduced in detail.

[0071] Specifically, the sliding window capture module 130 includes a register chain 1301, an adder tree 1302, a comparator 1303, and a counter 1304.

[0072] In one embodiment of the present invention, a sliding window cropping module 130 performs a sliding window cropping operation on the summed data of each pixel to obtain compressed data of each pixel, including:

[0073] After obtaining the summation data for each pixel, the summation data for each pixel is sequentially input into the first register of the register chain 1301. The data of each register in the register chain 1301 is passed to the next-level register (i.e., the next-level register) in each clock cycle. The depth of the register chain 1301 (i.e., the amount of data that the register chain 1301 can store) is equal to the window length w of the sliding window. When the register chain 1301 is full, that is, when all registers in the register chain 1301 are full, the data in each register in the register chain 1301, i.e., w data, are input in parallel into the adder tree 1302 for summation. Then, the maximum value of the summation operation for each pixel is obtained through the comparator 1303, and the target index of each maximum value is obtained through the counter 1304 and the index register in the register chain 1301. Finally, the wavelet decomposition result within the address range corresponding to each target index is determined as the compressed data for each pixel and output. That is, only the data within the address range is output to complete the compression. It should be noted that after each pixel is processed, the comparator 1301, the counter 1304, and the index register are all set to zero.

[0074] For example, with a preset step size of 1 and a preset window length of 7, assuming the summation data of the current pixel includes discrete data of 137 data elements, it is represented as follows: By performing a sliding window operation on the discrete data and summing the data within each sliding window, a set of data (i.e., at least one target summation result) is obtained. Then compare the size of each data point after addition, assuming that among them... If the value is the largest, then... The maximum value is determined, and the data index within the current sliding window of the maximum value is determined as the target index. The address range of the wavelet decomposition data to be output is obtained based on the target index, and then the wavelet decomposition data of all pixels within the address range is output.

[0075] In summary, the SPAD imaging data compression system for extremely low illumination provided by this invention includes a data rearrangement module for arranging one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multi-row data, a pulsation calculation module (i.e., a pulsation array and an addition unit) capable of performing wavelet decomposition calculations in parallel, and a sliding window truncation module. Compared to other hardware systems, the system provided by this invention can upgrade the traditional serial processing architecture to a highly efficient parallel architecture, eliminate redundant calculations at the algorithm level, and adopt reconfigurable pulsation array multiplexing technology at the hardware level, thereby significantly shortening the computation cycle and complexity of the SPAD hardware system and improving the processing efficiency of the SPAD hardware system.

[0076] Furthermore, the SPAD imaging data compression system for extremely low illumination provided in this embodiment of the invention, by employing control logic based on multi-level registers and finite state machines, achieves the sequential arrangement of one-dimensional discrete data corresponding to each pixel into multiple rows of data with lower clock cycles and resource overhead, and performs zero-padding operation between the one-dimensional discrete data of different pixels. This process is closely integrated with the subsequent pulsation array calculation, forming an efficient pipeline structure and improving the processing efficiency of the SPAD hardware system.

[0077] In addition, the SPAD imaging data compression system for extremely low illumination provided in this embodiment of the invention can not only improve the efficiency of wavelet decomposition calculation of each PE unit by adjusting the weight parameter configuration of each PE unit, but also be compatible with other filtering schemes for one-dimensional discrete data, thus expanding the application scenarios of the SPAD hardware system.

[0078] It should be noted that the SPAD imaging data compression system for extremely low illumination provided in this embodiment of the invention can be applied to SPAD LiDAR, enabling SPAD LiDAR to build a three-dimensional map of the surrounding environment in real time based on the system, thereby helping to assist in autonomous driving and obstacle avoidance of vehicles, robot navigation, and the creation of realistic virtual scenes or augmented reality experiences.

[0079] In the several embodiments provided by this invention, it should be understood that the hardware system disclosed in this invention can be implemented in other ways. For example, the embodiments of the SPAD imaging data compression system described above are merely illustrative. For example, the division of units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0080] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0081] Another embodiment of the present invention provides a SPAD lidar, including a SPAD imaging data compression system suitable for extremely low illumination, the SPAD imaging data compression system being capable of performing the steps provided in the above embodiments.

[0082] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A SPAD imaging data compression system suitable for extremely low illumination, characterized in that, include: The data rearrangement module is used to arrange the one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multiple rows of data, and input the multiple rows of data corresponding to each pixel into the pulsation calculation module; The pulsation calculation module includes multiple parallel pulsation arrays and an addition unit. The addition unit and each pulsation array are used to perform wavelet decomposition calculation on the multi-row data of the corresponding pixel to obtain the wavelet decomposition result of the multi-row data of each pixel, and to sum the wavelet decomposition results of all pixels within a fixed neighborhood of each pixel to obtain the summation result of each pixel. The sliding window cropping module is used to perform a sliding window cropping operation on the summation result of each pixel to obtain compressed data of each pixel.

2. The SPAD imaging data compression system for extremely low illumination according to claim 1, characterized in that, The data rearrangement module includes a multi-level register and a finite state machine; The multi-level register is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, and introduce delays in the order of rows so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the pulsation array. The finite state machine is used to perform zero-padding between the one-dimensional discrete data corresponding to adjacent pixels, so that the output result of the wavelet decomposition calculation of each pulsating array is not affected by the adjacent pixels.

3. The SPAD imaging data compression system for extremely low illumination according to claim 2, characterized in that, The multi-level register includes a first-level register and multiple second-level registers; The first-level register is used to sequentially read m discrete data points from the one-dimensional discrete data corresponding to the first pixel in the block random access memory in each clock cycle, and arrange the m discrete data points into m rows in sequence. After arranging all the one-dimensional discrete data points corresponding to the first pixel into m rows, the register then sequentially reads m discrete data points from the one-dimensional discrete data corresponding to the next pixel in the block random access memory in each clock cycle, and arranges the m discrete data points into m rows in sequence, until all the one-dimensional discrete data points corresponding to each pixel are arranged into m rows, where m is the same as the number of rows in the pulsation array. The multiple second-level registers are used to delay the data in the i-th row, such that each row of data is delayed by one clock cycle relative to the previous row of data, 1 < i ≤ m.

4. The SPAD imaging data compression system for extremely low illumination according to claim 3, characterized in that, The finite state machine includes a first state, a second state, and a third state; When the finite state machine is in the first state, the finite state machine is used to wait for the target signal to start data arrangement of the first-level register; When the finite state machine receives the target signal, it switches from the first state to the second state. When the finite state machine is in the second state, it updates the address values ​​of the m discrete data in the one-dimensional discrete data of each pixel, so that the first-level register reads discrete data from the block random access memory according to the address values, and increments the address value of the currently read discrete data by one after each time the first-level register reads discrete data. When the counter indicates that the number of times the first-level register has read data is equal to a preset number, the finite state machine switches from the second state to the third state. When the finite state machine is in the third state, it performs the zero-padding operation between the one-dimensional discrete data corresponding to adjacent pixels, and returns to the second state after completing the zero-padding operation.

5. The SPAD imaging data compression system for extremely low illumination according to claim 1, characterized in that, The pulsation calculation module further includes a weight storage unit, which is used to assign corresponding weight parameters to each of the pulsation arrays. The wavelet decomposition calculation is performed on the multi-row data of the corresponding pixel using the addition unit and each of the pulsation arrays to obtain the wavelet decomposition result of the multi-row data of each pixel, including: Based on the addition unit, each of the pulsation arrays and the corresponding weight parameters, the wavelet decomposition calculation is performed on the multi-row data of each pixel using matrix multiplication and a preset data flow method to obtain the wavelet decomposition result of the multi-row data of each pixel.

6. The SPAD imaging data compression system for extremely low illumination according to claim 5, characterized in that, Each of the pulsating arrays includes multiple array-distributed processing units, and the number of processing units is m×n; The wavelet decomposition calculation is performed on multiple rows of data for each pixel based on the addition unit, each pulse array, and the corresponding weight parameters, using matrix multiplication and a preset data flow method, to obtain the wavelet decomposition result for the multiple rows of data for each pixel, including: First, based on the wavelet decomposition principle, the wavelet decomposition process is equivalent to multiplying a one-dimensional vector with a sparse matrix. Each column of the sparse matrix contains non-zero values ​​composed of wavelet low-frequency filter coefficients, and each non-zero value is the result of shifting the previous column down two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to a corresponding systolic array, so that each systolic array uses a data flow with fixed weights, performing an A×B+C operation once per clock cycle. Here, A is the input data of each processing unit, where the input data is one row of data from multiple rows of data for each pixel; B is the corresponding weight parameter; and C is the first partial sum passed by the processing unit above each processing unit, which is the sum of the A×B+C operation performed by the processing unit above each processing unit in each clock cycle. Finally, the addition unit sums the operation results of the processing unit in the last row of each systolic array to obtain a value of the wavelet decomposition result for the multiple rows of data for each pixel.

7. The SPAD imaging data compression system for extremely low illumination according to claim 6, characterized in that, The data stream methods include: The input data of each processing unit is passed from left to right along the row direction and remains unchanged, and the first part of each processing unit is passed from top to bottom along the column direction to perform the A×B+C operation.

8. The SPAD imaging data compression system for extremely low illumination according to claim 1, characterized in that, Each of the pulsating arrays includes multiple array-distributed processing units; The wavelet decomposition results of all pixels within a fixed neighborhood of each pixel are summed using the addition unit and each of the pulsation arrays to obtain the summation result for each pixel, including: Centered on each pixel, a fixed neighborhood range is determined at a fixed distance for each pixel. Through each pulsation array, the wavelet decomposition results of all pixels within the fixed neighborhood range are summed sequentially according to a preset number of channels. Then, the calculation results output by the processing unit in the last row of each pulsation array are accumulated through the addition unit until the summation data of all channels is calculated, so as to obtain the summation result of each pixel.

9. The SPAD imaging data compression system for extremely low illumination according to claim 8, characterized in that, The number of processing units is a×a; The process involves summing the wavelet decomposition results of all pixels within the fixed neighborhood range according to a preset number of channels using each of the pulsation arrays. Then, the addition unit accumulates the calculation results output by the processing unit in the last row of each pulsation array until the summation data for all channels is calculated, to obtain the summation result within each pixel. This includes: Each systolic array employs a fixed data flow, performing an A+D operation once per clock cycle. A represents the target input data for each processing unit in the preset channel, which is the wavelet decomposition result of summing each pixel across the preset number of channels. D represents the second part of the sum passed to the processing unit above each processing unit, which is the sum of the A+D operation performed by the processing unit above each processing unit in each clock cycle. Then, the addition unit sums the operation results output by the processing unit in the last row of each systolic array until the summation data of all channels is calculated to obtain the summation result for each pixel.

10. The SPAD imaging data compression system for extremely low illumination according to claim 1, characterized in that, The sliding window capture module includes a register chain, an adder tree, a comparator, and a counter; The sliding window cropping module performs a sliding window cropping operation on the summed data of each pixel to obtain compressed data for each pixel, including: The summation result of each pixel is sequentially input into the first register in the register chain. The data of each register in the register chain is passed to the subsequent register in each clock cycle. When all registers in the register chain are filled, the data in each register is input in parallel into the adder tree for summation. Then, the maximum value of the summation operation of each pixel is obtained through the comparator, and the target index of each maximum value is obtained through the counter and the index register in the register chain. Finally, the wavelet decomposition result within the address range corresponding to each target index is determined as the compressed data of each pixel and output.

Images