SPAD imaging data compression system suitable for extremely low illumination

Abstract

The invention discloses an SPAD imaging data compression system suitable for extremely low illumination, and the system comprises a data rearrangement module which is used for arranging one-dimensional discrete data corresponding to a plurality of pixels received by an SPAD array into multiple rows of data, and inputting the multiple rows of data corresponding to each pixel into a pulsation calculation module; the pulsation calculation module comprises a plurality of parallel pulsation arrays and an addition unit, and the addition unit and each pulsation array are used for performing wavelet decomposition calculation on the multi-row data of the corresponding pixel to obtain a wavelet decomposition result of the multi-row data of each pixel; carrying out summation on the wavelet decomposition results of all the pixels in the fixed neighborhood range of each pixel to obtain a summation result of each pixel; and the sliding window interception module is used for carrying out sliding window interception operation on the summation result of each pixel so as to obtain compressed data of each pixel. According to the invention, the processing time of the SPAD hardware system can be reduced, and the processing efficiency of the SPAD hardware system can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital image processing, and in particular relates to a SPAD imaging data compression system suitable for extremely low illumination. Background Art

[0002] Currently, in high-resolution, high-frame-rate applications, the amount of data generated per second by large-scale Single Photon Avalanche Diode (SPAD) arrays is too large, even exceeding the bandwidth of advanced data buses such as the Universal Serial Bus (USB) by an order of magnitude. Therefore, in order to solve this bottleneck, wavelet transform is usually used to compress the data collected by the SPAD array.

[0003] However, traditional wavelet transform implementation methods usually rely on a multi-stage filter architecture. However, after the multi-stage filter performs convolution operations, downsampling operations are usually performed, which will result in a large amount of useless calculations. These redundant operations not only waste a large amount of computing resources of the SPAD hardware system, but also significantly increase the processing time of the SPAD hardware system and reduce the processing efficiency of the SPAD hardware system.

[0004] Therefore, how to reduce the processing time of the SPAD hardware system is a technical problem that needs to be solved urgently. Summary of the Invention

[0005] To address the issue of how to reduce the processing time of a SPAD hardware system, the present invention provides a SPAD imaging data compression system suitable for extremely low illumination. The technical problem to be solved by the present invention is achieved through the following technical solutions: The present invention provides a SPAD imaging data compression system suitable for extremely low illumination, comprising: A data rearrangement module is used to arrange the one-dimensional discrete data corresponding to the multiple pixels received by the SPAD array into multiple rows of data, and input the multiple rows of data corresponding to each pixel into the pulsation calculation module; The systolic calculation module includes multiple parallel systolic arrays and adding units. The adding unit and each systolic array are used to perform wavelet decomposition calculations on multiple rows of data corresponding to pixels to obtain wavelet decomposition results of the multiple rows of data for each pixel, and sum the wavelet decomposition results of all pixels in a fixed neighborhood of each pixel to obtain a summed result for each pixel. The sliding window interception module is used to perform a sliding window interception operation on the summation result of each pixel to obtain compressed data of each pixel.

[0006] In one embodiment of the present invention, the data rearrangement module includes multiple levels of registers and a finite state machine; The multi-level register is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, introduce delays in sequence according to the rows so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the systolic array; The finite state machine is used to perform zero padding operation between one-dimensional discrete data corresponding to adjacent pixels, so that the output result of wavelet decomposition calculation of each systolic array will not be affected by adjacent pixels.

[0007] In one embodiment of the present invention, the multi-level register includes a first-level register and a plurality of second-level registers; The first-level register is used to sequentially read m discrete data of the one-dimensional discrete data corresponding to the first pixel from the block random access memory in each clock cycle, and sequentially arrange the m discrete data into m rows. After all the one-dimensional discrete data corresponding to the first pixel are arranged in m rows, the first-level register is used to sequentially read m discrete data of the one-dimensional discrete data corresponding to the next pixel from the block random access memory in each clock cycle, and sequentially arrange the m discrete data into m rows, until all the one-dimensional discrete data corresponding to each pixel are arranged into m rows, where m is the same as the number of rows of the systolic array. The plurality of second-level registers are used to delay the i-th row of data, so that each row of data is delayed by one clock cycle relative to the previous row of data, 1<i≤m.

[0008] In one embodiment of the present invention, the finite state machine includes a first state, a second state, and a third state; When the finite state machine is in the first state, the finite state machine is used to wait for receiving a target signal for the first-level register to start data arrangement; when the finite state machine receives the target signal, the finite state machine switches from the first state to the second state. When the finite state machine is in the second state, the finite state machine is used to update the address values ​​of m discrete data in the one-dimensional discrete data of each pixel, so that the first-level register reads the discrete data from the block random access memory according to the address value, and after the first-level register reads the discrete data once, the address value of the discrete data currently read by the first-level register is increased by one; when the counter indicates that the number of reads of the first-level register is equal to the preset number, the finite state machine switches from the second state to the third state. When the finite state machine is in the third state, the finite state machine is used to perform a zero-padding operation between the one-dimensional discrete data corresponding to adjacent pixels, and return to the second state after completing the zero-padding operation.

[0009] In one embodiment of the present invention, the systolic calculation module further includes a weight storage unit, the weight storage unit being configured to assign corresponding weight parameters to each systolic array; The wavelet decomposition calculation is performed on the multiple rows of data corresponding to the pixel through the addition unit and each systolic array to obtain the wavelet decomposition results of the multiple rows of data for each pixel, including: Based on the addition unit, each systolic array and the corresponding weight parameters, a matrix multiplication method and a preset data flow method are used to perform wavelet decomposition calculation on multiple rows of data for each pixel to obtain wavelet decomposition results for the multiple rows of data for each pixel.

[0010] In one embodiment of the present invention, each systolic array includes a plurality of array-distributed processing units, and the number of the processing units is m×n; Based on the addition unit, each systolic array and the corresponding weight parameters, a matrix multiplication method and a preset data flow method are used to perform wavelet decomposition calculations on multiple rows of data for each pixel, and the wavelet decomposition results of the multiple rows of data for each pixel are obtained, including: First, based on the wavelet decomposition principle, the wavelet decomposition process is equivalent to the multiplication of a one-dimensional vector and a sparse matrix. The non-zero value of each column of the sparse matrix is ​​composed of the wavelet low-frequency filter coefficients, and each non-zero value in the column is the result of shifting the previous column down by two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to the corresponding systolic array, so that each systolic array adopts a data flow method with fixed weights and performs an A×B+C operation once in each clock cycle, where A is the input data of each processing unit, the input data is one row of data in multiple rows of data for each pixel, B is the corresponding weight parameter, and C is the first partial sum transmitted by the upper processing unit of each processing unit. The first partial sum is the sum of the A×B+C operation performed by the upper processing unit in each clock cycle. Finally, the operation results of the processing units in the last row of each systolic array are summed by the addition unit to obtain a value of the wavelet decomposition result of the multiple rows of data for each pixel.

[0011] In one embodiment of the present invention, the data flow mode includes: The input data of each processing unit is transmitted from left to right along the row direction and remains unchanged, and the first part of each processing unit is transmitted from top to bottom along the column direction to perform the operation A×B+C.

[0012] In one embodiment of the present invention, each systolic array includes a plurality of array-distributed processing units; The wavelet decomposition results of all pixels within a fixed neighborhood of each pixel are summed through the addition unit and each systolic array to obtain the summation result for each pixel, including: With each pixel as the center, a fixed neighborhood range of each pixel is determined at a fixed distance. Through each systolic array, the wavelet decomposition results of all pixels in the fixed neighborhood range are summed in sequence according to the preset number of channels. Then, the calculation results output by the processing units in the last row of each systolic array are accumulated through the addition unit until the sum data of all channels are calculated to obtain the sum result of each pixel.

[0013] In one embodiment of the present invention, the number of processing units is a×a; Through each systolic array, the wavelet decomposition results of all pixels in a fixed neighborhood are summed according to the preset number of channels. Then, the calculation results output by the processing units in the last row of each systolic array are accumulated through the adding unit until the sum data of all channels are calculated to obtain the summation result of each pixel, including: Each systolic array uses a fixed data flow method, performing an A+D operation once per clock cycle. A is the target input data for each processing unit in a preset channel. The target input data is the wavelet decomposition result summed for each pixel in the preset number of channels. D is the second partial sum transmitted by the processing unit above each processing unit. The second partial sum is the sum of the A+D operations performed by the processing unit above each clock cycle. Then, the operation results output by the processing units in the last row of each systolic array are summed through the addition unit until the summed data of all channels are calculated to obtain the summed result for each pixel.

[0014] In one embodiment of the present invention, the sliding window interception module includes a register chain, an adder tree, a comparator, and a counter; The sliding window interception module performs a sliding window interception operation on the summed data of each pixel to obtain the compressed data of each pixel, including: The summation result of each pixel is input into the first register in the register chain in sequence. The data of each register in the register chain is passed to the subsequent register in each clock cycle. When all the registers in the register chain are filled, the data in each register is input into the adder tree in parallel for summation operation. Then, the maximum value of the summation operation of each pixel is obtained through the comparator, and the target index of each maximum value is obtained through the counter and the index register in the register chain. Finally, the data in the address range corresponding to each target index is determined as the compressed data of each pixel and output.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention provides a SPAD imaging data compression system suitable for extremely low-light conditions. The system comprises a data rearrangement module for arranging the one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multiple rows of data, and a systolic calculation module (i.e., a systolic array and an addition unit) capable of performing wavelet decomposition calculations in parallel. Compared to other hardware systems, the system provided by the present invention upgrades the traditional serial processing architecture to a highly efficient parallel architecture, eliminating redundant computations at the algorithm level and employing systolic array multiplexing technology at the hardware level. This significantly reduces the computational cycle and complexity of the SPAD hardware system, thereby improving its processing efficiency.

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a structural diagram of a SPAD imaging data compression system suitable for extremely low illumination provided by an embodiment of the present invention; Figure 2 This is a state transition diagram of a finite state machine provided by an embodiment of the present invention; Figure 3 This is a data flow diagram of wavelet decomposition based on PE units provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the following is a detailed description of a SPAD imaging data compression system suitable for extremely low illumination proposed in accordance with the present invention, in combination with the accompanying drawings and specific implementation methods.

[0019] The aforementioned and other technical contents, features, and effects of the present invention are clearly presented in the following detailed description of the specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the intended purpose can be obtained. However, the accompanying drawings are provided for reference and illustration purposes only and are not intended to limit the technical solutions of the present invention.

[0020] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations are intended to cover non-exclusive inclusion, such that an article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the article or device comprising the element.

[0021] The present invention aims to solve the problem of how to reduce the processing time of the SPAD hardware system and proposes a SPAD imaging data compression system suitable for extremely low illumination. Figure 1 The system 100 includes: multiple parallel data rearrangement modules 110, a systolic calculation module 120 and a sliding window interception module 130, the data rearrangement module 110 is connected to the systolic array 1201 in the systolic calculation module 120, and the systolic calculation module 120 is connected to the sliding window interception module 130.

[0022] The data rearrangement module 110 is used to arrange the one-dimensional discrete data corresponding to each pixel received by the SPAD array into multiple rows of data, and input the multiple rows of data corresponding to each pixel into the systolic array.

[0023] For example, each pixel corresponds to 1×1024 discrete data, and the data rearrangement module 110 can rearrange the 1×1024 discrete data into 8×128 discrete data (i.e., multiple rows of data), that is, rearrange the 1 row and 1024 columns of discrete data corresponding to each pixel into 8 rows and 128 columns, and input the 8 rows and 128 columns of discrete data corresponding to each pixel into the systolic array.

[0024] The systolic calculation module 120 includes multiple parallel systolic arrays 1201 and an adding unit 1202. The adding unit 1202 and each systolic array 1201 are used to perform wavelet decomposition calculations on multiple rows of data corresponding to pixels to obtain wavelet decomposition results for the multiple rows of data for each pixel, and to sum the wavelet decomposition results of all pixels within a fixed neighborhood of each pixel to obtain summed data for each pixel.

[0025] The sliding window interception module 130 is used to perform a sliding window interception operation on the summation result of each pixel to obtain compressed data of each pixel.

[0026] The data rearrangement module 110 is first described in detail below.

[0027] Specifically, such as Figure 1As shown, the data rearrangement module 110 includes a multi-stage register 1101 and a finite state machine 1102 .

[0028] Among them, the multi-level register 1101 is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, and introduce delays in sequence according to the row order, so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the pulsating array.

[0029] It should be noted that, in one embodiment of the present invention, the multi-level register 1101 includes a first-level register and multiple second-level registers.

[0030] Among them, the first-level register is used to read m discrete data in the one-dimensional discrete data corresponding to each pixel from the block random access memory (BRAM) in sequence in each clock cycle, and arrange the m discrete data into m rows in sequence. After all the one-dimensional discrete data corresponding to the first pixel are arranged in m rows, a read zero operation is performed (the read address is changed to the address where zero is stored in the BRAM) for n cycles, and then m discrete data in the one-dimensional discrete data corresponding to the next pixel are read in sequence from the block random access memory in each clock cycle, and the m discrete data are arranged into m rows in sequence. Then the read zero operation is continued for n cycles, and then the next pixel is read, until all the one-dimensional discrete data corresponding to each pixel are arranged into m rows, where m is the same as the number of rows of the systolic array.

[0031] Exemplarily, when m is equal to 8, n is equal to 10, and each pixel corresponds to 1×1024 discrete data, the first-level register reads the 1st to 8th discrete data of the 1×1024 discrete data corresponding to the first pixel from the block random access memory in sequence in each clock cycle, and arranges the 8 discrete data into 8 rows in sequence, and then reads the 9th to 16th discrete data of the 1×1024 discrete data corresponding to the first pixel from the block random access memory in sequence, and arranges the 8 discrete data into 8 rows in sequence, until the 1×1024 discrete data corresponding to the first pixel are arranged into m rows, reads zero from the BRAM for 10 cycles, and then reads the 1st to 8th discrete data of the 1×1024 discrete data corresponding to the next pixel from the block random access memory in sequence in each clock cycle, and arranges the 8 discrete data into 8 rows in sequence, until the 1×1024 discrete data corresponding to each pixel are arranged into m rows.

[0032] The plurality of second-level registers are used to delay the i-th row of data, so that each row of data is delayed by one clock cycle relative to the previous row of data, 1<i≤m.

[0033] For example, there is one second-level register in the second row of data, which is used to delay the second row of data by one clock cycle relative to the first row of data; there are two second-level registers in the third row of data, which are used to delay the third row of data by one clock cycle relative to the second row of data; there are three second-level registers in the fourth row of data, which are used to delay the fourth row of data by one clock cycle relative to the third row of data...; there are seven second-level registers in the eighth row of data, which are used to delay the eighth row of data by one clock cycle relative to the seventh row of data, and so on.

[0034] The finite state machine 1102 includes a first state (IDLE state), a second state (RD_DATA state) and a third state (RD_ZERO state). Specifically, Figure 2 As shown, Figure 2 This is a state transition diagram of a finite state machine provided by an embodiment of the present invention. When the finite state machine is in the first state, the finite state machine is used to wait for receiving a target signal for the first-level register to start data arrangement; when the finite state machine receives the target signal, the finite state machine switches from the first state to the second state. When the finite state machine is in the second state, the finite state machine is used to update the address value of the one-dimensional discrete data corresponding to each pixel, and after the first-level register reads discrete data once, the address value of the discrete data currently read by the first-level register is increased by one; when the counter indicates that the number of times the first-level register reads is equal to the preset number, the finite state machine switches from the second state to the third state. When the finite state machine is in the third state, the finite state machine is used to perform a zero-padding operation between the one-dimensional discrete data corresponding to adjacent pixels, and return to the second state after completing the zero-padding operation.

[0035] After introducing the data rearrangement module 110 , the pulsation calculation module 120 will be described in detail.

[0036] Specifically, such as Figure 1 As shown, each systolic array 1201 of the systolic computing module 120 includes a plurality of array-distributed processing elements (PE) units 1203. The systolic computing module 120 also includes a weight storage unit 1204. The weight storage unit 1204 is used to assign corresponding weight parameters to the PE units 1203 of each systolic array 1201. The PE units 1203 in the first row of each systolic array 1201 can be connected to the weight storage unit 120 to vertically transmit the corresponding weight parameters assigned by the weight storage unit 120 to all PE units 1203 in the systolic array 1201. It should be noted that Figure 1 The array-distributed PE units 1203 in the systolic array 1201 are merely exemplary, and the number of the PE units 1204 in the systolic array 1201 is not specifically limited.

[0037] In one embodiment of the present invention, the wavelet decomposition calculation is performed on the multiple lines of data corresponding to the pixel by the adding unit 1202 and each systolic array 1201 to obtain the wavelet decomposition results of the multiple lines of data for each pixel, including: Based on the adding unit 1202, each systolic array 1201 and the corresponding weight parameters, a matrix multiplication method and a preset data flow method are used to perform wavelet decomposition calculation on the multiple rows of data of each pixel to obtain the wavelet decomposition results of the multiple rows of data of each pixel.

[0038] Specifically, each systolic array 1201 includes m×n PE units 1203. The weight storage unit 1204 first converts the one-dimensional discrete wavelet decomposition process into the multiplication of a one-dimensional vector (i.e., one-dimensional discrete data) and a sparse matrix based on the wavelet decomposition principle. The non-zero values ​​in each column of the sparse matrix consist of filter coefficients, and each non-zero value in each column is the result of shifting the previous column down by two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to the corresponding PE unit 1203 of the systolic array 1201. This allows the PE unit 1203 of each systolic array 1201 to perform an A×B+C operation once per clock cycle using a fixed-weight data flow. Among them, A is the input data of each PE unit 1203, and the input data is one row of multiple rows of data for each pixel. B is the corresponding weight parameter. C is the first partial sum transmitted by the PE unit 1203 above each PE unit 1203. The first partial sum is the sum of the A×B+C operation performed by the upper PE unit 1203 in each clock cycle. Finally, the operation results of the PE units 1203 in the last row of each systolic array 1201 are summed by the addition unit 1202 to obtain the wavelet decomposition result of the multiple rows of data for each pixel.

[0039] For example, Figure 3 As shown, Figure 3 This is a data flow diagram of wavelet decomposition based on PE units provided by an embodiment of the present invention. Figure 3 As shown, the data on the left (1, 2...m...m+1) is m rows of data for each pixel. Each row of data is delayed by one cycle compared to the previous row of data. Each systolic array 1201 includes m×n PE units 1203 (it should be noted that Figure 3The diagram only illustrates one systolic array 1201 in the systolic computation module 120. Each systolic array 1201 processes m rows of data for a corresponding pixel. Multiple systolic arrays process m rows of data for multiple pixels in parallel. Each PE unit 1203 performs the operation A × B + C once per clock cycle. W11 through Wmn on each PE unit 1203 represent weight parameters for each PE unit 1203 in each systolic array 1201.

[0040] Specifically, the data flow described above is as follows: among the multiple array-distributed PE units 1203, the input data of each PE unit 1203 is transmitted from left to right along the row direction and remains unchanged, while the first partial sum of each PE unit 1203 is transmitted from top to bottom along the column direction to perform the A×B+C operation. For example, for any row of any systolic array 1201, the input data of the i-th column in the current cycle will be moved to the PE unit 1203 in the i+1-th column in the next cycle. For any column of any systolic array 1201, the first partial sum of the i-th row in the current cycle will be moved to the PE unit 1203 in the i+1-th row in the next cycle to perform the A×B+C operation.

[0041] In one embodiment of the present invention, the wavelet decomposition results of all pixels within a fixed neighborhood of each pixel are summed by the adding unit 1202 and each systolic array 1201 to obtain summed data for each pixel, including: With each pixel as the center, a fixed neighborhood range of each pixel is determined at a fixed distance. The wavelet decomposition results of all pixels in the fixed neighborhood range are summed sequentially according to a preset number of channels through the PE unit 1203 of each systolic array 1201. Then, the operation results output by the PE units 1203 in the last row of each systolic array 1201, i.e., the partial sums, are accumulated through the adding unit 1202 until the sum data of all channels are calculated to obtain the sum data of each pixel.

[0042] Specifically, each systolic array 1201 includes a×a PE units 1203. Each PE unit 1203 uses a fixed data flow method to perform an A+D operation once in each clock cycle. A is the target input data of each PE unit 1203 in a preset channel. The target input data is the wavelet decomposition result of summing each pixel in the preset channel. The wavelet decomposition result can form a two-dimensional data, wherein the first batch of target input data is the wavelet decomposition result of summing all pixels from row 1 to row a in the preset channel, and the second batch of target input data immediately following is the wavelet decomposition result of summing all pixels from row 2 to row a+1 in the preset channel. The wavelet decomposition results summed in the channel are calculated until all the summed data of all channels are calculated. D is the second partial sum transmitted by the upper PE unit 1203 of each PE unit 1203. The second partial sum is the sum of the A+C operation performed by the upper PE unit 1203 in each clock cycle. Then, the operation results of the PE units 1203 in the last row of each systolic array 1201 (the second partial sum of the PE units 1203 in the last row) are summed through the addition unit 1202 to obtain the summed data of each pixel. The summed data of one pixel is output in each clock cycle, and finally the summed result of the entire image can be obtained.

[0043] It should be noted that the systolic array 1201 for summing the wavelet decomposition results within a fixed neighborhood of each pixel by channel and the systolic array 1201 for performing wavelet decomposition calculations on multiple rows of data for each pixel are a multiplexed systolic array. In specific implementation, the systolic array includes max{m,a} rows and max{n,a} columns of PE units, that is, the systolic calculation module 120 including multiple systolic arrays 1201 is a multiplexed module. Constructing this multiplexed module can reduce the integration complexity of the SPAD hardware system while improving the processing efficiency of the SPAD hardware system.

[0044] Specifically, the data flow described above is as follows: among the multiple array-distributed PE units 1203, the input data of each PE unit 1203 is transmitted from left to right along the row direction and remains unchanged. The second partial sum of each PE unit 1203 is transmitted from top to bottom along the column direction to perform the A+C operation. For example, for any row of any systolic array 1201, the input data of the i-th column in the current cycle will be moved to the PE unit 1203 in the i+1-th column in the next cycle. For any column of any systolic array 1201, the second partial sum of the i-th row in the current cycle will be moved to the PE unit 1203 in the i+1-th row in the next cycle to perform the A+C operation.

[0045] After introducing the pulsation calculation module 120 , the sliding window interception module 130 will be described in detail.

[0046] Specifically, the sliding window interception module 130 includes a register chain 1301 , an adder tree 1302 , a comparator 1303 , and a counter 1304 .

[0047] In one embodiment of the present invention, the sliding window interception module 130 performs a sliding window interception operation on the summed data of each pixel to obtain compressed data of each pixel, including: After obtaining the summed data for each pixel, the summed data for each pixel is sequentially input into the first register of register chain 1301. The data in each register in register chain 1301 is transferred to the subsequent register (i.e., the next-level register) during each clock cycle. The depth of register chain 1301 (i.e., the amount of data that can be stored in register chain 1301) is equal to the window length w of the sliding window. When register chain 1301 is full, i.e., when all registers in register chain 1301 are filled, the data in each register in register chain 1301, i.e., w data, is input in parallel into adder tree 1302 for summation. A comparator 1303 then determines the maximum value of the summation for each pixel. A counter 1304 and an index register in register chain 1301 then determine the target index for each maximum value. Finally, the wavelet decomposition result within the address range corresponding to each target index is determined as the compressed data for each pixel and output. Ultimately, only the data within that address range is output to complete the compression. It should be noted that after processing each pixel, the comparator 1301 , the counter 1304 and the index register are all reset to zero.

[0048] For example, the preset step size is 1, the preset window length is 7, and it is assumed that the summed data of the current pixel includes discrete data of 137-bit data elements, which is expressed as , perform a sliding window interception operation on the discrete data, add the data in each sliding window, and then obtain a set of data (that is, at least one target summation result) Then compare the size of each data after addition, assuming that The value of is the largest, then The maximum value is determined, and the data index in the current sliding window of the maximum value is determined as the target index. The address range of the wavelet decomposition data to be output is obtained according to the target index, and then the wavelet decomposition data of all pixels in the address range are output.

[0049] In summary, the SPAD imaging data compression system for extremely low illumination, provided by an embodiment of the present invention, includes a data rearrangement module for arranging the one-dimensional discrete data corresponding to multiple pixels received by the SPAD array into multiple rows of data, as well as a systolic calculation module (i.e., a systolic array and an addition unit) and a sliding window interception module capable of performing wavelet decomposition calculations in parallel. Compared to other hardware systems, the system provided by the present invention can upgrade the traditional serial processing architecture to a highly efficient parallel architecture, eliminate redundant computations at the algorithm level, and utilize reconfigurable systolic array multiplexing technology at the hardware level to significantly shorten the computational cycle and complexity of the SPAD hardware system, thereby improving the processing efficiency of the SPAD hardware system.

[0050] In addition, the SPAD imaging data compression system suitable for extremely low illumination provided by an embodiment of the present invention adopts control logic based on multi-level registers and finite state machines to achieve the arrangement of the one-dimensional discrete data corresponding to each pixel into multiple rows of data in sequence with a relatively low clock cycle and resource overhead, and performs zero-padding operations between the one-dimensional discrete data of different pixels. This process is closely connected with the subsequent pulsating array calculation, forming an efficient pipeline structure, thereby improving the processing efficiency of the SPAD hardware system.

[0051] In addition, the SPAD imaging data compression system suitable for extremely low illumination provided by the embodiment of the present invention can not only improve the efficiency of wavelet decomposition calculation of each PE unit by adjusting the weight parameter configuration of each PE unit, but also be compatible with other filtering schemes for one-dimensional discrete data, thereby expanding the application scenarios of the SPAD hardware system.

[0052] It should be noted that the SPAD imaging data compression system suitable for extremely low illumination provided by the embodiment of the present invention can be applied to SPAD lidar, so that the SPAD lidar can build a three-dimensional map of the surrounding environment based on the system in real time, thereby helping to assist vehicle autonomous driving and obstacle avoidance, robot navigation, and create realistic virtual scenes or augmented reality experiences.

[0053] In the several embodiments provided by the present invention, it should be understood that the hardware system disclosed by the present invention can be implemented in other ways. For example, the embodiment of the SPAD imaging data compression system described above is merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or ignoring or not implementing some features.

[0054] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0055] Another embodiment of the present invention provides a SPAD laser radar, including a SPAD imaging data compression system suitable for extremely low illumination, and the SPAD imaging data compression system suitable for extremely low illumination can implement the steps provided in the above embodiment.

[0056] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. For those skilled in the art of the present invention, without departing from the concept of the present invention, several simple deductions or substitutions can be made, which should be considered to fall within the scope of protection of the present invention.

Claims

1. A SPAD imaging data compression system suitable for extremely low illumination, characterized in that: include: a data rearrangement module, configured to arrange the one-dimensional discrete data corresponding to the plurality of pixels received by the SPAD array into a plurality of rows of data, and input the plurality of rows of data corresponding to each of the pixels into the pulsation calculation module; The systolic calculation module includes a plurality of parallel systolic arrays and an adding unit, wherein the adding unit and each systolic array are used to perform wavelet decomposition calculation on multiple rows of data corresponding to a pixel to obtain a wavelet decomposition result of the multiple rows of data for each pixel, and sum the wavelet decomposition results of all pixels within a fixed neighborhood range of each pixel to obtain a summed result for each pixel; The sliding window interception module is used to perform a sliding window interception operation on the summation result of each pixel to obtain compressed data of each pixel.

2. The SPAD imaging data compression system suitable for extremely low illumination according to claim 1, characterized in that: The data rearrangement module includes a multi-level register and a finite state machine; The multi-level register is used to arrange the one-dimensional discrete data corresponding to each pixel in the three-dimensional histogram into multiple rows of data, introduce delays in sequence according to the rows, so that each row of data is delayed by one clock cycle relative to the previous row of data, and input the multiple rows of data corresponding to each pixel into the systolic array; The finite state machine is used to perform a zero-padding operation between one-dimensional discrete data corresponding to adjacent pixels, so that the output result of the wavelet decomposition calculation performed by each systolic array will not be affected by adjacent pixels.

3. The SPAD imaging data compression system suitable for extremely low illumination according to claim 2, characterized in that: The multi-level register includes a first-level register and a plurality of second-level registers; The first-level register is used to sequentially read m discrete data from the block random access memory in each clock cycle among the one-dimensional discrete data corresponding to a first pixel, and sequentially arrange the m discrete data into m rows. After all the one-dimensional discrete data corresponding to the first pixel are arranged in m rows, the first-level register is used to sequentially read m discrete data from the block random access memory in each clock cycle among the one-dimensional discrete data corresponding to a next pixel, and sequentially arrange the m discrete data into m rows, until all the one-dimensional discrete data corresponding to each pixel are arranged into m rows, where m is the same as the number of rows of the systolic array. The plurality of second-level registers are used to delay the i-th row of data, so that each row of data is delayed by one clock cycle relative to the previous row of data, 1<i≤m.

4. The SPAD imaging data compression system suitable for extremely low illumination according to claim 3, characterized in that: The finite state machine includes a first state, a second state and a third state; When the finite state machine is in the first state, the finite state machine is used to wait for receiving a target signal for the first-stage register to start data arrangement; When the finite state machine receives the target signal, the finite state machine switches from the first state to the second state. When the finite state machine is in the second state, the finite state machine is used to update the address values ​​of the m discrete data in the one-dimensional discrete data of each pixel, so that the first-level register reads discrete data from the block random access memory according to the address value, and after the first-level register reads discrete data once, the address value of the discrete data currently read by the first-level register is increased by one; when the counter indicates that the number of times the first-level register has been read is equal to the preset number, the finite state machine switches from the second state to the third state. When the finite state machine is in the third state, the finite state machine is used to perform the zero-padding operation between the one-dimensional discrete data corresponding to adjacent pixels, and return to the second state after completing the zero-padding operation.

5. The SPAD imaging data compression system suitable for extremely low illumination according to claim 1, characterized in that: The systolic calculation module further includes a weight storage unit, wherein the weight storage unit is used to assign corresponding weight parameters to each of the systolic arrays; The wavelet decomposition calculation is performed on the multiple rows of data corresponding to the pixel by the adding unit and each of the systolic arrays to obtain a wavelet decomposition result of the multiple rows of data for each of the pixels, including: Based on the adding unit, each of the systolic arrays and the corresponding weight parameters, the wavelet decomposition calculation is performed on the multiple rows of data of each pixel using a matrix multiplication method and a preset data flow method to obtain a wavelet decomposition result of the multiple rows of data of each pixel.

6. The SPAD imaging data compression system suitable for extremely low illumination according to claim 5, characterized in that: Each of the systolic arrays includes a plurality of array-distributed processing units, and the number of the processing units is m×n; The wavelet decomposition calculation is performed on the multiple rows of data of each pixel using a matrix multiplication method and a preset data flow method based on the adding unit, each of the systolic arrays, and the corresponding weight parameters to obtain a wavelet decomposition result for the multiple rows of data of each pixel, including: First, based on the wavelet decomposition principle, the wavelet decomposition process is equivalent to the multiplication of a one-dimensional vector and a sparse matrix. The non-zero values ​​in each column of the sparse matrix are composed of wavelet low-frequency filter coefficients, and each column of the non-zero values ​​is the result of shifting the previous column down by two units. The sparse matrix includes m×n regularly arranged repeating units. Then, each weight parameter of the repeating unit is mapped to a corresponding systolic array, so that each systolic array uses a fixed-weight data flow method to perform an A×B+C operation once per clock cycle, where A is the input data of each processing unit, the input data is a row of data in the multiple rows of data for each pixel, B is the corresponding weight parameter, and C is the first partial sum transmitted by the upper processing unit of each processing unit, and the first partial sum is the sum of the A×B+C operation performed by the upper processing unit in each clock cycle. Finally, the operation results of the processing units in the last row of each systolic array are summed by the addition unit to obtain a value of the wavelet decomposition result for the multiple rows of data for each pixel.

7. The SPAD imaging data compression system suitable for extremely low illumination according to claim 6, characterized in that: The data flow mode includes: The input data of each processing unit is transferred from left to right along the row direction and remains unchanged, and the first part of each processing unit is transferred from top to bottom along the column direction to perform the A×B+C operation.

8. The SPAD imaging data compression system suitable for extremely low illumination according to claim 1, characterized in that: Each of the systolic arrays includes a plurality of array-distributed processing units; Summing the wavelet decomposition results of all pixels within a fixed neighborhood of each pixel by the adding unit and each of the systolic arrays to obtain a summation result for each pixel, including: A fixed neighborhood range of each pixel is determined with a fixed distance as the center, and wavelet decomposition results of all pixels within the fixed neighborhood range are summed sequentially according to a preset number of channels through each systolic array. Then, the calculation results output by the processing units in the last row of each systolic array are accumulated through the adding unit until the sum data of all channels are calculated to obtain the sum result of each pixel.

9. The SPAD imaging data compression system suitable for extremely low illumination according to claim 8, characterized in that: The number of the processing units is a×a; The method includes summing the wavelet decomposition results of all pixels within the fixed neighborhood according to a preset number of channels through each systolic array, and then accumulating the operation results output by the processing units in the last row of each systolic array through the adding unit until the sum data of all channels are calculated to obtain the sum result within each pixel, including: Each systolic array uses a fixed data flow method to perform an A+D operation once during each clock cycle. A is the target input data of each processing unit in the preset channel, where the target input data is the wavelet decomposition result summed for each pixel in the preset number of channels. D is the second partial sum transmitted by the upper processing unit of each processing unit, where the second partial sum is the sum of the A+D operation performed by the upper processing unit during each clock cycle. Then, the addition unit sums the operation results output by the processing units in the last row of each systolic array until the summed data of all channels are calculated to obtain the summed result for each pixel.

10. The SPAD imaging data compression system suitable for extremely low illumination according to claim 1, characterized in that: The sliding window interception module includes a register chain, an adder tree, a comparator and a counter; Performing a sliding window interception operation on the summed data of each pixel by the sliding window interception module to obtain compressed data of each pixel, including: The summation result of each pixel is sequentially input into the first register in the register chain, and the data of each register in the register chain is transferred to the subsequent register in each clock cycle. When all the registers in the register chain are filled, the data in each register is input in parallel into the adder tree for summation operation. Then, the maximum value of the summation operation of each pixel is obtained through the comparator, and the target index of each maximum value is obtained through the counter and the index register in the register chain. Finally, the wavelet decomposition result within the address range corresponding to each target index is determined as compressed data for each pixel and output.

Images