Pulse video signal visualizing FPGA hardware and reconstruction method

Abstract

The application belongs to the technical field of image processing, and discloses a pulse video signal visualization FPGA hardware and a reconstruction method, which comprises a Capture module, an SNN_IP algorithm module and a MIPI_OUT module, the Capture module is used for storing a pulse signal captured by a camera and pre-processing the pulse signal, the SNN_IP algorithm module is used for reconstructing gray pixel data from the pre-processed pulse data by using a TFP algorithm, and the MIPI_OUT module is used for parallel output of the reconstructed pixel data.The application solves the problem in two parts of algorithm optimization and system level optimization respectively. In the aspect of algorithm optimization, the TFP algorithm is used to process the video pulse stream on the reconstruction algorithm, and the fast reconstruction of the pulse pixel is realized. In the aspect of hardware level optimization, the pipeline structure is adopted, and the running time is greatly reduced under the premise of limiting resources.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and in particular relates to an FPGA hardware and reconstruction method for visualizing pulse video signals. Background Technology

[0002] Modern cameras use CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) to capture light and record motion, resulting in a vast amount of digital images and videos. Traditional motion pictures are represented as frame-based sequences of images and videos. However, cameras cannot record every moment in real time, leading to discrete simulation as a compromise. Scenes within an exposure time are compressed into a single frame, blurring the motion changes within that time. While high-speed cameras exist with image exposure times exceeding 1 / 1,000 of a second or frame rates exceeding 250 frames per second, the microsecond-level motion changes of high-speed objects are missing. For high-speed tracking or detection tasks, every moment is crucial, and these consecutive frames must be compared to reconstruct temporal changes—a computationally expensive and challenging task. Summary of the Invention

[0003] The purpose of this invention is to provide FPGA hardware and reconstruction method for visualizing pulse video signals, so as to solve the above-mentioned technical problems.

[0004] To address the aforementioned technical problems, the specific technical solution of the FPGA hardware and reconstruction method for visualizing pulse video signals according to the present invention is as follows:

[0005] An FPGA hardware for visualizing pulse video signals includes a Capture module, an SNN_IP algorithm module, and a MIPI_OUT module. The Capture module is used to store and preprocess pulse signals captured by a camera. The SNN_IP algorithm module is used to reconstruct grayscale pixel data from the preprocessed pulse data using the TFP algorithm. The MIPI_OUT module is used to output the reconstructed pixel data in parallel.

[0006] Furthermore, the SNN_IP module consists of a Load_date module, a Trans_date module, and a Recon module, which are responsible for downloading pulse data, data storage conversion, and algorithm reconstruction, respectively.

[0007] Furthermore, the Capture module includes a Capture_top module and a storage module. The Capture_top module is directly connected to four camera sensors. It captures data from the four cameras, reassembles the data into complete and error-free data based on the position of the frame header, stores it in pingpong RAM, and finally transmits it to the storage module sequentially using the axi_s protocol. The four camera sensors are of the same model and have the same driver chip.

[0008] Furthermore, the cameras are configured sequentially using SPI to ensure normal startup. The connection protocol between the cameras and hardware devices adopts the CSI protocol. The Capture module reassembles the data and reconstructs it in the form of LVDS signals before it enters a data storage module. The storage module reads and writes to DDR4 in a pipeline-like manner through the Axim protocol to complete the data acquisition and storage functions of the Capture module.

[0009] Furthermore, the Capture_top module includes the Capture_sub module, the power_on module, and the spi_core module. The Capture_sub module is used to process the data signal from the CMOS sensor and output it to the synchronous FiFO, and finally output it via AXI.

[0010] The power_on module is a power-on signal generation module used to generate the power-on timing required by the CMOS sensor.

[0011] The spi_core module is used to receive the AXI signal from the host and then forward this signal to the SPI signal of the CMOS_sensor to realize the timing conversion function.

[0012] Furthermore, the Capture module includes seven hardware modules: ser2par module, tapUpdateCtrl module, delayRstGen module, frame_sync module, asyncFifo module, pixel_reorder module, and pp_pix module, which are connected sequentially.

[0013] The ser2par module, tapUpdateCtrl module, and delayRstGen module are used to match the input CIS camera clock and reset, and to convert the input LVDS signal into a normal electrical signal; the frame_sync module is used to process the actual data input from each CIS camera into 100 bits.

[0014] The asyncFifo module is an asynchronous FIFO module used for asynchronous clock domain data interaction in FIFO, with an input of 100MHz and an output of 100MHz to match different clock phases;

[0015] The pixel_reorder module is a data parallel-to-serial reordering module. Through the parallel-to-serial reordering module, the data from four separate cameras is combined into one for output.

[0016] The pp_pix module is a 4-ping-pong buffer that can store up to 4 frames of data and outputs the data with a delay via ping-pong.

[0017] Furthermore, the Load_date module of the SNN_IP algorithm module is responsible for downloading pulse data; the trans_date module of the SNN_IP algorithm module is responsible for pulse data storage and conversion, and optimizes the data storage method of the captured pulse pixels.

[0018] The recon module of the SNN_IP algorithm module is responsible for the algorithmic reconstruction of pulse data.

[0019] Furthermore, the MIPI_OUT module uses the MIPI protocol as the data output interface for the entire reconstruction module and adopts Xilinx's mipi_csi_tx IP as the physical interface. During the system initialization phase, the internal registers of the mipi_csi_tx IP are initialized and configured through the axi_lite interface. After configuring the number of lanes and image data format of the MIPI interface, the corresponding MIPI rate is calculated to achieve read and write rate matching. After the reconstructed single-frame image data is stored in DDR, the data stream in axi_stream format will flow into the mipi_csi_tx module through VDMA to realize the output of the reconstructed image.

[0020] This invention also discloses a method for visual reconstruction of pulse video signals. The method is executed on the aforementioned hardware and uses the TFP algorithm for data reconstruction, including the following steps:

[0021] Step 1: Retinal stimulation sampling

[0022] In a pulse camera, light intensity is converted into voltage by a photosensor. Once the analog-to-digital converter completes the signal conversion and outputs digital brightness intensity, the accumulator at each pixel accumulates the intensity. Different brightness intensities result in different accumulation rates. For a single pixel, if the accumulated intensity reaches the allocation threshold... This triggers a spike, indicating that the brightness at that point is sufficiently high. Simultaneously, the corresponding accumulator resets, releasing all the charge.

[0023]

[0024] Construct a pixel array and set a photoreceptor and its corresponding accumulator in each pixel. Output and reset are triggered asynchronously. At each sampling time, if a spike is just excited, output a digital signal "1"; otherwise, generate "0". Bright pixels show more photons collected in the pixel, resulting in a larger ADC value. It is easier and faster to exceed the scheduling threshold. Based on this principle, the peak pattern of texture analysis is reconstructed.

[0025] Step 2: Image Texture Acquisition

[0026] By replaying the spike signals, historical footage can be depicted. In the TFP algorithm, there is a time window for collecting spikes within a specific period. By counting these spikes, the texture is calculated as follows:

[0027]

[0028] The size of the time window is w, t i N represents the previous w time points. w This refers to the total number of pulses collected within the time window. C represents the maximum dynamic range for reconstruction; a larger size means a longer playback period and includes more spikes. The time window size is set to the dispatch threshold. At the same time, the texture is accurately reconstructed, and the TFP method recovers textures with different dynamic ranges by adjusting the time window size to different contrast values.

[0029] Furthermore, an algorithm of reconstructing one frame from seven frames is adopted, with the time window size determined to be 7, i.e., w=7. The number of pulse "1"s in the seven neighboring frames is determined statistically, and the estimated reconstructed pixel value is determined by the count of "1"s. The reconstructed pixel is an 8-bit grayscale pixel. Parallel and pipelined design is employed. In the input section, the storage array is partitioned into seven arrays for parallel input. In the output section, eight pixels are output in parallel per cycle. Linear pipelined design is used in the reconstruction stage. The circuit design selects 2GB of DDR4 memory. The dependency relationship between data storage, reconstruction, and output is: data storage followed by reconstruction, reconstruction followed by output. The pipeline also follows this relationship. In this sequential design, the address configuration is as follows: Capture sequentially stores data at sixteen consecutive addresses; SNN reconstruction reads data sequentially from the Capture module and then writes it back to sixteen consecutive addresses for storage; MIPI output reads the results from the SNN reconstruction sequentially and outputs them. Regarding pipeline setup, since there are no images to reconstruct and output sequentially at the beginning, the pipeline has two setup times: the first stage only performs Capture data storage, and the second stage performs the first reconstruction outside of storage. From the second stage onwards, all three processes run in parallel, thus driving all modules simultaneously and achieving synchronous acquisition and output.

[0030] The FPGA hardware and reconstruction method for visualizing pulse video signals of the present invention have the following advantages: The present invention addresses these advantages through both algorithm optimization and system-level optimization. First, regarding algorithm optimization: In the reconstruction algorithm, we use the TFP algorithm to process the video pulse stream, achieving rapid reconstruction of pulse pixels. Second, regarding hardware-level optimization: By adopting a pipeline structure, the running time is significantly reduced under resource constraints. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the pulse video signal visualization reconstruction process of the present invention;

[0032] Figure 2 This is a diagram showing the camera pixel data transmission of the present invention;

[0033] Figure 3 This is a schematic diagram of the hardware module circuit connection of the Capture module of the present invention;

[0034] Figure 4 This is a schematic diagram of data reconstruction in the Capture module of the present invention;

[0035] Figure 5 This is a functional flowchart of the Capture_top module of the present invention;

[0036] Figure 6 This is a schematic diagram of the trans_date module of the present invention;

[0037] Figure 7 This is a schematic diagram illustrating the reconstruction of textures from playback using TFP according to the present invention;

[0038] Figure 8 This is a schematic diagram of TFP pixel reconstruction according to the present invention;

[0039] Figure 9 This is a schematic diagram of the system flow stage optimization of the present invention. Detailed Implementation

[0040] To better understand the purpose, structure, and function of this invention, the following detailed description of a pulse video signal visualization reconstruction method and FPGA hardware is provided in conjunction with the accompanying drawings.

[0041] Compared to traditional cameras, high-speed pulse cameras can record events at a frame rate of 40,000 frames per second, offering high speed, real-time performance, and intelligence, helping people "chase light and electricity." Pulse cameras mimic the retinal coding principle of primates, with each pixel independently converting light signals into pulse streams, accurately and completely recording the spatiotemporal changes of light. Therefore, this invention, based on the high-definition, ultra-high-speed pulse stream output by a high-speed pulse camera, implements network acceleration computation on an FPGA to research a high-speed pulse detection acceleration system. This system adapts to the extremely high sampling rate of high-speed pulse cameras, achieving direct detection in the pulse domain without complex image reconstruction and enhancement.

[0042] like Figure 1 As shown, the present invention discloses an FPGA hardware for visualizing pulse video signals, comprising a Capture module, an SNN_IP algorithm module, and a MIPI_OUT module. The Capture module stores and preprocesses the pulse signals captured by the camera; the SNN_IP algorithm module reconstructs grayscale pixel data from the preprocessed pulse data using the TFP algorithm; and the MIPI_OUT module outputs the reconstructed pixel data in parallel. The SNN_IP module consists of a Load_date module, a Trans_date module, and a Recon module, which are responsible for downloading pulse data, data storage and conversion, and algorithm reconstruction, respectively.

[0043] like Figure 2As shown, the Capture module includes a Capture_top module and a storage module. The Capture_top module is directly connected to four camera sensors. It captures data from these four cameras, reassembles the data into complete and error-free data based on the frame header position, stores it in PingPong RAM, and finally transmits it sequentially to the storage module using the AXI_S protocol. The four sensors are of the same model and use the same driver chip. The cameras are configured sequentially using SPI to ensure normal startup. The camera-device connection protocol uses CSI. The Capture module uses... Figure 4 The data is reassembled using a combination method. After reconstruction in LVDS signal form, it enters a data storage module. The storage module reads and writes to DDR4 in a pipeline-level manner via the axim protocol to complete the data acquisition and storage functions of the Capture module.

[0044] like Figure 5 As shown, the Capture_top module mainly includes the Capture_sub module, the power_on module, and the spi_core module.

[0045] The Capture_sub module processes the data signals from the CMOS sensor and outputs them to the synchronous FiFO, and finally outputs them via AXI.

[0046] The power_on module generates the power-on signal, producing the power-on timing required by the CMOS sensor. The spi_core module receives the AXI signal from the host and forwards it to the CMOS sensor's SPI signal, thus implementing timing conversion.

[0047] like Figure 3 As shown, the Capture module includes seven hardware modules: ser2par, tapUpdateCtrl, delayRstGen, frame_sync, asyncFifo, pixel_reorder, and pp_pix. These modules are connected sequentially.

[0048] The ser2par, tapUpdateCtrl, and delayRstGen modules are used to match the input CIS camera clock and reset, and to convert the input LVDS signal into a normal electrical signal for processing.

[0049] The frame_sync module is used to process the actual data input from each CIS camera into 100 bits. The actual data input from each CIS camera is 105 bits, of which 5 bits are used for synchronization verification. After frame_sync, the data is processed into 100 bits.

[0050] The asyncFifo module is an asynchronous FIFO module used for asynchronous clock domain data interaction in FIFO. The input is 100MHz and the output is 100MHz, which is to match different clock phases.

[0051] The pixel_reorder module is a data parallel-to-serial reordering module. It takes 100 bits of data at a clock frequency of 100MHz as input and outputs 500 bits of data at a clock frequency of 100MHz. Through the parallel-to-serial reordering module, the data from four separate cameras is combined into one for output.

[0052] The pp_pix module is a 4-ping-pong buffer that can store up to 4 frames of data and outputs the data with a delay via ping-pong.

[0053] The Load_date module of the SNN_IP algorithm is responsible for downloading pulse data. The download method includes the following steps:

[0054] Step 1: Retinal stimulation sampling

[0055] In a pulse camera, light intensity is converted into voltage by a photosensor. Once the analog-to-digital converter (ADC) completes the signal conversion and outputs a digital brightness intensity, the accumulator at each pixel accumulates the intensity. Different brightness intensities result in different accumulation rates. For a single pixel, if the accumulated intensity reaches a distribution threshold... This triggers a spike, indicating that the brightness at that point is sufficiently high. Simultaneously, the corresponding accumulator resets, in which all charge is discharged.

[0056]

[0057] A pixel array is constructed, with a photoreceptor and its corresponding accumulator set in each pixel. Output and reset are triggered asynchronously. At each sampling moment, if a spike has just been excited, a digital signal "1" is output; otherwise, "0" is generated. Compared to traditional cameras, the Spike camera only cares about brightness. This is because the accumulation rate of brightness intensity varies greatly across different pixels; the patterns of the scheduled spike sequences also differ from one another. For brighter pixels, "1" is displayed more frequently than for darker ones. This idea is easy to explain: brighter pixels show more photons collected at the pixel, leading to a larger ADC value that exceeds the scheduling threshold more easily and quickly. Based on this principle, the peak patterns of texture analysis can be reconstructed. The response of ganglion cells is very similar to that of the object's contours, as demonstrated by the decoding delay.

[0058] Step 2: Image Texture Acquisition

[0059] By replaying the spike signals, historical footage can be depicted. In the TFP algorithm, there exists a time window for collecting spikes within a specific period. By counting these spikes, the texture is calculated as follows:

[0060]

[0061] like Figure 3 As shown, the size of the time window is w, t i N represents the previous w moments. w This is the total number of pulses collected within the time window. C refers to the maximum dynamic range for reconstruction. A larger size means a longer playback period and includes more spikes. The time window size is set to the dispatch threshold. At that time, the texture is accurately reconstructed. Furthermore, the TFP method can recover textures with different dynamic ranges by adjusting the time window size to different contrast values.

[0062] The `trans_date` module of the SNN_IP algorithm is responsible for pulse data storage and transformation. It optimizes the data storage method for captured pulse pixels to facilitate reconstruction in the next module, `recon`, and also improves implementation speed.

[0063] The recon module of the SNN_IP algorithm is responsible for the algorithmic reconstruction of pulse data. This invention uses an algorithm that reconstructs one frame from seven frames, determining the time window size to be 7, i.e., w = 7. (An example of a time window is shown below.) Figure 6 As shown, the number of pulses ("1") is determined by statistically analyzing pulse data from a time neighborhood of 7 frames. The estimated reconstructed pixel value is then determined based on the count of these "1"s. The reconstructed pixel is an 8-bit grayscale pixel. An example image is shown below. Figure 7 As shown. Furthermore, this module employs parallel and pipelined design. For example... Figure 8 As shown, in the input section, the storage array is partitioned into 7 arrays for parallel input. In the output section, 8 pixels are output in parallel per cycle. In the reconstruction phase, row pipelining is used, which saves hardware resources and greatly reduces the time required for implementation.

[0064] MIPI_OUT Module: This invention uses the MIPI protocol as the data output interface for the entire reconstruction module. Considering system stability, Xilinx's mipi_csi_tx IP is used as the physical interface of this invention. During the system initialization phase, the internal registers of the mipi_csi_tx IP need to be initialized and configured through the axi_lite interface. After configuring the number of lanes and image data format of the MIPI interface, the corresponding MIPI rate is calculated to achieve read and write rate matching. After the reconstructed single-frame image data is stored in DDR, the data stream in axi_stream format will flow into the mipi_csi_tx module through VDMA, thereby realizing the output of the reconstructed image.

[0065] Each module operates independently yet is controlled by the SOC, including the Capture module, SNN_IP module, and MIPI_OUT module. They all operate in a pipeline by continuously receiving control signals from the SOC and systematically configuring different addresses through the axi_lite protocol. The pipeline structure is as follows: Figure 9 As shown.

[0066] The circuit design selected 2GB of DDR4 memory. Since the data storage, reconstruction, and output processes are roughly dependent on the following data relationship: data is stored first, then reconstructed, and then output. Therefore, the pipeline is designed in this order. The address configuration is as follows: Capture sequentially stores data at sixteen consecutive addresses; SNN reconstruction reads data sequentially from the Capture module and then writes it to sixteen consecutive addresses for storage; MIPI output reads the results of SNN reconstruction sequentially and outputs them. Regarding pipeline setup, since there are no images to reconstruct and output sequentially at the beginning, the pipeline has two setup times: the first stage only performs Capture data storage, and the second stage performs the first reconstruction outside of storage. After the second stage, all three processes run in parallel. This simultaneously drives all modules, achieving synchronous acquisition and output.

[0067] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for visual reconstruction of pulse video signals, the method being executed on pulse video signal visualization FPGA hardware, the FPGA hardware including a Capture module, an SNN_IP algorithm module, and a MIPI_OUT module; the Capture module is used to store and preprocess pulse signals captured by a camera; the SNN_IP algorithm module is used to reconstruct grayscale pixel data from the preprocessed pulse data using the TFP algorithm; the MIPI_OUT module is used to output the reconstructed pixel data in parallel; the Capture module includes a Capture_top module and a storage module, the Capture_top module being directly connected to four external channels. The camera sensor captures data from four cameras, reassembles the data into complete and error-free data based on the position of the frame header, stores it in ping-pong RAM, and finally transmits it sequentially to the storage module via the AXI_S protocol. The four camera sensors are of the same model and use the same driver chip. The cameras are configured sequentially using SPI to ensure normal startup. The connection protocol between the cameras and hardware devices adopts the CSI protocol. The Capture module reassembles the data, reconstructs it in LVDS signal form, and then sends it to a data storage module. The storage module reads and writes to DDR4 via the AXI_S protocol in a pipeline-level manner to complete the data acquisition and storage functions of the Capture module. The Capture_top module includes the Capture_sub module, the power_on module, and the spi_core module. The Capture_sub module is used to process the data signals from the CMOS sensor and output them to the synchronous FiFO, and finally output them via AXI. The power_on module is a power-on signal generation module used to generate the power-on timing required by the CMOS sensor. The spi_core module is used to receive the AXI signal from the host and then forward the signal to the SPI signal of the CMOS_sensor to realize the timing conversion function. The Capture module includes seven hardware modules: ser2par, tapUpdateCtrl, delayRstGen, frame_sync, asyncFifo, pixel_reorder, and pp_pix, which are connected sequentially. The ser2par module, tapUpdateCtrl module, and delayRstGen module are used to match the input CIS camera clock and reset, and to convert the input LVDS signal into a normal electrical signal. The frame_sync module is used to process the actual data input from each CIS camera into 100 bits; The asyncFifo module is an asynchronous FIFO module used for asynchronous clock domain data interaction in FIFO, with an input of 100MHz and an output of 100MHz to match different clock phases; The pixel_reorder module is a data parallel-to-serial reordering module. Through the parallel-to-serial reordering module, the data from four separate cameras is combined into one for output. The pp_pix module is a 4-ping-pong buffer that can store up to 4 frames of data and outputs the data with a delay via ping-pong. Its features are, The method uses the TFP algorithm for data reconstruction, and includes the following steps: Step 1: Retinal stimulation sampling In a pulse camera, light intensity is converted into voltage by a photosensor. Once the analog-to-digital converter completes the signal conversion and outputs digital brightness intensity, the accumulator at each pixel accumulates intensity. Different brightness intensities result in different accumulation rates. For a pixel, if the accumulated intensity reaches the distribution threshold φ, a spike is excited, indicating that the brightness at that point is sufficiently high. Simultaneously, the corresponding accumulator resets, where all charge is discharged. ， Construct a pixel array and set a photoreceptor and its corresponding accumulator in each pixel. Output and reset are triggered asynchronously. At each sampling time, if a spike is just excited, output a digital signal "1"; otherwise, generate "0". Bright pixels show more photons collected in the pixel, resulting in a larger ADC value. It is easier and faster to exceed the scheduling threshold. Based on this principle, the peak pattern of texture analysis is reconstructed. Step 2: Image Texture Acquisition By replaying the spike signals, historical footage can be depicted. In the TFP algorithm, there is a time window for collecting spikes within a specific period. By counting these spikes, the texture is calculated as follows: ， The size of the time window is w, t i N represents the previous i time points. w The total number of pulses collected within the time window, and C refers to the maximum dynamic range to be reconstructed. A larger size means a longer playback period and includes more spikes. When the time window size is set to the dispatch threshold φ, the texture is accurately reconstructed. Meanwhile, the TFP method recovers textures with different dynamic ranges by adjusting the time window size to different contrast values.

2. The method according to claim 1, characterized in that, The SNN_IP algorithm module consists of the Load_date module, the Trans_date module, and the Recon module, which are responsible for downloading pulse data, data storage conversion, and algorithm reconstruction, respectively.

3. The method according to claim 2, characterized in that, The Load_date module of the SNN_IP algorithm module is responsible for downloading pulse data; The trans_date module of the SNN_IP algorithm module is responsible for pulse data storage and conversion, and optimizes the data storage method of the captured pulse pixels; The recon module of the SNN_IP algorithm module is responsible for the algorithmic reconstruction of pulse data.

4. The method according to claim 1, characterized in that, The MIPI_OUT module uses the MIPI protocol as the data output interface for the entire reconstruction module and adopts Xilinx's mipi_csi_tx IP as the physical interface. During the system initialization phase, the internal registers of the mipi_csi_tx IP are initialized and configured through the axi_lite interface. After configuring the number of lanes and image data format of the MIPI interface, the corresponding MIPI rate is calculated to achieve read and write rate matching. After the reconstructed single-frame image data is stored in DDR, the data stream in axi_stream format will flow into the mipi_csi_tx module through VDMA to realize the output of the reconstructed image.

5. The method according to claim 1, characterized in that, An algorithm reconstructing one frame from seven frames is employed, with the timewindow size set to 7 (w=7). The number of "1" pulses within the seven-frame time neighborhood is determined statistically, and the estimated reconstructed pixel value is based on this count. The reconstructed pixel is an 8-bit grayscale pixel. Parallel and pipelined design is used. In the input section, the storage array is partitioned into seven parallel arrays. In the output section, eight pixels are output in parallel per cycle. Linear pipelined design is employed during the reconstruction phase. The circuit design utilizes 2GB of DDR4 memory. The dependency relationship between data storage, reconstruction, and output is as follows: data is stored first, then reconstructed, and so on. The output is processed in the same order as the pipeline. The address configuration is as follows: Capture stores data at sixteen consecutive addresses sequentially; SNN reconstruction reads data sequentially from the Capture module and writes it to sixteen consecutive addresses for storage; MIPI output reads the results of SNN reconstruction sequentially and outputs them. In terms of pipeline setup, since there is no valid data at the beginning of reconstruction and output, the pipeline has two setup times. The first stage only performs Capture data storage, and the second stage performs the first reconstruction outside of storage. After the second stage, all three are performed in parallel. In this way, all modules are driven at the same time, realizing synchronous acquisition and output.

Images