A digital hardware implementation device based on an image fusion algorithm and a method thereof

Abstract

This invention proposes a digital hardware implementation device and method based on an image fusion algorithm. The architecture includes a pixel array, a system control module, a driver module, and a data readout module. The pixel array is used for real-time photosensitive imaging of a first and second image, and outputs the image pixel information as voltage or current signals. The system control module is responsible for the chip's system control and on-chip / off-chip communication, and performs full-chip information configuration. The driver module provides row-dimensional pixel drive signals and column-dimensional pixel readout drive signals to the array. The data readout module converts the voltage or current generated by the array into quantized information, and stores, extracts, compresses, and caches the quantized information, ultimately transmitting it to the external chip via a port. This invention provides an efficient, reliable, and configurable on-chip design scheme for image fusion algorithms, enabling complete on-chip mapping of the image fusion algorithm.

Description

Technical Field

[0001] This invention proposes a digital hardware implementation device and method based on an image fusion algorithm, belonging to the field of integrated circuits. Background Technology

[0002] Image fusion algorithms require on-chip implementation of alternating driving, imaging, and data readout of Image 1 and Image 2. If a large-scale pixel requirement is also present, it's impossible to drive an ultra-large image array using simple off-chip direct control or writing state control to on-chip registers. A built-in high-performance hardware implementation based on the image fusion algorithm is necessary. This is because: firstly, there are limits to the number of internal and external serial ports and clock frequency, creating a bottleneck in data bandwidth. Therefore, off-chip complexity needs to be moved on-chip to reduce bandwidth requirements, while a high-performance interface is required to meet the data output bandwidth demands of the ultra-large image array; secondly, the fusion algorithm needs to alternate between real-time dual modes. Imaging requires high functionality, necessitating coordinated operation of various modules to achieve simultaneous function switching and correctly drive the image array into the corresponding working state. Otherwise, data misalignment and timing-related image noise will occur. The generated data also needs to be output to external devices quickly; otherwise, data stacking will occur, leading to timing chaos and data loss. Therefore, a reasonable architecture design is required to optimize timing. Finally, if a large number of external signals are introduced, noise will severely interfere with the system. The hardware needs to be compatible with both Image 1 and Image 2, and must meet all parameter design specifications for both. Therefore, the noise tolerance is low, and using more external input signals will directly reduce image output performance.

[0003] The parallel-to-serial conversion interface proposed in Chinese patent application CN202110522201.8, "Methods, Apparatus and Computer-Readable Medium for Imaging," cannot handle large-scale pixel output data because: firstly, to achieve the reading of large-scale pixel data, buffering, rearrangement, compression, and high-speed readout operations are required, while the parallel-to-serial conversion module can only achieve simple readout timing and has a single function; secondly, the large amount of large-scale pixel data is enormous, and without high-speed interface and compression processing, the data cannot be output to the off-chip within the specified time, which will lead to data loss.

[0004] For the reasons mentioned above, it is of great significance to design an on-chip hardware implementation device and method that can realize image fusion algorithm. Summary of the Invention

[0005] The purpose of this invention is to provide a digital hardware implementation device and method based on an image fusion algorithm, thereby making the on-chip implementation of the image fusion algorithm reliable and efficient.

[0006] The technical solution adopted by the architecture of this invention is as follows:

[0007] A digital hardware implementation device based on an image fusion algorithm includes a pixel array, a system control module, a driving module, and a data readout module. The pixel array, composed of pixel devices, is used for real-time photosensitive imaging of a first image and a second image, and outputs the image pixel information as voltage or current signals. The system control module is used for on-chip and off-chip communication, configuring the information of the driving module and the data readout module. The driving module provides the pixel array with row-dimensional pixel control signals and column-dimensional pixel readout driving signals required for imaging. The data readout module converts the voltage or current signals output by the pixel array into digital quantized information, stores, extracts, compresses, and caches the quantized information, and finally transmits it to the outside of the chip through a port.

[0008] Furthermore, the system control module includes a system control top-level module, an on-chip slave module, and an on-chip master module. The system control top-level module includes a register group. The on-chip slave module uses the SPI protocol to configure and read all register parameters that interact with external devices. The on-chip master module uses the APB protocol to configure data for the mobile industry processor interface. The register group is connected to both the on-chip slave module and the on-chip master module.

[0009] Furthermore, the driving module includes a temperature sensor module, a phase-locked loop module, a bias module, a ramp generation module, a column drive generation logic module, and a row drive generation logic module. The temperature sensor module is used to collect on-chip temperature signals and transmit them off-chip. The phase-locked loop module is used to generate a high-frequency clock signal to improve data bandwidth tolerance. The bias module is used to generate the voltage and current bias signals required for the entire chip. The column drive generation logic module and the row drive generation logic module are used to provide column drive state voltages and row drive state voltages to the pixel array, respectively. The ramp generation module outputs a ramp drive signal and transmits it to the pixel array along with the row drive state voltages and column drive state voltages.

[0010] Further, the data readout module includes an analog readout module, a digital counter module, a first image data rearrangement module, a second image data rearrangement module, a second image data compression module, an automatic test pattern generation module, a first-in-first-out (FIFO) data buffer module, a MIPI interface module, and a MIPI module. The analog readout module receives the voltage or current signal output by the pixel array and converts it into digital quantized information, then connects to the digital counter module and the second image data compression module. The digital counter module converts the first image digital level signal into first image quantized digital bit information, then connects to the first image data rearrangement module. The first image data rearrangement module stores and rearranges the first image quantized digital bit information, then connects to the FIFO data buffer module. The second image data rearrangement module stores and rearranges the second image digital level signal into quantized digital bit information, then connects to the FIFO data buffer module. The level signal is converted into second image quantized digital bit information, and data storage and rearrangement are realized. This is followed by a second image data compression module and a first-in-first-out (FIFO) data buffer module. The second image data compression module is used to compress the image quantized digital bit information, followed by the FIFO data buffer module. The automatic test pattern generation module is used to automatically generate test patterns to verify the functionality of the FIFO data buffer module, MIPI interface module, and MIPI module, followed by the FIFO data buffer module. The FIFO data buffer module is used for data buffering, followed by the MIPI interface module. The MIPI interface module is used to package data into the data interface type required by MIPI, and add enable signals required by various MIPI protocols, followed by the MIPI interface module. The MIPI module is used to achieve high-speed data transmission.

[0011] The present invention also provides a method for operating a digital hardware implementation device based on an image fusion algorithm, comprising the following steps:

[0012] S1, the system control module establishes data communication with the external chip to realize data interaction;

[0013] S2, the entire chip enters standby mode, that is, the digital part is in standby mode, the on-chip analog power supply is turned on normally, the high-frequency clock is turned on, and the temperature drive is turned on.

[0014] S3, a global trigger signal is input from outside the chip, the mode of the first image is configured first on the chip, and the operation is started. The driving module starts to generate the driving signal corresponding to the first image. The pixel array receives this driving signal and enters the corresponding working mode, and at the same time generates the analog current and voltage signal corresponding to the first image.

[0015] S4, according to the timing of each line read out, performs digital quantization on the analog current and voltage signals corresponding to the first image, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data buffering, and high-speed output of the first image data.

[0016] S5, upon receiving the first image output end signal, configure the working mode of the second image, and input a global trigger signal from outside the chip;

[0017] S6, the driving module starts to generate a driving signal corresponding to the second image, the pixel array receives this driving signal and enters the corresponding working mode, and at the same time generates an analog current and voltage signal corresponding to the second image;

[0018] S7 performs digital quantization on the analog current and voltage signals corresponding to the second image according to the timing of each line read out, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data compression, data buffering and high-speed output of the second image data.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] (1) The architecture proposed in this invention can first realize all the complex imaging functions of larger-scale pixels, and is compatible with two image output functions, thus solving the problem that existing architectures can only handle the simple imaging functions of small pixel arrays.

[0021] (2) The compression part in the data readout module of the present invention can perform specific hardware data compression for image fusion algorithms, and the overall architecture of the data readout module adopts a high-pipeline timing method, which improves the parallelism of the module operation. Combined with the high-speed output interface module, it can solve the problem of big data bandwidth data stream transmission, transmit alternating image data to the outside in a timely manner, ensure the normal operation of the system, and improve the frame rate index of the image sensor.

[0022] (3) The architecture of the present invention can ensure high-quality output of images and improve the reliability of data transmission. This is because higher bit values ​​are used to quantize pixel gray values. At the same time, an independent automatic test pattern generation module is added to the data readout module to ensure the reliability of the hardware path. In contrast, the existing architecture only provides simple verification processing for each line, and the verification scheme is monotonous and cannot be adjusted. Attached Figure Description

[0023] Figure 1 This is an overall structural diagram of a digital hardware implementation device based on an image fusion algorithm according to an embodiment of the present invention;

[0024] Figure 2 This is a structural diagram of the system control module in an embodiment of the present invention;

[0025] Figure 3 This is a structural diagram of the driver module in an embodiment of the present invention;

[0026] Figure 4 This is a structural diagram of the data reading module in an embodiment of the present invention;

[0027] Figure 5 This is a schematic diagram of the working timing of the data reading module when outputting image 1 in an embodiment of the present invention. Detailed Implementation

[0028] This embodiment provides a digital hardware implementation device based on an image fusion algorithm. The image fusion algorithm is based on the algorithm proposed in CN202110522201.8, "Methods, Apparatus and Computer-Readable Media for Imaging," and includes: capturing and quantizing a target field of view at a first resolution to obtain a first image with a first bit width; capturing and differentially processing the target field of view at a second resolution to obtain a second image with a first bit width, wherein the differential processing includes: quantizing the difference between a pixel captured at the second resolution and its adjacent or nearby pixels to obtain the quantized difference as the value of the corresponding pixel in the second image; and fusing the first image and the second image to obtain a third image, wherein the first resolution is lower than the second resolution, and the first bit width is higher than the second bit width.

[0029] The architecture of this embodiment includes a pixel array, a system control module, a driver module, and a data readout module. The system control module is responsible for the overall data interaction with external components, storing configuration information, and allocating configuration parameters according to the needs of the driver module and the data readout module to realize data interaction 1 and data interaction 2. The driver module needs to connect to the data interaction 1 path to realize the timing configuration of the driver module. After configuration, it waits for the start-up trigger signal to provide the pixel array with row-dimensional pixel drive signals and column-dimensional pixel readout drive signals to drive the pixel array. The pixel array, composed of pixel devices, is responsible for real-time light sensing. It needs to receive the corresponding drive signals of WL (WordLine) and BL (BitLine) for pixel control and output pixel information in the form of voltage or current. The data readout module needs to connect to the data interaction 2 path to realize the timing configuration of the data readout module. After configuration, it starts synchronously with the driver module in conjunction with the start-up trigger signal to transmit the data stream of each row to the external component via the high-speed MIPI (Mobile Industry Processor Interface) protocol.

[0030] Specifically, the architecture block diagram of this embodiment is as follows: Figure 1As shown in the diagram, Sys_Control represents the system control module, Driver represents the driver module, DataReadout represents the data readout module, PixelArray represents the pixel array module, DataInteraction1 represents the configuration write parameters of the system control module to the driver module, including the working status parameters transmitted back to the system control module by the driver module, DataInteraction2 represents the data interaction between the system control module and the data readout mode, DataInteraction represents the data interaction between the on-chip system and the off-chip system, WL Driver represents the row drive signal output by the row drive module in the driver module, BL Driver represents the column drive signal output by the column drive module in the driver module, AnalogPixelSignal represents the array analog signal generated by the row and column drive signals, ImageData represents the image data generated by the data readout module, the black interactive arrow represents communication with the off-chip system, and the black interface represents connection to the off-chip system.

[0031] The system control module block diagram is as follows: Figure 2 As shown in the diagram, Sys_Top represents the top-level system control module in the system control module, which internally includes a register bank; SPI_Slave represents the on-chip slave module using the SPI protocol; APB_Master represents the on-chip master module using the APB protocol; and Data Interaction1 and Data Interaction2... Figure 1The descriptions are consistent: CS is the chip select port in the SPI protocol, MOSI is the master output / slave input port in the SPI protocol, MISO is the slave output / master input port in the SPI protocol, SCK is the system clock in the SPI protocol, Psel is the select signal port in the APB protocol, PEnable is the enable signal in the APB protocol, PRData is the read data in the APB protocol, PWData is the write data in the APB protocol, PAddr is the address signal in the APB protocol, and PWrite is the read / write type signal in the APB protocol. Black interactive arrows represent communication with external devices, black interfaces represent connections to external devices, and gray and non-solid arrows represent on-chip data interaction. The system control module includes the Sys_Top module, the SPI_Slave module, and the APB_Master module. The SPI_Slave module is used to configure and read all register parameters for interaction with external devices. After the chip is powered on, the system control module needs to be initialized. First, it reads the fixed data of the test register in the register group via the SPI protocol. After the test is passed, it writes data to the SPI_Slave module in batches. One data packet corresponds to data and address. The data packet is received by the Sys_Top module and stored in the corresponding position in the register group. The registers maintain a continuous output state until all parameters of the chip are configured. At this time, the Register Bank in Sys_Top will be full of data. At this time, data interaction channel 1 and data interaction channel 2 will be continuously open. The configuration parameters will be continuously input to the driver module and the data reading module. After the configuration of the above two modules is completed, the MIPI module also needs to be configured, namely APB_Master. This module is an APB (Advanced Peripheral Bus) adapter bridge built on the SPI module. It can configure MIPI data in the form of SPI (Serial Peripheral interface) input and APB output. This step can be decomposed into SPI address, data writing to the register group, and APB translation function with APB trigger signal to transmit the corresponding configuration information via the APB protocol. The system control module can read the status values ​​of key registers at any stage of the operation of the drive module and the data readout module, store them in the register group, and then read them out to the external chip via SPI, thus enabling flexible determination of the chip's operating status.

[0032] The driver module block diagram is as follows Figure 3 As shown in the figure, Temp Sensor is the temperature sensor module, PLL is the phase-locked loop module, Bias is the bias module, Ramp Generator is the ramp generation module, BLogic is the column drive generation logic, WLogic is the row drive generation logic, and Data Interaction1 and Figure 1The descriptions are consistent; WL Driver and BL Driver are the same. Figure 1 As described in the text, RAMP WL is the ramp drive signal for WL in Image 1 mode, and HVWL is the fixed-level drive signal for WL in Image 2 mode. The drive module includes a Temp Sensor module, a PLL module, a Bias module, a Ramp Gen module, a Blogic module, and a WLogic module. The driver module needs to provide digital and analog drive for the entire chip. First, it needs to establish data interaction with the system control module, writing configuration parameters to all sub-modules. The PLL module is responsible for generating a high-frequency clock for the entire chip to improve data bandwidth tolerance; this clock drives most of the chip's registers. The Temp Sensor is responsible for collecting on-chip temperature signals and transmitting them externally, including the temperature sensor digital drive module and the temperature sensor data processing module. The Bias module is responsible for generating the voltage and current bias signals required by the entire chip. The WLogic module needs to provide a timing-flexible and adjustable row drive state voltage HVWL for the ultra-large-scale pixel array, implementing specific complex functions (such as image 1 state drive, image 2 drive, image 2 windowing drive, etc.). Simultaneously, WLogic needs to transmit state information to the Ramp Gen module, which will output the synchronization ramp drive signal RAMP WL required for image 1 based on the corresponding state timing information. The RAMP WL signal and the HVWL signal together constitute WL. The Driver signal corresponds to the current system function mode (Image 1 full resolution, Image 1 downsampling 2x, Image 2 full resolution, Image 2 windowing, etc.). Before the WLogic module starts working, it needs to receive a trigger signal to jump from standby to working state. This trigger signal is synchronously transmitted to the BLogic module, enabling them to work synchronously. After receiving the trigger signal, the BLogic module generates a column drive signal BLDriver that is aligned with HVWL and has adjustable timing. Finally, the Temperature Sensor module is set as the temperature information drive module. The drive module finally transmits the synchronized WL Driver and BL Driver signals to the pixel array. The pixel array receives the row and column voltages and begins to perform the corresponding mode reset, exposure, and readout of different image modes.

[0033] Data readout module, such as Figure 4As shown in the diagram, the READOUT module is the analog readout module, Counter is the digital counter module, Data1Reorder is the image 1 data rearrangement module, Data2Reorder is the image 2 data rearrangement module, Compress is the image 2 compression module, ATPG is the automatic test pattern generation module, FIFO is the first-in-first-out data buffer module, N is the quantity, MIPI_Interface is the MIPI interface module, MIPI is the mobile industry processor interface module, Analog_Signal, Data Interaction2, and Image Data are consistent with the description in Image 1, AD_CompP and AD_CompN are the data output interfaces of the analog readout module, Data2 is the level data of Image 2, and Data1 is the counter data of Image 1. The data readout module includes the READOUT module, Counter module, Data1Reorder module, Data2Reorder module, Compress module, ATPG module, N FIFO modules, N MIPI_Interface modules, and N MIPI modules. Since this architecture is based on a data fusion algorithm, it needs to read out images 1 and 2, which requires high data bandwidth. Therefore, the MIPI (Mobile Industry Processor Interface) is used as the output. MIPI uses the APB (Advanced Peripheral Bus) protocol for parameter configuration. Therefore, an APB bridge, namely the APB_Master module, is added to the system control module for MIPI control.

[0034] First, the data readout module needs to establish data interaction with the system control module, passing in the configuration parameters of all sub-modules. Second, in accordance with the module's working status, the READOUT module receives analog current and analog voltage signals from the pixel array, converts the analog information into quantized signals for digital circuits, and outputs the comparator quantized signals for Image 1 and Image 2. This module needs to pay attention to noise performance. Since the on-chip circuit design needs to take area into account, this module needs to be reusable and able to provide two types of image readout. Therefore, this module also needs to be compatible with the parameter design specifications of the two types of images and has high requirements for the noise of the input signal, which directly determines the quality of the generated image. The READOUT module represents the information of image 1 as a level time width and the information of image 2 as a level value. The data path of image 1 output by READOUT will be connected to the Counter group. The counter will use the clock to measure the level width to obtain the specific digital bit stream of image 1. This bit stream is passed to the DataReorder1 module to buffer the data of image 1 and perform functional data filtering (such as downsampling by 8x resolution). The data is then packaged and passed to the FIFO module using the on-chip unified transmission protocol. The READOUT module represents the information of image 2 as level values ​​and passes it to the Data2Reorder module for data acquisition and buffering. It also performs functional data filtering (such as full sampling and windowing) and packages and passes the data to the FIFO module and the Compress module using the on-chip unified transmission protocol. To further reduce the data output bandwidth, the data of image 2 is... Based on the characteristics, on-chip data compression is performed. The Compress module receives data packets from the Data2Reorder module, compresses them, and outputs data packets with the original protocol, which are then passed to the FIFO module. To test the functionality and data integrity of subsequent transmission modules, an ATPG module is designed. This module is followed by the FIFO module, MIPI_Interface module, and MIPI module to test the functionality of this data path. The ATPG module generates a custom test pattern and is equipped with an interrupt signal. If the subsequent FIFO data is about to overflow, the ATPG module will enter an intermediate standby mode and continue transmission when the subsequent data path is no longer congested. The four data types of the DataReorder1 module, Data2Reorder module, Compress module, and ATPG module all use the same data interface protocol, so subsequent modules can process the data stream according to the same protocol.

[0035] To address the high data bandwidth issue, it is necessary to increase the data flow rate and improve the parallelism of each module. The timing diagram of the data reading module for Image 1 is as follows: Figure 5As shown in the figure (Row(n)Read is the row reading stage for the nth row, Row(n+1)Read is the row reading stage for the (n+1)th row, READOUT, Counter, Data1Reorder, FIFO, MIPI_Interface, MIPI and Figure 4 (As described in the text), with each line as a data readout cycle, the parallelism of each module is the highest, resulting in the shortest readout time and the highest frame rate. Since the bandwidth requirement of image 1 is smaller than that of image 2, the FIFO will not be full. Higher parallelism may lead to asynchronous operation of different modules, so a FIFO module is added as a buffer for the front and rear data streams. A reasonable FIFO depth is calculated according to the pixel bit scale. N FIFO parallel modules are added to the data readout module to store data of different bit widths at a unified address. The FIFO module can control the data access in real time according to the transmission capacity of the subsequent module. The N FIFO modules pass the data to N MIPI interface modules MIPI_Interface. This module can receive data serial ports adapted to the FIFO and provide appropriate interrupt signals for the FIFO. At the same time, MIPI_Interface can package the data into the data interface type required by MIPI and add the enable signals required by various MIPI protocols. The N MIPI modules need to receive data types that conform to the MIPI protocol and transmit the data at high speed to the off-chip according to the configured working mode.

[0036] This embodiment also provides a method for operating the above-mentioned digital hardware implementation device, the specific steps of which are as follows:

[0037] S1, the system control module needs to establish data communication with the external chip (data interaction includes writing configuration parameters and reading key registers). First, read the information on the chip and establish a handshake mechanism. Then, write the data to be configured and the corresponding address into the chip. After the communication between the chip and the external chip ends, the communication between the chip and the external chip is carried out to realize data interaction 1 and data interaction 2.

[0038] S2, after the on-chip communication is established, the entire chip enters standby mode, that is, the digital part is in standby mode, the on-chip analog power supply is turned on normally, the high-frequency clock is turned on, and the temperature drive is turned on.

[0039] S3, a global trigger signal is input from outside the chip. The chip first configures the mode of image 1 and starts working. The row and column drive module starts to generate the drive signal corresponding to image 1. The pixel array receives this drive signal and enters the corresponding working mode, and at the same time generates the analog current and voltage signal corresponding to image 1.

[0040] S4, according to the timing of each line read out, performs digital quantization on the analog current and voltage signals corresponding to Image 1, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data caching, and high-speed output of Image 1 data operations.

[0041] S5, receive the output end signal of image 1, configure the working mode of image 2, and input a global trigger signal from outside the chip;

[0042] S6, the row and column drive module starts to generate the drive signal corresponding to image 2. The pixel array receives this drive signal and enters the corresponding working mode, and at the same time generates the analog current and voltage signal corresponding to image 2.

[0043] S7 performs digital quantization on the analog current and voltage signals corresponding to Image 2 according to the timing of each line read out, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data compression, data buffering, and high-speed output of Image 2 data operations.

Claims

1. A digital hardware implementation device based on an image fusion algorithm, characterized in that, The device includes a pixel array, a system control module, a drive module, and a data readout module; The pixel array, composed of pixel devices, is used for real-time photosensitive imaging of the first image and the second image, and outputs the image pixel information in the form of voltage or current signals. The system control module is used for on-chip and off-chip communication of the chip, and for configuring information for the driver module and the data readout module. The driving module is used to provide the pixel array with pixel control signals in the row dimension and pixel readout driving signals in the column dimension required for imaging. The data readout module is used to convert the voltage or current signal output by the pixel array into digital quantization information, and to store, extract, compress and cache the quantization information, and finally transmit it to the outside of the chip through the port; The data readout module includes an analog readout module, a digital counter module, a first image data rearrangement module, a second image data rearrangement module, a second image data compression module, an automatic test pattern generation module, a first-in-first-out data buffer module, a MIPI interface module, and a MIPI module, wherein... The analog readout module receives the voltage or current signal output by the pixel array and converts it into digital quantization information, which is then connected to the digital counter module and the second image data compression module. The digital counter module is used to convert the first image digital level signal into first image quantized digital bit information, which is then connected to the first image data rearrangement module. The first image data rearrangement module is used to store and rearrange the quantized digital bit information of the first image, followed by a first-in-first-out data buffer module; The second image data rearrangement module is used to convert the second image digital level signal into the second image quantized digital bit information, and to realize data storage and rearrangement, followed by the second image data compression module and the first-in-first-out data buffer module; The second image data compression module is used to compress the binary digital bit information of the image, and is followed by a first-in-first-out data buffer module; The automatic test pattern generation module is used to automatically generate test patterns to verify the functional correctness of the first-in-first-out data cache module, the MIPI interface module, and the MIPI module, and is then connected to the first-in-first-out data cache module. The first-in-first-out data caching module is used for data caching and is followed by the MIPI interface module. The MIPI interface module is used to package data into the data interface type required by MIPI, and add the enable signals required by various MIPI protocols, and then connect to the MIPI interface module. The MIPI module is used to achieve high-speed data transmission.

2. The digital hardware implementation device based on the image fusion algorithm according to claim 1, characterized in that, The system control module includes a top-level system control module, an on-chip slave module, and an on-chip master module. The system control top-level module includes a register group; the on-chip slave module uses the SPI protocol to configure and read all register parameters that interact with the external chip; the on-chip master module uses the APB protocol to configure data for the mobile industry processor interface; the register group is connected to both the on-chip slave module and the on-chip master module.

3. The digital hardware implementation device based on the image fusion algorithm according to claim 1, characterized in that, The driving module includes a temperature sensor module, a phase-locked loop module, an offset module, a ramp generation module, a column drive generation logic module, and a row drive generation logic module, wherein... The temperature sensor module is used to collect on-chip temperature signals and transmit them to the off-chip; the phase-locked loop module is used to generate high-frequency clock signals to improve data bandwidth tolerance; the bias module is used to generate the voltage and current bias signals required for the entire chip; the column drive generation logic module and the row drive generation logic module are used to provide column drive state voltage and row drive state voltage to the pixel array respectively; the ramp generation module outputs a ramp drive signal and transmits it to the pixel array together with the row drive state voltage and the column drive state voltage.

4. The method for operating a digital hardware implementation device based on an image fusion algorithm as described in claim 1, characterized in that, The method includes the following steps: S1, the system control module establishes data communication with the external chip to realize data interaction; S2, the entire chip enters standby mode, that is, the digital part is in standby mode, the on-chip analog power supply is turned on normally, the high-frequency clock is turned on, and the temperature drive is turned on. S3, a global trigger signal is input from outside the chip, the mode of the first image is configured first on the chip, and the operation is started. The driving module starts to generate the driving signal corresponding to the first image. The pixel array receives this driving signal and enters the corresponding working mode, and at the same time generates the analog current and voltage signal corresponding to the first image. S4, according to the timing of each line read out, performs digital quantization on the analog current and voltage signals corresponding to the first image, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data buffering, and high-speed output of the first image data. S5, upon receiving the first image output end signal, configure the working mode of the second image, and input a global trigger signal from outside the chip; S6, the driving module starts to generate a driving signal corresponding to the second image, the pixel array receives this driving signal and enters the corresponding working mode, and at the same time generates an analog current and voltage signal corresponding to the second image; S7 performs digital quantization on the analog current and voltage signals corresponding to the second image according to the timing of each line read out, and performs analog-to-digital signal conversion, data storage, data segmentation, data rearrangement and extraction, data compression, data buffering and high-speed output of the second image data.

Images