Wide-spectrum image acquisition and identification system based on FPGA (Field Programmable Gate Array)

Abstract

The invention discloses a wide-spectrum image acquisition and recognition system based on an FPGA (Field Programmable Gate Array) and belongs to the technical field of crossing of photoelectric sensing and intelligent edge calculation. According to the system, a device array is constructed by utilizing the adjustable photoelectric characteristic of a quantum dot material, wide spectrum sensing of light signals from visible light to a near infrared band is realized, and multi-channel photoelectric signals generated by the array are directly introduced into an FPGA platform near a sensing end to be processed. Through integrating functional modules of analog signal acquisition, analog-to-digital conversion, data caching, neural network reasoning and the like in the FPGA, cooperative operation and integrated processing of optical signal acquisition, data conversion and intelligent identification are realized. Compared with an identification scheme based on a general processor, the FPGA parallel computing and reconfigurable advantages are brought into full play, the data transmission overhead is reduced, the speed is increased, and the overall power consumption is reduced. The system is compact in structure, high in integration level and capable of completing real-time sensing and digital recognition of wide-spectrum optical signals on the edge side.

Description

Technical Field

[0001] This invention relates to the fields of FPGA, quantum dot detector arrays, artificial neural networks, and edge image acquisition and recognition, specifically an FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array. Background Technology

[0002] In the field of perception, such as autonomous driving in complex environments, short-wave infrared (SWIR, 0.9-1.7 μm) imaging exhibits unique advantages in environments such as fog and low light by analyzing the reflection / absorption characteristics of a target to photons of specific wavelengths. However, silicon-based detectors are limited by a 1.12 eV bandgap (cutoff wavelength λ). c The effective response range is typically limited to 1000 nm, meaning traditional silicon processes cannot meet the requirements of SWIR imaging. Quantum dot (QD) devices, through size-dependent quantum confinement, can tune the response band to the SWIR range. Furthermore, traditional SWIR image processing relies heavily on general-purpose processors like CPUs and GPUs. Limited by the von Neumann architecture's separation of computation and storage, running intelligent algorithms like neural networks often suffers from high power consumption, large data transmission overhead, and insufficient real-time performance, making it difficult to meet the low-power and fast-response requirements of edge computing. In contrast, FPGAs, with their highly parallel hardware architecture and excellent reconfigurability, can be deployed near the quantum dot imaging array to achieve high-speed acquisition of output signals and accelerated intelligent computing, matching the low-latency requirements of edge sensing. Summary of the Invention

[0003] To address this, the present invention provides an FPGA-based image acquisition and recognition system based on a wide-spectrum quantum dot detector array, enabling high-speed acquisition and intelligent computational recognition of the detector array's output signals. The system first constructs a 4x4 wide-spectrum response detector array by coupling all-inorganic metal halide perovskite (CsPbBr3) with narrow-bandgap lead sulfide (PbS) quantum dots, where each detection unit corresponds to an independent pixel in the digital image. Using an FPGA as the main control chip, analog switches are used to time-division select the signals of each column of the array. The selected current signals are converted to voltage by operational amplifiers and amplified, then digitized by an analog-to-digital converter (ADC) and stored in the FPGA. A neural network (ANN) model pre-deployed on the FPGA performs real-time recognition of the digital signals, and the recognition results are displayed on a digital tube. Simultaneously, the current data of each array unit is uploaded to a host computer via a serial port for analysis. This system can acquire, convert, and recognize 4x4 pixel digital image signals.

[0004] To achieve the above system, the present invention achieves its purpose through the following technical solution: The FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array of the present invention mainly includes two PCB boards, one of which is the main board, which includes an FPGA chip, an AD conversion module, a digital tube display module and a serial port module; the other is the sub-board, which includes an analog switch module, an operational amplifier module, a power supply module, and a 4x4 quantum dot device array, which are bonded to the sub-board by a pressure bonding process.

[0005] Furthermore, the main control FPGA chip is manufactured by Xilinx, specifically model XC7A35TFGG484-2; an analog switch can control four signals, respectively controlling the reading of electrical signals from different devices on the chip, specifically model MAX4634; the operational amplifier is manufactured by Texas Instruments, capable of operational amplification of four electrical signals, specifically model OPA4192; the AD conversion chip is manufactured by Analog Devices Inc., capable of up to eight channels of analog-to-digital conversion, specifically model AD7606; serial communication uses an RS232 interface; power supply uses dedicated power chips to provide ±5V, with the +5V power supply chip manufactured by LINEAR TECHNOLOGY, model LTC1522; and the -5V power supply chip manufactured by LINEAR TECHNOLOGY, model LTC1983-5.

[0006] Furthermore, the structure of the quantum dot device is as follows: a source electrode and a drain electrode are disposed on a silicon oxide wafer. On the silicon oxide wafer, between the source electrode and the drain electrode, from bottom to top, are disposed the following: a monolayer graphene film, which serves as a conductive layer to provide a carrier transport channel; a PbS-TBAI quantum dot film, wherein TBAI is tetrabutylammonium iodide, used to perform surface ligand exchange on the PbS quantum dots to change their electrical transport properties; a CsPbBr3 quantum dot film, used for light absorption and photogenerated carrier generation; and a Parylene film, which serves as an encapsulation and protective layer to isolate water and oxygen and improve the environmental stability of the device.

[0007] Furthermore, both the source and drain electrodes are Ti / Au, the graphene film is a single-layer graphene film, and it is transferred onto a silicon oxide wafer in deionized water; the PbS-TBAI quantum dot film and the CsPbBr3 quantum dot film are prepared by spin coating, wherein the PbS-TBAI quantum dot film has 3 layers, the CsPbBr3 quantum dot film has 1 layer, and the Parylene film has 1 layer.

[0008] Furthermore, the neural network model is an ANN neural network with a structure of 16 neurons in the input layer, 1 hidden layer containing 12 hidden neurons, and 4 neurons in the output layer. The neural network is trained using Matlab software. After training is complete and the recognition accuracy reaches over 95%, the weights and biases of the neural network are extracted and saved. Subsequently, the neural network model containing the weights and biases is deployed on an FPGA.

[0009] Furthermore, the 16 neurons in the input layer of the neural network represent 16 devices, and the 4 neurons in the output layer represent 4 recognition results (0, 1, 4, 7).

[0010] Furthermore, the digital pixel size for digital recognition is 4x4. The specific method of recognition is to project the 4x4 digital image onto a 4x4 device array using a mask, so that each device represents a pixel, thereby performing recognition.

[0011] Furthermore, the FPGA development uses Vivado, the hardware description language uses Verilog, and the PCB design uses Altium Designer.

[0012] Compared with the prior art, the present invention has at least the following beneficial effects:

[0013] (1) Compared with traditional computer architecture, FPGA completes data storage and operation within the same system, reducing the waste of resources caused by data transmission and significantly improving the efficiency of neural network recognition.

[0014] (2) A photoelectric device with a wide spectral response range was prepared by using two quantum dot materials, PbS-TBAI quantum dots and CsPbBr3 quantum dots, to achieve a wide spectral response of the device to light in the visible to near-infrared band (520-1550 nm).

[0015] (3) The image acquisition and recognition system of the present invention integrates data acquisition, recognition and display functions. It is simple to operate and easy to use. Users only need to send commands to it using the host computer software. No other operations are required. The system response delay is about 0.02 s, and the operation redundancy is reduced by 90%, which meets the ISO 9241-110 human-computer interaction efficiency standard. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of a single quantum dot device of the present invention;

[0017] Figure 2This invention relates to an FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array;

[0018] Figure 3 This is the absorption spectrum of the PbS quantum dots of the present invention;

[0019] Figure 4 The current curves and PPF effect of the quantum dot device of the present invention under 1550 nm light are shown.

[0020] Figure 5 This is a schematic diagram of the quantum dot device array of the present invention;

[0021] Figure 6 This is a schematic diagram of the ANN neural network structure of the present invention;

[0022] Figure 7 A mask for the digital image of this invention;

[0023] Figure 8 This is an image showing the recognition result of the image acquisition and recognition system of the present invention for the digit "7". Detailed Implementation

[0024] To make the technical solutions of the present invention clearer, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0025] Obviously, the following embodiments are all embodiments of a part of the present invention, intended to further illustrate the solutions and technologies of the present invention, and are not all embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive step are within the scope of protection of the present invention.

[0026] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary implementations of this disclosure. As used herein, when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art, unless the context otherwise requires. Embodiments will now be described in detail with reference to the accompanying drawings.

[0028] This invention provides an FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array, the structure of which is as follows: Figure 1 As shown, the specific implementation process is as follows:

[0029] (1) First, two quantum dot materials, PbS-TBAI quantum dots and CsPbBr3 quantum dots, were used to fabricate a 4x4 device array. The structure of a single device is as follows: Figure 2 As shown, the device array structure is as follows Figure 3 As shown.

[0030] (2) Construct an ANN neural network using Matlab, which contains three layers: an input layer, one hidden layer, and an output layer. The sigmoid function is used as the activation function. The specific network structure is as follows: Figure 4 As shown, digital images are used as the dataset for training, which is performed until the recognition rate reaches over 95%. After the ANN neural network is trained, the weights and biases are saved for later deployment on the FPGA.

[0031] (3) Connect the system hardware. First, connect the motherboard's power connector, JTAG program download interface, and serial port interface to the computer in sequence. Then, connect the secondary board's power supply interface to the 5V output power connector. Use DuPont wires to connect the outputs of the four operational amplifiers to the A1, A2, A3, and A4 channels of the AD7606 module in sequence. Then, use DuPont wires to connect the control ports of the analog switches to the GPIO ports on the motherboard. Finally, use a clamp to fix the mask with the digital pattern directly above the device array soldered to the secondary board PCB, so that the digital pattern is aligned with the corresponding device. At this point, the hardware connection is complete.

[0032] (4) Next, turn on the power to the main board and the sub-board, and download the program written in Verilog language to the FPGA.

[0033] (5) The FPGA receives the command from the user to configure the AD7606 chip via the serial port. After analysis and decoding, the FPGA configures the AD7606 chip to have 4 sampling channels, a conversion rate of 200 Ksps, and 16 sampling data points according to the command. After configuration, it sends the command to start sampling again.

[0034] (6) After sampling begins, the FPGA controls the acquisition of current signals from devices at different positions in the device array via analog switches. The current signal output from each device array is converted into a voltage signal by a 1 KΩ resistor. The voltage signal is amplified by the OPA4192 operational amplifier and output to the AD7606. The FPGA controls the AD7606 to complete analog-to-digital conversion, converting the voltage signal from analog to digital and finally storing it in the FPGA. When a device array signal is output, amplified, converted and stored, the AD7606 chip will output a Convert_done signal to inform the FPGA that this signal has been acquired and can control the analog switches to acquire the electrical signal of the next device array.

[0035] (7) After the electrical signals of each device on the 4x4 device array are collected and stored in the internal storage unit of the FPGA, the FPGA retrieves the stored digital electrical signals and transmits one path to the neural network recognition already deployed on the FPGA, and sends the recognition result to the digital tube module for display; the other path is sent to the serial port transmission module, which transmits the electrical signal data of the device array back to the PC host computer software.

[0036] (8) Users can view the results of ANN neural network recognition of numbers through digital tube display, and can also view the electrical signal value of each device on the device array through host computer software. At this time, the electrical signal value is a hexadecimal digital quantity. If the analog quantity of device current signal needs to be known, it needs to be converted according to the range and data bit width of AD7606.

[0037] Figure 1 The structure of a single device is mainly shown. There are source and drain electrodes on both sides of the silicon oxide wafer. Between the source and drain electrodes on the silicon oxide wafer, from silicon oxide upwards, are monolayer graphene film, PbS-TBAI quantum dot film, CsPbBr3 quantum dot film, and Parylene film.

[0038] Figure 2This paper mainly illustrates the structure and logic of an FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array. The entire system consists of eight parts: PCB_Control, PCB, AD7606_Control, FIFO, ANN_Recognize, hex_disp, UART_RX, and UART_TX. The UART_RX module receives commands from the PC and sends them to the FPGA, enabling it to configure the AD7606 and PCB_Control modules. The PCB_Control module controls the analog switches on the secondary PCB board by receiving the Scan_Start signal from the AD7606_Control module and the Convert_done signal from the UART_RX module, thereby reading the current signals of different devices on the device array. The read device current signals are converted into voltage signals by the OPA4192 operational amplifier, amplified, and finally transmitted to the AD7606 for analog-to-digital conversion. The AD7606_Control module's main function is to control the AD7606 chip to acquire the current signals. The analog voltage signal is converted into a digital voltage signal and buffered in a FIFO. The UART_TX module is responsible for retrieving the data from the FIFO and sending it to the PC, allowing users to view the electrical signals of each device through the host computer software. The ANN_Recognize module retrieves the data buffered in the FIFO and feeds it into the neural network for recognition. After recognition, it sends the recognition result to the hex_disp module. The hex_disp module is responsible for sending the recognition result to the digital tube for display. If the recognition is correct, all eight digital tubes will display the corresponding recognition result, such as "00000000" or "77777777". If the recognition is incorrect, all eight digital tubes will display "FFFFFFFF".

[0039] Figure 3 The main image shows the absorption spectrum of PbS quantum dots. It can be seen that the absorption band of PbS quantum dots covers the visible to near-infrared range, with an absorption peak at 1500 nm. This proves that near-infrared response of devices can be achieved using PbS quantum dot materials.

[0040] Figure 4 This mainly demonstrates the photocurrent and PPF effect of quantum dot devices in the 1550 nm infrared band.

[0041] Figure 5 This mainly illustrates the structure of a 4x4 device array. The device array is read in a column-wise manner, that is, the first four devices in the first column are read first, then the four devices in the second column are read, and so on.

[0042] Figure 6The diagram mainly shows the constructed ANN neural network structure. The network consists of three layers: an input layer, a hidden layer, and an output layer. The input layer has 16 neurons, each representing a device on a 4x4 array. The input layer is mapped to the hidden layer using the Sigmoid function. The hidden layer has 12 neurons, which are mapped to the hidden layer output layer using the Sigmoid function. The output layer has 4 neurons, representing the four digit recognition results: "0", "1", "4", and "7".

[0043] Figure 7 This is a digital image mask made of steel. When using it, the image on the mask needs to be aligned with the device array. The mask is printed with digital images of "0", "1", "4", and "7", which are used to project the corresponding digital images onto the device array.

[0044] The secondary PCB board is powered by 5V and connected to the GPIO ports on the FPGA motherboard via DuPont wires. It mainly consists of four parts: A, B, C, and D. Part A is the power supply module, responsible for providing +5V, -5V, and +1.2V voltages; Part B is a 4-channel operational amplifier OPA4192, responsible for converting and amplifying the current signals output from the device array; Part C is a 4-channel analog switch MAX4634, responsible for controlling which column of the device array's signal is acquired; and Part D is a 4x4 device array soldered to the secondary PCB board.

[0045] The image acquisition and recognition system simulation uses the Vivado built-in simulation platform. The entire system is simulated by writing a Testbench file. The waveform generated by the simulation shows that the Testbench uses the analog number "7" to transmit electrical signal data from 16 devices on the device array. After buffering, the data is recognized and the final recognition result is displayed as "77777777".

[0046] Figure 8 The actual effect diagram of the FPGA image acquisition and recognition system based on a wide-spectrum response quantum dot device array is demonstrated. As can be seen, after a series of operations such as acquisition, amplification, AD conversion, storage, recognition and display of the device array, the recognition result "77777777" is finally displayed on the digital tube.

Claims

1. An FPGA image acquisition and recognition system based on a broadband quantum dot detector array, characterized in that, First, a 4x4 broadband response detector array is constructed by coupling an all-inorganic metal halide perovskite (CsPbBr3) with narrow-bandgap lead sulfide (PbS) quantum dots, where each detector unit corresponds to an independent pixel in the digital image. An FPGA is used as the main control chip, and analog switches are used to time-division select the signals of each column of the array. The selected current signal is converted into voltage by an operational amplifier and amplified, and then digitized by an analog-to-digital converter (AD) and stored in the FPGA. Digital signals are identified in real time based on a neural network (ANN) model pre-deployed on the FPGA. The identification results are then displayed on a digital tube, and the current data of each unit in the array is uploaded to the host computer for analysis via a serial port.

2. The FPGA image acquisition and recognition system based on a wide-spectrum quantum dot detector array as described in claim 1, characterized in that, It includes two PCB boards. One is the main board, which contains an FPGA chip, an AD conversion module, a digital tube display module, and a serial port module. The other is the secondary board, which contains an analog switch module, an operational amplifier module, a power supply module, and a 4x4 quantum dot device array that is bonded to the secondary board using a pressure bonding process.

3. The FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 2, characterized in that, An analog switch can control four signals, which in turn control the reading of electrical signals from different devices on the chip; an operational amplifier can amplify four electrical signals; and an AD conversion chip can perform analog-to-digital conversion on up to eight channels. Serial communication uses an RS232 interface; power supply is provided by a dedicated power chip with ±5V.

4. The FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 1, characterized in that, The structure of the quantum dot device is as follows: a source electrode and a drain electrode are disposed on a silicon oxide wafer. On the silicon oxide wafer, between the source electrode and the drain electrode, from bottom to top, are disposed the following: a monolayer graphene film, which serves as a conductive layer to provide a carrier transport channel; a PbS-TBAI quantum dot film, wherein TBAI is tetrabutylammonium iodide, used for surface ligand exchange of PbS quantum dots to improve electrical transport performance; a CsPbBr3 quantum dot film, used for light absorption and photogenerated carrier generation; and a Parylene film, serving as an encapsulation and protective layer.

5. An FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 4, characterized in that, In the quantum dot device, both the source and drain electrodes are Ti / Au, and the graphene film is a single-layer graphene film transferred onto a silicon oxide wafer in deionized water. The PbS-TBAI quantum dot film and the CsPbBr3 quantum dot film are prepared by spin coating, wherein the PbS-TBAI quantum dot film has 3 layers, the CsPbBr3 quantum dot film has 1 layer, and the Parylene film has 1 layer.

6. The FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 1, characterized in that, The neural network model is an ANN neural network with an input layer of 16 neurons, a hidden layer of 12 neurons, and an output layer of 4 neurons. The training of the neural network is completed using Matlab software. After the neural network is trained and the recognition accuracy reaches more than 95%, the weight parameters and bias parameters of the neural network are extracted and saved. Then, the neural network model containing the weight parameters and bias parameters is deployed on an FPGA.

7. An FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 1, characterized in that, The 16 neurons in the input layer of the neural network represent 16 devices, and the 4 neurons in the output layer represent 4 recognition results (0, 1, 4, 7).

8. An FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 1, characterized in that, The digital pixel size of the digital recognition is 4x4. The specific recognition method is to project the 4x4 digital image onto a 4x4 device array using a mask, so that each device represents a pixel.

9. An FPGA image acquisition and recognition system based on a broadband quantum dot detector array as described in claim 1, characterized in that, The FPGA development uses Vivado, the hardware description language uses Verilog, and the PCB design uses Altium Designer.

Images