Camera Link camera data acquisition and image recognition system

Abstract

The utility model discloses a CameraLink camera data acquisition and image identification system, which belongs to the technical field of image processing and embedded systems, and is characterized in that a Hiisis 3559 processor and an FPGA (Field Programmable Gate Array) are used as main control chips, and interface circuits for storage, camera data input, image data output, RS-422, RS-232, Ethernet and the like, and power supplies required by the working of the chips are expanded on the basis of the main control chips. The FPGA is responsible for receiving image data of the two paths of CameraLink cameras, analyzing and preprocessing the data and then sending the data to the Hisilicon 3559 processor, and the Hisilicon 3559 processor operates algorithms such as image recognition and tracking after obtaining the image data, analyzes the image data in real time and outputs a recognition result. According to the utility model, the efficient acquisition and preprocessing of CameraLink data are realized through the FPGA, the requirements of high bandwidth and low delay are met, the system integrates the functions of data acquisition, preprocessing, image identification and communication, is suitable for various application scenes, and the real-time acquisition and processing of image data are realized through the cooperative work of the FPGA and the processor.

Description

Technical Field

[0001] This utility model relates to the field of image processing and embedded systems technology, and in particular to a CameraLink camera data acquisition and image recognition system. Background Technology

[0002] With the rapid development of industrial automation, artificial intelligence, and machine vision, the demand for high-resolution, high-frame-rate image acquisition and real-time processing is increasing. CameraLink cameras, due to their high bandwidth and low latency, are widely used in industrial inspection, medical imaging, and other fields. However, the large data volume and high transmission rate of CameraLink cameras make it difficult for traditional image processing systems to meet real-time processing requirements. Furthermore, image recognition algorithms typically require high-performance computing platforms, and a single processor often struggles to handle both data acquisition and the real-time processing of complex algorithms.

[0003] In existing technologies, FPGAs are typically used for data acquisition and preprocessing, followed by image recognition using a processor. However, existing solutions have shortcomings in terms of data throughput, processing efficiency, system integration, and system reliability, making it difficult to meet the demands for high real-time performance and high precision image processing. Utility Model Content

[0004] The purpose of this invention is to provide a CameraLink camera data acquisition and image recognition system to solve the problems of insufficient throughput, low processing efficiency and low integration in the prior art.

[0005] To address the aforementioned technical problems, this utility model provides a CameraLink camera data acquisition and image recognition system based on a processor and an FPGA;

[0006] The FPGA provides two CameraLink camera data input interfaces, two image data output interfaces, and four isolated RS-422 interfaces. The processor provides two Gigabit Ethernet interfaces and one RS-232 interface.

[0007] The processor is a Hi3559ARFCV100 chip, which integrates dual-core A73 and dual-core A53. The big.LITTLE architecture and dual operating system achieve a balance between power consumption and startup time; the FPGA chip is a YX4F300T-900I.

[0008] The CameraLink camera data input interface is implemented based on the GM8284DD chip. The GM8284DD chip is a 28-bit serial-to-parallel converter receiver used in the receiving part of image transmission. It converts the serially input 4-channel LVDS data stream into 28-bit LVTTL / LVCMOS data. The input terminal of the GM8284DD chip is connected to the VPX connector, and the output terminal of the GM8284DD chip is connected to the FPGA.

[0009] The image data output interface is implemented based on the GM8285C chip, which is a 28-bit parallel-to-serial converter transmitter used in the transmission part of image transmission. It converts the parallel input 28-bit LVTTL / LVCMOS data into 4 pairs of LVDS differential signals and outputs 1 pair of LVDS differential clocks. The input terminal of the GM8285C chip is connected to the FPGA, and the output terminal of the GM8285C chip is connected to the VPX connector.

[0010] The RS-422 interface is implemented based on the ADM2682EBRIZ chip, which is a 5KV rms isolated RS-422 transceiver with an integrated isolated power supply.

[0011] The RS-232 interface is implemented based on a 3-5.5V powered, dual-channel, full-duplex RS-232 transceiver;

[0012] The Ethernet interface is based on a highly integrated Ethernet transceiver, XYI9322QNI, which conforms to the 1000Base-T, 100Base-TX, and 10Base-Te IEEE 802.3 standards. The XYI9322QNI Ethernet transceiver transmits data to the Hi3559ARFCV100 chip through the RGMII interface, and the MDI interface of the XYI9322QNI is connected to the VPX connector through a transformer.

[0013] In one embodiment, the CameraLink camera data input interface, the image data output interface, the RS-422 interface, and the Ethernet interface are all connected to the VPX connector and led out of the system through the VPX connector.

[0014] In one embodiment, there are two types of VPX connectors: the P0 connector is model VPX20-1111-0004, and the P1 to P3 connectors are model VPX20-1132-0001. The P0 connector is connected to the power supply voltage, the P1 connector is connected to the MDI interface of the XYI9322QNI through a transformer, the P2 connector is an RS-422 transceiver, and the P3 connector is connected to the output terminal of the GM8285C chip and the input terminal of the GM8284DD chip.

[0015] In one embodiment, the RS-232 interface is connected to a J30J connector and brought out outside the system via the J30J connector for programming firmware onto the Hi3559ARFCV100 chip.

[0016] In one embodiment, the Hi3559ARFCV100 chip includes two DDR4 memory controllers, each of which is connected to two DDR4 memory chips. Each DDR4 memory chip has a data width of 16 bits and a capacity of 4Gb. The Hi3559ARFCV100 chip has a total of four external DDR4 memory chips, achieving a memory capacity of 2GB.

[0017] The CameraLink camera data acquisition and image recognition system provided by this utility model has the following beneficial effects:

[0018] (1) High throughput: The FPGA enables efficient acquisition and preprocessing of CameraLink data, meeting the requirements of high bandwidth and low latency;

[0019] (2) High-performance processing: The Hisilicon 3559 processor has powerful computing capabilities and can efficiently run complex image recognition algorithms;

[0020] (3) High integration: The system integrates data acquisition, preprocessing, image recognition and communication functions, making it suitable for a variety of application scenarios;

[0021] (4) Strong real-time performance: Real-time acquisition and processing of image data is achieved through the collaborative work of FPGA and processor. Attached Figure Description

[0022] Figure 1 This is the overall design block diagram of this utility model;

[0023] Figure 2 This is a power supply design block diagram of this utility model;

[0024] Figure 3 This is a block diagram of the DDR4 design surrounding the Hisilicon 3559 processor;

[0025] Figure 4This is a block diagram of the CameraLink camera interface module design of this utility model;

[0026] Figure 5 This is a block diagram of the image data output design of this utility model;

[0027] Figure 6 This is a block diagram of the RS-422 interface design of this utility model;

[0028] Figure 7 This is a block diagram of the RS-232 interface design of this utility model;

[0029] Figure 8 This is a block diagram of the eMMC storage circuit design of this utility model;

[0030] Figure 9 This is a block diagram of the Ethernet interface design of this utility model;

[0031] Figure 10 This is a block diagram of the power-on and reset timing design of this utility model;

[0032] Figure 11 This is a block diagram of the data interaction design between the HiSilicon chip and the FPGA of this utility model;

[0033] Figure 12 This is a block diagram of a DDR3 design for an FPGA. Detailed Implementation

[0034] The CameraLink camera data acquisition and image recognition system proposed in this utility model will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of this utility model will become clearer from the following description. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of this utility model.

[0035] This utility model provides a CameraLink camera data acquisition and image recognition system, based on a Hisilicon 3559 processor and FPGA, and its overall design block diagram is as follows. Figure 1 As shown, the main chips included are: HiSilicon 3559 processor (Hi3559ARFCV100), FPGA (YX4F300T-900I), DDR4 memory chips, DDR3 memory chips, eMMC storage chips, GM8284DD chip, GM8285C chip, ADM2682EBRIZ chip, AiP3232 chip, Ethernet PHY chip, etc. External interfaces include 2 Ethernet interfaces, 2 camera data input interfaces, 2 image data output interfaces, 4 RS-422 interfaces, 1 RS-232 interface, 1 JTAG interface, and 1 power interface.

[0036] The camera data input interface is implemented based on the GM8284DD chip, a 28-bit serial-to-parallel converter receiver used in the receiving section of image transmission. It converts the serially input four-channel LVDS data stream into 28-bit LVTTL / LVCMOS data. The input of the GM8284DD chip is connected to a VPX connector, and its output is connected to the FPGA. The image data output interface is implemented based on the GM8285C chip, a 28-bit parallel-to-serial converter transmitter used in the transmitting section of image transmission. It converts the parallel input 28-bit LVTTL / LVCMOS data into four pairs of LVDS differential signals and outputs one pair of LVDS differential clocks. The input of the GM8285C chip is connected to the FPGA, and its output is connected to a VPX connector. The RS-422 interface is implemented based on the ADM2682EBRIZ chip, a 5KV... The RS-422 transceiver is RMS isolated and has an integrated isolated power supply. The RS-232 interface is implemented based on a 3-5.5V powered, dual-channel, full-duplex RS-232 transceiver. The Ethernet interface is implemented based on a highly integrated Ethernet transceiver, XYI9322QNI, which conforms to 1000Base-T, 100Base-TX, and 10Base-Te IEEE802.3 standards. The XYI9322QNI Ethernet transceiver transmits data to the Hi3559ARFCV100 chip through the RGMII interface. The MDI interface of the XYI9322QNI is connected to the VPX connector through a transformer.

[0037] There are two types of VPX connectors: the P0 connector is model VPX20-1111-0004, and the P1 to P3 connectors are model VPX20-1132-0001. The P0 connector is connected to the power supply voltage, the P1 connector is connected to the MDI interface of the XYI9322QNI through a transformer, the P2 connector is connected to the RS-422 transceiver, and the P3 connector is connected to the output terminal of the GM8285C chip and the input terminal of the GM8284DD chip.

[0038] The key circuit designs in this invention are as follows:

[0039] (1) Power supply design:

[0040] This system is powered by the backplane via the P0 connector, with a supply voltage of 12V. The voltages required for operation of each chip in the system are summarized as follows: FPGA_VDD_1V0, FPGA_VDD_1V8_AUX, FPGA_VDD_1V5, FPGA_VDD_2V5, FPGA_VDD_3V3, FPGA_VTT_0V75, MGTAVCC_1V0, MGTAVTT_1V2, CPU_DVDD_GPU, CPU_DVDD_CPU, CPU_DVDD, CPU_DVDD_MEDIA, CPU_VDD_1V8, CPU_VDD_3V3, CPU_VPP_2V5, CPU_VDDQ_1V2, VDD_1V8PMC, VDD_5V0. Each power supply is converted by a step-down DC-DC power chip. The voltage of the FPGA_VDD_1V0 power network is 1.0V, the voltage of the FPGA_VDD_1V8_AUX power network is 1.8V, the voltage of the FPGA_VDD_1V5 power network is 1.5V, the voltage of the FPGA_VDD_2V5 power network is 2.5V, the voltage of the FPGA_VDD_3V3 power network is 3.3V, the voltage of the FPGA_VTT_0V75 power network is 0.75V, the voltage of the MGTAVCC_1V0 power network is 1.0V, the voltage of the MGTAVTT_1V2 power network is 1.2V, and the voltage of the CPU_DVDD_GPU power network is... The voltages of the following power supply networks are as follows: CPU_DVDD_CPU power network: 0.8V~1.0V; CPU_DVDD power network: 0.8V~1.0V; CPU_DVDD_MEDIA power network: 0.8V~1.0V; CPU_VDD_1V8 power network: 1.8V; CPU_VDD_3V3 power network: 3.3V; CPU_VPP_2V5 power network: 2.5V; CPU_VDDQ_1V2 power network: 1.2V; VDD_1V8PMC power network: 1.8V; VDD_5V0 power network: 5.0V. The power supply chip models used in this invention are: BD4644SI, BD1205I, and JPL51200P. The power supply design block diagram is shown below. Figure 2 As shown.

[0041] 2) DDR4 design of the Hisilicon 3559 processor:

[0042] The Hisilicon 3559 processor includes two DDR4 memory controllers, each connected to two DDR4 memory chips. Each DDR4 memory chip has a 16-bit data bus width and a 4Gb capacity. The Hisilicon 3559 processor connects to a total of four DDR4 memory chips, achieving a 2GB memory capacity. The design architecture is as follows: Figure 3 As shown. The DDR4 chip model is SCB12Q4G160AF-07QI or LS6D4423-H. Other compatible packages and DDR4 chips with the same data width, capacity, and speed class can also be directly substituted.

[0043] (3) System camera data input design:

[0044] In this system, the FPGA receives image data from two cameras, parses and preprocesses the data, and then sends it to the HiSilicon 3559 processor. The HiSilicon 3559 processor then executes image recognition and tracking algorithms to perform real-time analysis of the image data. The system supports both visible light and infrared cameras. Visible light cameras form images by receiving and recording light in the visible light band, capturing the true colors and details of objects. Infrared cameras acquire images by detecting and converting the infrared radiation emitted by objects, and the images they provide are often characterized primarily by temperature distribution. The data outputs of both visible light and infrared cameras conform to the CameraLink protocol. The system's camera data input interface uses CameraLinkBase mode, with a maximum effective data bandwidth of 2.04Gbps. The core of the CameraLink protocol is the video data signal, which mainly consists of five pairs of differential signals: four pairs of video data signals and one pair of synchronization clock signals. The design block diagram of the camera data input interface is shown below. Figure 4 As shown, five pairs of differential signals are connected from the camera to the system. First, they are protected by TVS diodes to prevent electrostatic discharge and other interference signals from damaging the interface chips on the board. Then, the differential signals are filtered by a common-mode inductor to remove common-mode interference signals and improve signal quality. The five pairs of differential signals are then converted from serial to parallel using a GM8284DD chip, converting them into 28 bits of parallel data and one clock signal at a 1:7 ratio. Of the 28 bits of parallel data, 24 bits are image data, and the other 4 bits are video control signals, including frame synchronization, line synchronization, data valid, and idle / standby signals. Figure 4 The five pairs of differential signals are connected to the P3 connector and then to an external camera via the VPX backplane.

[0045] (5) System image data output design:

[0046] In the system, the Hisilicon 3559 processor sends the processed image data back to the FPGA via the LVDS interface. The FPGA then uses the GM8285C chip to convert the 28-bit parallel data into four pairs of LVDS differential data signals and one pair of LVDS differential clock signals. Figure 5 The five pairs of differential signals are connected to the P3 connector and communicate with other modules through the VPX backplane.

[0047] (6) RS-422 interface design of the system:

[0048] The system provides four RS-422 interfaces, such as Figure 6 As shown, the FPGA is connected to four external RS-422 transceivers, model ADM2682EBRIZ, which have 5kV isolation and integrated isolation power supplies. The RS-422 signals are connected to the P2 connector and communicate with other modules through the VPX backplane.

[0049] (7) RS-232 interface design of the system:

[0050] The system provides one RS-232 interface to the outside world, such as Figure 7 As shown, the HiSilicon 3559 processor includes multiple serial ports, of which only UART0 is used for firmware burning. The UART0 level of the HiSilicon 3559 processor is 1.8V. First, a level conversion chip, AiPTS0108TA20.TR, is needed to convert the 1.8V to 3.3V, and then connect it to the AiP3232SA16.TR chip. The AiP3232SA16.TR chip implements the RS-232 interface. The RS-232 signal is connected to the J30J-15ZKWP7-J connector. Firmware for the Hi3559ARFCV100 can be burned through this interface.

[0051] (8) System eMMC storage circuit design:

[0052] The bootloader, kernel, and file system of the Hisilicon 3559 processor in the system are stored in eMMC memory, such as... Figure 8 As shown, the eMMC memory contains 8-bit data signals, as well as DS, CLK, CMD, and Reset signals.

[0053] (9) Ethernet interface design of the system:

[0054] The HiSilicon 3559 processor includes two MAC controllers: MAC0 and MAC1. These two MAC controllers are connected to the Ethernet PHY chip XYI9322QNI via the RGMII interface, expanding to two Ethernet interfaces. The design architecture is as follows: Figure 9As shown, both Ethernet interfaces support speeds of 10 / 100 / 1000 Mbit / s. The two Ethernet ports are connected to the P1 connector and communicate with other circuit modules through the VPX backplane.

[0055] (10) System power-on and reset timing design:

[0056] The power-on and reset timing of the system is as follows: Figure 10 As shown, the FPGA chip and HiSilicon 3559 processor can only function properly if powered on in the correct sequence. According to... Figure 10 As shown in the sequence, the system's 12V main power supply is powered on first. Then, the power conversion chip on the board starts working, first generating the VDD_5V0 power supply. After the VDD_5V0 power supply is powered on, VDD_1V8PMC and FPGA_VDD_1V0 are powered on. After these two power supplies are powered on, CPU_DVDD_GPU, CPU_DVDD_CPU, CPU_DVDD, CPU_DVDD_MEDIA, and FPGA_VDD_1V8_AUX are powered on. After these five power supplies are powered on normally, FPGA_VDD_3V3, FPGA_VDD_1V5, FPGA_VDD_2V5, CPU_VDD_3V3, and CPU_VDD_1V8 are powered on. After these five power supplies are powered on normally, FPGA_VTT_0V75, MGTAVCC_1V0, MGTAVTT_1V2, CPU_VPP_2V5, and CPU_VDDQ_1V2 are powered on. At this point, all power supplies of the system are powered on. Figure 10 The document only lists the order in which the power supplies are powered on, without specifying the interval between power-on. In actual use, this interval can be controlled between 5 and 40 ms.

[0057] (11) Design of data communication between the HiSilicon 3559 processor and FPGA in the system:

[0058] The communication method between the HiSilicon 3559 processor and the FPGA in the system is as follows: Figure 11 As shown, the HiSilicon 3559 processor sends the processed image data back to the FPGA through the LVDS interface. The LVDS interface contains one clock signal and four data signals. The HiSilicon 3559 processor sends interrupt signals to the FPGA through the GPIO interface.

[0059] (12) Block diagram of DDR3 design for FPGA:

[0060] The DDR3 design architecture of the FPGA in the system is as follows: Figure 12 As shown, two 16-bit, 4Gb DDR3 chips are connected to BANK33 and BANK34 of the FPGA.

[0061] The above description is only a description of the preferred embodiment of the present utility model and is not intended to limit the scope of the present utility model in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A CameraLink camera data acquisition and image recognition system, characterized by, The processor and the FPGA are based on; The FPGA provides two-way CameraLink camera data input interface, two-way image data output interface, four-way isolated RS-422 interface, the processor provides two-way gigabit Ethernet interface, one-way RS-232 interface; The model of the processor is Hi3559ARFCV100 chip, which integrates dual-core A73 and dual-core A53, and the size core architecture and dual operating system make the power consumption and start time balanced; the model of the FPGA chip is YX4F300T-900I; The CameraLink camera data input interface is realized based on GM8284DD chip, which is a 28-bit serial-parallel conversion receiver, used for the receiving part in image transmission, converting four-way LVDS data stream into 28bits LVTTL / LVCMOS data, the input end of the GM8284DD chip is connected with the VPX connector, and the output end of the GM8284DD chip is connected with the FPGA; The image data output interface is realized based on GM8285C chip, which is a 28-bit parallel-serial conversion transmitter, used for the sending part in image transmission, converting 28bits LVTTL / LVCMOS data into four pairs of LVDS differential signals, and outputting one pair of LVDS differential clock, the input end of the GM8285C chip is connected with the FPGA, and the output end of the GM8285C chip is connected with the VPX connector; The RS-422 interface is realized based on ADM2682EBRIZ chip, which is a 5KV rms isolated RS-422 transceiver, integrated with an isolated power supply; The RS-232 interface is realized based on a 3-5.5V power supply, two-way, full-duplex RS-232 transceiver; The Ethernet interface is realized based on a highly integrated Ethernet transceiver XYI9322QNI, which meets the 1000Base-T, 100Base-TX and 10Base-Te IEEE 802.3 standards, and the Ethernet transceiver XYI9322QNI transmits data with the Hi3559ARFCV100 chip through the RGMII interface, and the MDI interface of the XYI9322QNI is connected to the VPX connector through a transformer.

2. The Camera Link camera data acquisition and image recognition system of claim 1, wherein, The CameraLink camera data input interface, the image data output interface, the RS-422 interface and the Ethernet interface are all connected to the VPX connector, and are led out of the system through the VPX connector.

3. The Camera Link camera data acquisition and image recognition system of claim 1, wherein, The VPX connector model has two types, the model of the P0 connector is VPX20-1111-0004, and the model of the P1-P3 connector is VPX20-1132-0001; wherein, the P0 connector is connected with a power voltage, the P1 connector is connected with an MDI interface of the XYI9322QNI through a transformer, the P2 connector is an RS-422 transceiver, and the P3 connector is connected with an output end of the GM8285C chip and an input end of the GM8284DD chip.

4. The Camera Link camera data acquisition and image recognition system of claim 1, wherein, The RS-232 interface is connected to the J30J connector, and is led out of the system through the J30J connector, and is used for burning firmware of the Hi3559ARFCV100 chip.

5. The Camera Link camera data acquisition and image recognition system of claim 1, wherein, The Hi3559ARFCV100 chip contains two DDR4 memory controllers, each of which is connected with two DDR4 memory particle chips, the data bit width of a single DDR4 memory particle is 16 bits, and the capacity is 4 Gb, and the Hi3559ARFCV100 chip is externally connected with four DDR4 memory particles, so that 2 GB memory capacity is realized.

Images