Image acquisition device based on USB4.0

Abstract

The invention discloses an image acquisition device based on USB4.0, which belongs to the technical field of image acquisition devices and comprises a DDR4 module, a POC circuit module, a current detection module of the POC circuit module, a deserialization circuit module, an ASM2464PPHY chip module and an FPGA processing chip. The FPGA processing chip comprises a PCIE GEN4x4 transceiver module, a mode selection module, an I2C / I3C / SPI module, a DPHY module, an SLVS-EC module, an image interpolation module, an arbitrary image cutting module, an effective frame processing module and a non-processing module; according to the image acquisition device, the latest transmission protocol that the acquisition device is connected with the camera is used, and a plurality of transmission protocols compatible with USB4, Thunderbolt 3 (TBT3), Thunderbolt 2, USB3.2 and USB3.1 are adopted, so that the image data can be uploaded to a PC (Personal Computer) under the actual effective transmission bandwidth that the highest USB4 or Thunderbolt 3 exceeds 20G, and the effective bandwidth of USB3.0 is only 2.8 G, and therefore, the image transmission efficiency is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image acquisition devices, and in particular relates to an image acquisition device based on USB4.0. Background Art

[0002] With the booming development of the imaging camera industry, it involves three major camera industry fields: mobile phones, car cameras, and industrial cameras. Mobile phone cameras already have 400 million pixel CMOS chips, and require a test board for multi-channel simultaneous detection. Domestic new-capacity car cameras are also gradually moving towards 17M pixels. In the field of industrial cameras, there is also a growing demand for area array cameras with hundreds of millions of pixels and high frame rate area array cameras with hundreds of frames or even hundreds of frames. High-speed image transmission is a must, and USB3.0 can no longer meet its transmission needs. 10 Gigabit is also difficult and inconvenient in the R&D stage, and needs to be equipped with a desktop or PCIE adapter box. Summary of the invention

[0003] The present invention overcomes the deficiencies of the prior art and provides an image acquisition device based on USB 4.0 to solve the problems existing in the prior art.

[0004] To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: an image acquisition device based on USB4.0, including a USB4.0 chip and an FPGA processing chip, the FPGA processing chip including a PCIE GEN4x4 transceiver module, a mode selection module, an I2C / I3C / SPI module, a DPHY module, an SLVS-EC module, an image interpolation module, an arbitrary image cropping module, a valid frame processing module, and a non-processing module, the image interpolation module and the arbitrary image cropping module are respectively connected to a module selection module and to a storage device, the storage device is used to store the acquired image information and calculation results, and the mode selection module is used to send the processed image information or results to the PC through the USB4 PHY chip.

[0005] In a preferred embodiment of the present invention, the image acquisition device supports USB4.0 protocol and high-speed transmission with a PC with a USB4.0 interface, with a transmission rate of up to 40Gbps, and the image acquisition device is also compatible with Thunderbolt 3 protocol, and the image acquisition device is also backward compatible with Thunderbolt 2, USB3.2, and USB3.1 protocols, and the protocol configuration is completed by the PHY chip configuration module.

[0006] In a preferred embodiment of the present invention, the image acquisition device uses AU15P xili nx, and the FPGA is connected to the serial deserialization chip through M IPI to simultaneously acquire images from four vehicle-mounted cameras and realize real-time transmission with a computer and video recording requirements. The FPGA supports COMS sensors such as M IPI and supports SLVS-EC interface CMOS sensors.

[0007] In a preferred embodiment of the present invention, the image interpolation module comprises:

[0008] A valid RAW data sending unit, used for identifying valid frames of image information in the storage, and sending the valid frames of image information to the PC memory through a sending module;

[0009] The interpolation processing unit is used to perform interpolation processing on the valid frames of the image information. According to the host computer instruction, the FPGA converts the color space of its data and stores it in DDR4, and then copies the converted image data to the corresponding memory address of the computer according to the host computer instruction.

[0010] In a preferred embodiment of the present invention, the image interpolation module further includes an unprocessed image processing unit and an image arbitrary sampling unit, and the unprocessed image processing unit and the image arbitrary sampling unit are electrically connected between the interpolation processing unit and the storage, respectively.

[0011] In a preferred embodiment of the present invention, the FPGA is connected to the chip ASM2464P via PCI E4.0 X4 to achieve USB4.0 image transmission, and the image acquisition device supports vehicle-mounted cameras with multiple deserializer chips.

[0012] In a preferred embodiment of the present invention, the FPGA uses three DDR4 memories for high-speed caching of the vehicle-mounted camera image data, and the mode selection module selects one of the image interpolation module and the arbitrary image cropping module to cache the image information processing.

[0013] The present invention solves the defects existing in the background technology and has the following beneficial effects:

[0014] (1) The image acquisition device of the present invention uses the latest transmission protocol for connecting the acquisition device to the camera, and is compatible with multiple transmission protocols such as USB4, Thunderbolt 3 (TBT3), Thunderbolt 2, USB3.2, and USB3.1, achieving an actual effective transmission bandwidth of more than 20G of USB4 or Thunderbolt 3 to upload image data to the PC. However, the effective bandwidth of USB3.0 is only 2.8G, thereby greatly improving the image transmission efficiency;

[0015] (2) The image acquisition device of the present invention is realized by developing a soft core to acquire the MIPIDPHY RX signal of the vehicle camera module after serial deserialization. The small board of the deserialization chip is independently designed and can be applicable to a variety of chips. The deserialization board provides a wide range power supply POC design of 4.5V-18V, which can take into account the testing of most vehicle camera modules on the market and MIPIDPHY mobile phone camera modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0017] Figure 1 It is a schematic diagram of modules of a preferred embodiment of the present invention;

[0018] Figure 2 A connection circuit of a PHY chip according to a preferred embodiment of the present invention;

[0019] Figure 3 A connection circuit between an FPGA and a PHY chip in a preferred embodiment of the present invention;

[0020] Figure 4 A connection circuit between FPGA and DDR4 in a preferred embodiment of the present invention;

[0021] Figure 5 A connection circuit between a PHY chip and a type-c interface in a preferred embodiment of the present invention;

[0022] Figure 6 A connection circuit for POC power supply and power current detection in a preferred embodiment of the present invention;

[0023] Figure 7 This is a connection circuit of the FPGA and the M IPI connection with the deserialization board and the SLVS-EC camera connector that is compatible with the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0024] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0025] Combination Figures 1 to 7 As shown, this embodiment provides an image acquisition device based on USB4.0, including a DDR4 module, a POC circuit module and its current detection module, a deserialization circuit module, an ASM2464P PHY chip module and an FPGA processing chip.

[0026] The FPGA processing chip includes PCI E GEN4 x4 transceiver module, mode selection module, I2C / I 3C / SPI module, DPHY module, SLVS-EC module, image interpolation module, arbitrary image cropping module, effective frame processing module, and no processing module;

[0027] Combination Figure 3 and Figure 4 As shown, the PCI E GEN4 x4 transceiver module is the connection circuit between the FPGA and the PHY chip, and the ASM2464P PHY chip module is the connection circuit between the PHY chip. Figure 3 The FLASH chip GD25Q80 is used to configure the ASM2464P PHY chip. The FLASH chip GD25Q80 is connected to the ASM2464P PHY chip through the SPI interface. The ASM2464P can download the configuration information of the host computer to the SPI FLASH chip.

[0028] This embodiment of Figure 3 and Figure 4 High-speed transmission is achieved through the PCI E GEN4 x4 protocol, which enables high-speed transmission of data between FPGA and PHY chip, and its speed exceeds that of USB4.0.

[0029] like Figure 5 As shown in the figure, the DDR4 module is a connection diagram between FPGA and DDR4*3. DDR4*4 is used to store the collected image information. Four DDR4 chips are used to connect to the FPGA.

[0030] The sending module is used to send the image processed by the image interpolation module or other processing modules to the host computer;

[0031] The mode selection module is used to obtain instructions from the host computer and control the image interpolation module and other processing modules to process the image in the memory DDR4*3 according to the instructions;

[0032] The image interpolation module is used to interpolate the image in the memory DDR4*3;

[0033] In this embodiment, the memory DDR4 is used to store the collected image information; the FPGA processing chip adopts the conventional FPGA XCAU15P-2FFVB676I, and is developed using the FPGA program, which is also applicable to other FPGA models, but is not limited to its specific chip model, package, and brand; according to different test requirements of the host computer, the mode selection module performs different mode tests, and the image interpolation module interpolates the image in the memory DDR4*3 according to the host computer instruction. The image interpolation module can be used to process a certain component of the image information. Compared with the traditional host computer that requires a complete frame of the image to perform interpolation calculation processing, the image interpolation processing is performed in the FPGA processing chip, which has higher processing efficiency and also reduces the workload of the PC end of the host computer; other processing modules perform processing other than interpolation processing on the image in the memory DDR4*3 according to the host computer instruction, such as cropping, ISP, feature extraction, etc.; when the image interpolation module selected by the mode selection module processes the image information, the sending module sends the processed image information to the PC through the PCI EGEN4 x4 interface via the ASM2464PPHY chip.

[0034] Furthermore, the image interpolation module includes a valid frame processing unit and an interpolation processing unit; the valid frame processing unit is connected to the memory DDR4*3, and the interpolation processing unit is connected to the valid frame processing unit and the mode selection module respectively;

[0035] The valid frame processing unit is used to identify valid frames of image information, including the detection of the PASSLOCK signal of the deserialization chip, the monitoring of the time between frames, the number of data in the frame data itself and the monitoring of the MIPI signal verification, etc.

[0036] The interpolation processing unit is used to perform interpolation processing on the valid frames of the image information.

[0037] In this embodiment, the valid frame processing unit is used to identify valid frames of image information, and send the valid frames of image information to the PC memory through the PCI E GEN4 x4 module via the USB4.0 PHY chip module, and only send the valid frames of the image to the PC end of the host computer, without sending invalid frames, thereby avoiding the influence of invalid frames on the host computer; the interpolation processing unit is used to perform interpolation processing on the valid frames of image information, convert the color space of the data according to the host computer instruction and store it in the memory DDR4*3, and then send the converted image data to the host computer through the PCI E GEN4 x4 module via the USB4.0 PHY chip module according to the host computer instruction.

[0038] Furthermore, the image interpolation module further includes a non-processing unit; the non-processing unit is respectively connected to the effective frame processing unit and the mode selection module;

[0039] The non-processing unit is used for not processing the valid frames of the image information.

[0040] In this embodiment, according to the host computer instruction, the non-processing unit does not perform any processing on the valid frame of the image information, and directly sends it to the host computer through the sending module.

[0041] Furthermore, the image interpolation module further includes an arbitrary image cropping unit; the arbitrary image cropping unit is respectively connected to the effective frame processing unit and the mode selection module;

[0042] In this embodiment, an image acquisition device based on USB4.0 performs different mode tests by a mode selection module according to different test requirements of the host computer, and moves the processing work originally to be completed on the PC side of the host computer to the FPGA processing chip, thereby reducing the processing workload of the PC side of the host computer and improving the operating efficiency of the PC side of the host computer.

[0043] Furthermore, the FPGA processing chip also includes an SLVS-EC module and a DPHY module; the SLVS-EC camera and the chip or camera data of the M IPI interface can be connected and data collected;

[0044] The PCI E GEN4 x4 module uses high-speed LVDS transmission through the GTX port in the FPGA. The image transmission strictly follows the PCI E GEN4 protocol. The DDR4 read and write module in the FPGA reads, writes and sends the DDR4 data at high speed.

[0045] The I2C / I3C / SPI module is used in the FPGA to configure the voltage value of the vehicle-mounted POC power supply (POC circuit module), configure the register of the serializer (deserialization circuit module), and read the current value of the current detection module corresponding to the POC power supply and calculate its power supply power, as well as configure the camera with SLVS-EC interface and MIPI interface and the camera chip transmitted through the deserialization circuit module.

[0046] like Figure 7 FIG. 1 is a connection diagram between FPGA and external SLVS and M IPI interface serializer or sensor.

[0047] In this embodiment, if Figure 5The connection circuit between the PHY chip and the type-c interface in the USB4.0 image acquisition device shown in the figure, the device is connected to the TYPE-C interface of the PC end that supports the USB4 protocol through the TYPE-C interface to achieve high-speed image transmission. At the same time, the hardware of the image acquisition device is compatible with Thunderbolt3 (TBT3), Thunderbolt 2, USB3.2, and USB3.1 transmission protocols, which improves the diversity of high-speed image transmission protocols between the PC and the image acquisition device and the compatibility of different computers.

[0048] like Figure 6 As shown, the POC power module of this embodiment provides a dual-channel 4.5V--18V power supply for the vehicle-mounted camera, and the POC power ripple is less than 10mv. Therefore, we need to add a feedback circuit to the input end of the camera to adjust the voltage of the input end more finely. The POC power supply is first stepped down by the 24V power supply through the DC / DC circuit, and then the stepped-down voltage is connected to the digital potentiometer through the LDO to obtain the required voltage. In this design, we use the MCP4728 DAC as a digital potentiometer, and the size of its voltage is connected to the FPGA by I2C to obtain the desired voltage value. The MCP4728 can control four digital voltages. In this circuit, we use PAC1954 to monitor the voltage and current of the output voltage, and the voltage and current values ​​read are also connected to the FPGA through I2C. The voltage drop at the two ends of the high-precision sampling resistor is obtained by PAC1954, so as to calculate its current value, and the voltage value of its output end can also be obtained.

[0049] In summary, a USB4.0 image acquisition device of this embodiment can support USB4, Thunderbolt3 (TBT3), Thunderbolt 2, USB3.2, USB3.1 and multiple transmission protocols to transfer image data to PC, and is suitable for high-pixel mobile phones, vehicle-mounted cameras, and circuits are also suitable for industrial cameras with large target surfaces, etc. It supports 40G rate to upload single-channel or multi-channel image data collected in the device to PC at high speed and conveniently, and provides 4-channel POC power supply. The device can greatly reduce the use of computer CPU without the participation of computer CPU by interpolating the data or original data or processing results, and greatly improve the speed of interpolation operation and acquisition of valid data, and realize the discrimination of valid frame data by FPGA, avoiding invalid data occupying bandwidth and consuming the CPU of the host computer.

[0050] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims

1. An image acquisition device based on USB4.0, characterized in that: It includes a USB4.0 chip and an FPGA processing chip. The FPGA processing chip includes a PCIE GEN4 x4 transceiver module, a mode selection module, an I2C / I3C / SPI module, a DPHY module, an SLVS-EC module, an image interpolation module, an arbitrary image cropping module, a valid frame processing module, and a non-processing module. The image interpolation module and the arbitrary image cropping module are respectively connected to the module selection module and connected to the storage. The storage is used to store the collected image information and calculation results. The mode selection module is used to send the processed image information or results to the PC through the USB4 PHY chip.

2. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The image acquisition device supports USB4.0 protocol and high-speed transmission with a PC with a USB4.0 interface, with a transmission rate of up to 40Gbps. The image acquisition device is also compatible with Thunderbolt 3 protocol, and is also backward compatible with Thunderbolt 2, USB3.2, and USB3.1 protocols. The protocol configuration is completed by the PHY chip configuration module.

3. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The image acquisition device uses AU15P xilinx, and the FPGA is connected to the serial deserialization chip through MIPI to simultaneously acquire images from four vehicle-mounted cameras and realize real-time transmission and video recording requirements with the computer. The FPGA supports COMS sensors such as MIPI and supports SLVS-EC interface CMOS sensors.

4. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The image interpolation module comprises: A valid RAW data sending unit, used for identifying valid frames of image information in the storage, and sending the valid frames of image information to the PC memory through a sending module; The interpolation processing unit is used to perform interpolation processing on the valid frames of the image information. According to the host computer instruction, the FPGA converts the color space of its data and stores it in DDR4, and then copies the converted image data to the corresponding memory address of the computer according to the host computer instruction.

5. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The image interpolation module further includes an unprocessed image processing unit and an image arbitrary image taking unit, and the unprocessed image processing unit and the image arbitrary image taking unit are electrically connected between the interpolation processing unit and the storage, respectively.

6. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The FPGA is connected to the chip ASM2464P via PCIE4.0 X4 to realize USB4.0 image transmission. The image acquisition device supports vehicle-mounted cameras with multiple deserializer chips.

7. The image acquisition device based on USB4.0 according to claim 1, characterized in that: The FPGA uses three DDR4 memories for high-speed caching of the vehicle camera image data, and the mode selection module selects one of the image interpolation module and the arbitrary image cropping module to cache the image information processing.

Images