High-speed area-array camera system based on 40G network transmission and image acquisition and transmission method

Abstract

The invention discloses a high-speed area-array camera system based on 40G network transmission and an image acquisition and transmission method, and belongs to the technical field of high-speed image acquisition and transmission. The technical problems that a traditional high-speed area-array camera is insufficient in bandwidth, high in system delay and low in multi-camera synchronization precision in a high-resolution and high-frame-rate scene are solved, and the technical scheme is characterized in that the high-speed area-array camera comprises a sensor array, a front-end FPGA module, a rear-end FPGA module and a software deployment end; the sensor array comprises a plurality of groups of image sensors used for collecting original image data and outputting differential data signals, the front-end FPGA module receives the differential data signals, performs image preprocessing and then outputs the signals through a high-speed serial interface, and the rear-end FPGA module performs data analysis and image recombination and then outputs the signals through a QSFP + interface in an optical fiber mode. And the software deployment end receives the image data in parallel through the multi-path optical fiber interface and performs synchronous fusion processing.

Description

Technical Field

[0001] This invention relates to the field of high-speed image acquisition and transmission technology, specifically to a high-speed area array camera system and image acquisition and transmission method based on 40G network transmission. Background Technology

[0002] In fields such as industrial machine vision, autonomous driving, and scientific imaging, high-speed area scan cameras need to simultaneously meet stringent requirements such as high resolution, high frame rate, low latency, and multi-device synchronization. As precision manufacturing continues to demand higher accuracy and speed in inspection, traditional camera systems are showing significant technical bottlenecks when dealing with high-performance scenarios such as 16K×12K resolution and 100fps frame rate.

[0003] Existing technical solutions mainly include two architectures. The first is an architecture using 10Gigabit Ethernet with a dedicated ISP, transmitting image data via 10Gbps Ethernet and relying on a host computer FPGA or software to perform image preprocessing, including noise reduction, HDR, distortion correction, etc. This solution can only support uncompressed transmission at 8K resolution and below 60fps. As the resolution increases further, the amount of raw data increases dramatically. Calculated at 16K×12K pixels, 12-bit color depth, and 100fps, the raw data volume reaches as high as 7.68Gbps, approaching the 10Gbps bandwidth limit. To achieve transmission, image compression is necessary, resulting in a loss rate of over 15% for critical details such as edges and textures, severely impacting subsequent detection accuracy. The second is the Camera Link architecture, using dedicated differential cables for transmission, which has significantly limited bandwidth, with the Base type only reaching 2.04Gbps and the Full type only 6.8Gbps. Multi-camera synchronization requires additional trigger boxes, resulting in complex wiring and poor system scalability.

[0004] The above-mentioned solutions suffer from four prominent problems in practical applications. First, bandwidth bottlenecks limit resolution improvement. 10GIGE and lower technologies cannot achieve uncompressed transmission in high-resolution, high-frame-rate scenarios, and image quality deteriorates significantly after compression. Second, system latency is too high. Traditional solutions typically have a total latency exceeding 5ms from image acquisition to data reception, failing to meet the stringent requirements of sub-millisecond response times in scenarios such as high-speed sorting and laser welding. Third, multi-camera synchronization accuracy is insufficient. Solutions relying on software triggering or low-precision hardware triggering suffer from synchronization errors greater than 1ms between multiple cameras, leading to cumulative deviations in applications requiring multi-view data fusion, such as 3D reconstruction and motion trajectory analysis. Fourth, system reliability and scalability are limited. Dedicated cables such as Camera Link are complex to install and difficult to expand, while the 10GIGE solution experiences packet loss rates exceeding 1% in industrial electromagnetic interference environments, causing image distortion and data loss.

[0005] Chinese patent document CN106412474A discloses a high-speed, lossless, ultra-high-definition industrial visual inspection method and system. It discloses a visual inspection system composed of an intelligent camera unit, an ultra-high-speed communication conversion unit, a computer display configuration unit, and a high-speed recording and broadcasting unit. The system uses an ISP processing FPGA module and a visual inspection FPGA module to preprocess and perform inspection operations on images. It employs a multi-channel optical fiber transmission technology for RAW data and RGB image data, achieving the technical effects of avoiding the loss of image format standardization information and overcoming transmission channel bandwidth limitations. However, it still suffers from problems such as transmission bandwidth not reaching the 40Gbps level, unclear end-to-end delay and multi-camera synchronization accuracy indicators, and a lack of software-based image synchronization and fusion mechanisms.

[0006] Chinese patent document CN111404652A discloses a multi-channel data acquisition and real-time mixed transmission platform based on FPGA. It discloses a data acquisition and transmission platform composed of an FPGA data processor, an external image acquisition circuit, an optical module, a lower-level processor, and a host computer module. This platform achieves multi-channel mixed transmission through an oversampling module, a multi-channel data mixing and verification module, and a data transmission switching state machine. It employs a SerDes protocol interface and a TLK1501 fiber optic transmission module for high-speed communication, achieving the technical effect of multi-channel high- and low-speed data acquisition and real-time mixed transmission with a transmission bandwidth of 6.25Gbps. However, it still suffers from problems such as the transmission bandwidth being only 6.25Gbps, which is insufficient to support 4K-level high-resolution, high-frame-rate transmission; only supporting two low-resolution detectors; and lacking a nanosecond-level multi-camera synchronization accuracy control mechanism. Summary of the Invention

[0007] The purpose of this invention is to provide: A high-speed area array camera system and image acquisition and transmission method based on 40G network transmission, and related technologies, are proposed to solve the technical problems of insufficient bandwidth, high system latency, and low multi-camera synchronization accuracy of traditional high-speed area array cameras in high-resolution and high-frame-rate scenarios, or a combination thereof.

[0008] Terminology Explanation: Unless otherwise defined, all technical terms in this document have the same meanings as commonly understood by one of ordinary skill in the art to which the subject matter of the claims pertains. Unless otherwise stated, all patents, patent inventions, and publications cited in this document are incorporated herein by reference in their entirety. If multiple definitions exist for terms in this document, the definitions in this chapter shall prevail.

[0009] It should be understood that the above brief description and the following detailed description are exemplary and for illustrative purposes only, and do not limit the subject matter of the invention in any way. In this invention, the singular is used in conjunction with the plural unless otherwise specifically stated. It should also be noted that, unless otherwise stated, the use of “or” or “or” means “and / or”. Furthermore, the use of the term “comprising” and other forms such as “including,” “containing,” and “contains” are not limiting.

[0010] Unless specifically defined herein, the use of all commercially available products herein employs standard techniques. For example, it may be carried out using the manufacturer's instructions for use with the kit, or in accordance with methods known in the art or the description of this invention. The techniques and methods described herein can generally be implemented according to conventional methods well known in the art, based on the descriptions in the various summary and more specific documents cited and discussed in this specification.

[0011] In a first aspect, the present invention provides: A high-speed area scan camera system based on 40G network transmission includes: The sensor array includes multiple sets of image sensors for acquiring raw image data of the target scene and outputting differential data signals; The front-end FPGA module is connected to the sensor array and is used to receive the differential data signal and perform image preprocessing, and output the preprocessed image data through a high-speed serial interface; The back-end FPGA module is connected to the front-end FPGA module and is used to receive preprocessed image data, perform data parsing and image reconstruction, and output the processed image data via a QSFP+ interface in a fiber optic manner. On the software deployment side, it is connected to the back-end FPGA module through multiple fiber optic interfaces to receive multiple image data in parallel and perform synchronous fusion processing.

[0012] Furthermore, the sensor array includes multiple sets of image sensors, each set of image sensors outputting differential data signals, synchronization signals, pixel clock signals and data enable signals through an LVDS interface.

[0013] Furthermore: the front-end FPGA module includes: The first clock subsystem is used to generate the Aurora interface clock, DDR cache clock, and system clock; An image preprocessing unit, connected to the sensor array, is used to receive multi-channel differential data signals and perform image stitching, format conversion, noise suppression, and timing alignment operations. The first storage subsystem includes DDR3 memory and Flash memory. The DDR3 memory is connected to the image preprocessing unit and is used to cache image data during the preprocessing process. The Flash memory is used to store FPGA configuration programs. The SERDES interface is connected to the first storage subsystem and is used to read cached image data and forward it to the back-end FPGA module. The first clock subsystem provides clock signals to the image preprocessing unit, the first storage subsystem, and the SERDES interface, respectively.

[0014] Furthermore: the back-end FPGA module includes: The second clock subsystem is used to generate the system clock, Aurora interface clock, DDR cache clock, and 10 Gigabit Ethernet clock. The data processing unit is connected to the front-end FPGA module via a GTX high-speed serial transceiver and is used to receive preprocessed image data and perform data parsing and image reconstruction. The second storage subsystem includes multiple sets of DDR4 memory and Flash memory. The DDR4 memory is connected to the data processing unit and is used to cache the reconstructed image data. The Flash memory is used to store backend programs and configuration information. A trigger interface, connected to the data processing unit, includes a trigger input interface and a trigger output interface, used to receive external trigger signals and output synchronous trigger signals; A QSFP+ interface is connected to the second storage subsystem for reading cached image data and outputting it through an MPO / MTP-12 fiber optic cable. The second clock subsystem provides clock signals to the data processing unit, the second storage subsystem, and the QSFP+ interface, respectively.

[0015] Furthermore, the software deployment terminal includes an image acquisition card and a memory management module. The image acquisition card is equipped with multiple SFP+ interfaces, each with a transmission rate of 10Gbps. These multiple SFP+ interfaces are connected to the QSFP+ interface of the back-end FPGA module via LC-Duplex optical fiber. The memory management module allocates an independent memory buffer area for each SFP+ interface based on the real-time data reception volume of each SFP+ interface. The software deployment terminal reads image data from each memory buffer area in network port order and stitches and merges them sequentially into a complete image.

[0016] Furthermore, the software deployment end also includes an interface and power management board, which is used to provide multiple power supplies for the sensor array, the front-end FPGA module and the back-end FPGA module, and reserves system configuration ports, log export interfaces and remote debugging interfaces.

[0017] Secondly, the present invention provides: An image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission includes the following steps: S1: Acquire raw image data of the target scene through a sensor array and transmit the raw image data in parallel to the front-end FPGA module in the form of differential signals; S2: The front-end FPGA module performs image preprocessing on the received multi-channel differential data signals. The image preprocessing includes image stitching, format conversion, noise suppression, and timing alignment. The preprocessed image data is then transmitted to the back-end FPGA module via a high-speed serial interface. S3: The back-end FPGA module performs data parsing and image reconstruction on the received image data, and outputs the processed image data in parallel via fiber optics through the QSFP+ interface. S4: The software deployment end receives image data in parallel through multiple fiber optic interfaces, dynamically allocates memory for the image acquisition card, and synchronously merges the image data from each channel into a complete image according to the network port order.

[0018] Further: In S1, the sensor array includes multiple sets of image sensors, and the differential data signal output by each set of image sensors includes LVDS differential data signal, synchronization signal, pixel clock signal and data enable signal.

[0019] Further: In S2, the format conversion converts the RAW raw data into MONO8 format data; the timing alignment includes synchronizing the synchronization signal, pixel clock signal, and data enable signal.

[0020] Furthermore, in S3, the back-end FPGA module receives an external trigger signal and controls the timing of data acquisition and data transmission according to the external trigger signal to achieve synchronous trigger control with external devices.

[0021] Furthermore: In S4, the software deployment terminal receives image data in parallel through multiple SFP+ interfaces, and the dynamic allocation includes allocating a corresponding memory area for each network port based on the data peak value of each network port.

[0022] Further: In S2, the front-end FPGA module caches the preprocessed image data in the DDR high-speed cache and transmits it to the back-end FPGA module via the Aurora protocol and the GTX high-speed serial transceiver at a transmission rate of 40Gbps.

[0023] The present invention has at least the following beneficial effects: 1. This invention breaks through the transmission bandwidth bottleneck. It achieves high-speed transmission of 40Gbps through a QSFP+ interface combined with MPO / MTP-12 optical fiber. The software deployment uses multiple SFP+ interfaces for parallel reception, supporting uncompressed transmission of MONO8 format images at 4K×2K resolution and 320fps frame rate. Actual bandwidth usage can reach 26Gbps, completely solving the problem of image detail loss caused by the need for compression transmission in traditional 10GIGE solutions under high-resolution, high-frame-rate scenarios.

[0024] 2. This invention significantly reduces system latency. It employs a dual-FPGA hierarchical pipeline architecture, with the front-end FPGA module handling image preprocessing and the back-end FPGA module handling data parsing and image reconstruction. Combined with DDR high-speed caching and Aurora protocol transmission, a complete data pipeline is formed from sensor acquisition to software fusion, compressing end-to-end latency to less than 1ms, a 400% improvement over the traditional 5ms latency, meeting the sub-millisecond real-time response requirements of scenarios such as high-speed sorting and laser welding.

[0025] 3. This invention achieves high-precision multi-camera synchronization. The back-end FPGA module is configured with trigger input and trigger output interfaces, and achieves synchronous triggering with external devices through a hardware-level trigger control mechanism. The multi-camera synchronization accuracy can reach within 50ns, which is more than 20 times higher than the traditional software triggering scheme, effectively supporting multi-view data fusion applications such as 3D reconstruction and motion trajectory analysis. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of a high-speed area array camera system based on 40G network transmission, provided by the present invention.

[0027] Figure 2 The flowchart illustrates a high-speed area array camera image acquisition and transmission method based on 40G network transmission, as provided by this invention.

[0028] Figure 3 This is a system architecture diagram of a 40GIGE high-speed area array camera in an embodiment of the present invention.

[0029] Figure 4 This is a diagram of the internal architecture of FPGA1 in an embodiment of the present invention. Detailed Implementation

[0030] The following non-limiting embodiments are intended to enable those skilled in the art to gain a more comprehensive understanding of the present invention, but do not limit the invention in any way. The following content is merely an exemplary description of the scope of protection claimed by the present invention, and those skilled in the art can make various changes and modifications to the present invention based on the disclosed content, and such changes should also fall within the scope of protection claimed by the present invention.

[0031] The present invention will be further described below by way of specific embodiments. Unless otherwise specified, all instruments, devices, equipment, reagents, products, etc., used in the embodiments of the present invention are obtained through conventional commercial means.

[0032] Example 1 like Figure 1 As shown, this invention provides a high-speed area array camera system based on 40G network transmission, including a sensor array, a front-end FPGA module, a back-end FPGA module, and a software deployment terminal. The sensor array includes multiple image sensors for acquiring raw image data of the target scene and outputting differential data signals. The front-end FPGA module is connected to the sensor array, receiving the differential data signals and performing image preprocessing, outputting the preprocessed image data through a high-speed serial interface. The back-end FPGA module is connected to the front-end FPGA module, receiving the preprocessed image data and performing data parsing and image reconstruction, outputting the processed image data via a QSFP+ interface in fiber optic mode. The software deployment terminal is connected to the back-end FPGA module through multiple fiber optic interfaces, receiving multiple image data streams in parallel and performing synchronous fusion processing. This system architecture realizes a complete data link from image acquisition, preprocessing, advanced processing to software fusion, supporting 40Gbps high-speed transmission and meeting the real-time requirements of high-resolution, high-frame-rate scenarios.

[0033] In one specific embodiment of this example, the sensor array includes multiple sets of image sensors. Each set of image sensors outputs differential data signals, synchronization signals, pixel clock signals, and data enable signals through an LVDS interface. Multiple sets of sensors acquire image data in parallel and transmit it to the front-end FPGA module via LVDS low-voltage differential signals, achieving synchronous acquisition of multi-view images.

[0034] In one specific embodiment of this example, the front-end FPGA module includes a first clock subsystem, an image preprocessing unit, a first storage subsystem, and a SERDES interface. The first clock subsystem generates the Aurora interface clock, the DDR cache clock, and the system clock. The image preprocessing unit is connected to the sensor array and receives multiple differential data signals, performing image stitching, format conversion, noise suppression, and timing alignment operations. The first storage subsystem includes DDR3 memory and Flash memory. The DDR3 memory is connected to the image preprocessing unit to cache image data during preprocessing, and the Flash memory stores the FPGA configuration program. The SERDES interface is connected to the first storage subsystem and reads the cached image data, forwarding it to the back-end FPGA module. The first clock subsystem provides clock signals to the image preprocessing unit, the first storage subsystem, and the SERDES interface. This structure achieves lightweight preprocessing and high-speed forwarding of image data.

[0035] In one specific embodiment of this example, the back-end FPGA module includes a second clock subsystem, a data processing unit, a second storage subsystem, a trigger interface, and a QSFP+ interface. The second clock subsystem generates the system clock, Aurora interface clock, DDR cache clock, and 10 Gigabit Ethernet clock. The data processing unit is connected to the front-end FPGA module via a GTX high-speed serial transceiver and receives preprocessed image data for data parsing and image reconstruction. The second storage subsystem includes multiple sets of DDR4 memory and Flash memory. The DDR4 memory is connected to the data processing unit to cache the reconstructed image data, and the Flash memory stores the back-end program and configuration information. The trigger interface is connected to the data processing unit and includes a trigger input interface and a trigger output interface for receiving external trigger signals and outputting synchronous trigger signals. The QSFP+ interface is connected to the second storage subsystem and reads the cached image data, outputting it via an MPO / MTP-12 fiber optic cable. The second clock subsystem provides clock signals to the data processing unit, the second storage subsystem, and the QSFP+ interface. This structure enables complex image processing and 40Gbps-level high-speed transmission.

[0036] In one specific embodiment of this example, the software deployment end includes an image acquisition card and a memory management module. The image acquisition card is equipped with multiple SFP+ interfaces, each with a transmission rate of 10Gbps. These multiple SFP+ interfaces are connected to the QSFP+ interfaces of the back-end FPGA module via LC-Duplex optical fibers. The memory management module allocates an independent memory buffer area for each SFP+ interface based on the real-time data reception volume of each interface. The software deployment end reads image data from each memory buffer area in network port order and sequentially stitches and fuses them into a complete image. This method achieves synchronous reception and lossless fusion of multiple data streams.

[0037] In one specific embodiment of this example, the software deployment terminal further includes an interface and a power management board, which provides multiple power supplies for the sensor array, the front-end FPGA module and the back-end FPGA module, and reserves system configuration ports, log export interfaces and remote debugging interfaces to ensure system power supply reliability and remote operation and maintenance capabilities.

[0038] Example 2 like Figure 2 As shown, this invention also provides an image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission, comprising the following steps: S1, acquiring raw image data of the target scene through a sensor array, and transmitting the raw image data in parallel to the front-end FPGA module in the form of differential signals; S2, the front-end FPGA module performs image preprocessing on the received multi-channel differential data signals, including image stitching, format conversion, noise suppression, and timing alignment, and transmits the preprocessed image data to the back-end FPGA module through a high-speed serial interface; S3, the back-end FPGA module performs data parsing and image reconstruction on the received image data, and outputs the processed image data in parallel via a QSFP+ interface using optical fiber; S4, the software deployment end receives image data in parallel through multiple optical fiber interfaces, dynamically allocates memory for the image acquisition card, and synchronously fuses the image data from each channel into a complete image according to the network port order. The above method forms a complete pipeline processing from acquisition to fusion, and the end-to-end latency can be compressed to less than 1ms.

[0039] In one specific embodiment of this example, the sensor array in S1 includes multiple sets of image sensors. The differential data signal output by each set of image sensors includes LVDS differential data signal, synchronization signal, pixel clock signal and data enable signal, so as to realize the synchronous acquisition and initial transmission of multiple images.

[0040] In one specific implementation of this embodiment, the format conversion in S2 converts the raw RAW data into MONO8 format data, and the timing alignment includes synchronizing and calibrating the synchronization signal, pixel clock signal and data enable signal to ensure the timing consistency of multiple data streams.

[0041] In one specific implementation of this embodiment, the back-end FPGA module in S3 receives an external trigger signal and controls the timing of data acquisition and data transmission according to the external trigger signal, thereby achieving synchronous trigger control with external devices and supporting multi-camera synchronization accuracy to reach the nanosecond level.

[0042] In one specific implementation of this embodiment, the software deployment terminal in S4 receives image data in parallel through multiple SFP+ interfaces, and dynamically allocates memory regions for each network port based on the data peak value of each network port, thereby achieving synchronous reception of multi-port data.

[0043] In one specific implementation of this embodiment, the front-end FPGA module in S2 caches the preprocessed image data into a DDR high-speed cache and transmits it to the back-end FPGA module via the Aurora protocol and a GTX high-speed serial transceiver at a transmission rate of 40Gbps, which meets the bandwidth requirements of high-resolution and high-frame-rate scenarios.

[0044] Example 3 The present invention will be further described below with reference to a practical embodiment.

[0045] like Figures 3-4 As shown, the implementation scheme of this invention is based on a multi-sensor collaborative acquisition and multi-FPGA hierarchical processing architecture to construct a high-speed image acquisition and intelligent processing system. The specific technical route and module collaboration logic are as follows: 1. Multi-sensor array: Parallel image acquisition.

[0046] The system is configured with up to four image sensors (Sensor1~Sensor4, taking the GMAX2509 model as an example) to capture raw images of the target scene. Each sensor outputs LVDS differential data signals, a synchronization signal (SYNC), a pixel clock (PCLK), and a data enable (DE). The raw image data is transmitted to the front-end FPGA (FPGA1) through multiple parallel links to achieve synchronous acquisition and initial transmission of multi-view images.

[0047] 2. Front-end FPGA (FPGA1): Preprocessing and data forwarding.

[0048] FPGA1 uses a high-performance FPGA chip (XCK7160T-2FFG676I) to perform core tasks such as image preprocessing and data transfer. Its hardware resource configuration and workflow are as follows: Storage and Clock Subsystem: Onboard 200MHz AuroraDDR high-speed cache for temporary storage of massive amounts of instantaneous image data; A 100MHz system clock (SYS) provides basic timing drive; an external 2GB DDR3 memory (MT41J256M16HA-125) expands the runtime data cache space; 256Mb Flash (IS25LP256D) stores the FPGA configuration program and basic parameters, ensuring rapid initialization upon power-up.

[0049] Data processing logic: After receiving multi-channel LVDS data from the sensor array, relying on the FPGA parallel computing architecture, it completes preprocessing operations such as image stitching (multi-sensor view fusion), format conversion (such as RAW raw data to MONO8 universal format), noise suppression (real-time noise reduction based on algorithms), and timing alignment (SYNC / PCLK / DE signal synchronization calibration); the processed standardized image data is forwarded to the back-end FPGA (FPGA2) through the SERDES high-speed serial interface.

[0050] 3. Back-end FPGA (FPGA2): Advanced processing and system interaction.

[0051] FPGA2 uses a higher-performance FPGA chip (XCKU040-2FFVA1156I), focusing on complex image processing and system-level control / communication. The hardware design and functional implementation are as follows: Storage and Clock Subsystem: Onboard 256Mb Flash (IS25LP256D) stores backend programs and configuration information; 100MHz system clock (SYS) ensures core timing; multiple external 1GB DDR4 memory modules (MT40A512M16LY-083) meet the needs of large data volume operations; 200MHz DDR high-speed cache and 312.5MHz 10 Gigabit Ethernet module (Gige) support high-speed data throughput and network-level transmission.

[0052] Data processing and interaction logic: Receives pre-processed image data from FPGA1, performs data parsing, image reconstruction, and other operations; achieves high-speed external communication through the QSFP interface (supports direct fiber optic connection to meet long-distance, high-bandwidth transmission requirements); uses trigger input (TRIGIN×2) and trigger output (TRIGOUT×2) interfaces to achieve synchronous triggering and coordinated control with external devices (such as motion control platforms and alarm systems); LED modules intuitively display the system's operating status (such as readiness / fault indications for each stage of acquisition, processing, and transmission).

[0053] 4. Interface and Power Management Board: Power supply and expansion.

[0054] The independent "interface and power board" performs two key functions: 1. Power Distribution: Provides multiple stable power supplies for the Sensor array, FPGA1, FPGA2 and various storage modules, covering different voltage domains such as FPGA core voltage, IO voltage and sensor-specific voltage, to ensure the reliability of system power supply.

[0055] 2. Interface expansion: Reserve external communication and management interfaces (such as system configuration port, log export interface, remote debugging interface) to support remote operation and maintenance, dynamic parameter configuration and operation data traceability of the system.

[0056] 5. The software deployment end synchronously receives and integrates data.

[0057] Leveraging the "4-in-1" characteristic of the fiber optic cable hardware, the software deployment module (SDK) also performs the corresponding parallel image reception and fusion operations. It's important to note that this image fusion involves stitching the images sequentially according to the network port order, eliminating the issue of missing key pixels and achieving synchronous data reception across all four ports with millisecond-level precision.

[0058] Verification of technical effectiveness and / or analysis of technical problem solving 1. Breakthrough in performance metrics.

[0059] Bandwidth and resolution: 4K×2K@320fps MONO8 uncompressed transmission bandwidth 26Gbps; latency: end-to-end latency ≤1ms (traditional 10GIGE solution 5ms→1ms, improvement of 400%). Synchronization accuracy: Multi-camera synchronization error ≤50ns (traditional software triggering error >1ms, a 20-fold improvement); Reliability: Packet loss rate <0.01% in industrial environment (near the inverter) (1% of traditional solution → 0.01%, a 100-fold improvement).

[0060] 2. Application value and prospects.

[0061] Industrial machine vision: supports scenarios such as "high-speed sorting (such as 3C electronic defect detection), laser welding (dynamic monitoring of the molten pool), AOI (automatic optical inspection)", improving the yield rate by more than 5%; 3. The synergistic value of key technologies.

[0062] Each module operates on a "data flow-driven + control flow-coordinated" logic, forming an end-to-end pipelined processing flow from "sensor input → trigger synchronization → clock alignment → DDR cache → GTX high-speed output," thus overcoming the technical bottlenecks of traditional solutions in terms of "high bandwidth, high synchronization, and high flexibility." This architecture is particularly suitable for fields such as machine vision (high-speed industrial cameras), high-speed communication, and precision measurement, achieving a performance breakthrough of "40Gbps-level data throughput + microsecond-level trigger response + millisecond-level system reconfiguration."

[0063] In short, the core of the invention is to build an integrated "acquisition-processing-transmission" solution for ultra-high-speed data scenarios through modular hardware design, high-speed protocol adaptation, and coordinated control of trigger, clock, and cache.

[0064] 4. Technological innovation and value.

[0065] This invention utilizes a hierarchical architecture of "multi-sensor parallel acquisition → front-end FPGA lightweight preprocessing → back-end FPGA advanced processing → high-speed external transmission / control," combining the parallel computing advantages of FPGAs with various types of high-speed storage / interface resources to achieve high-resolution, high-frame-rate real-time image acquisition and intelligent analysis. The modular design ensures the system's scalability (such as increasing the number of sensors, iterating FPGA models, and providing rich external interfaces), making it widely applicable to scenarios with high requirements for image real-time performance and intelligence, such as industrial inspection, intelligent security, and autonomous driving.

[0066] This invention fully covers Figure 3 The hardware hierarchy of “Sensor array → FPGA1 → FPGA2 → interface power board” clearly explains the signal flow, resource allocation and processing logic, highlighting the technical features of “hierarchical processing, high-speed interconnection and modular expansion”.

[0067] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present invention, and is not intended to limit the scope of protection of the present invention. Simple modifications or equivalent substitutions made by those skilled in the art to the technical solution of the present invention do not depart from the essence and scope of the technical solution of the present invention.

Claims

1. A high-speed area array camera system based on 40G network transmission, characterized in that, include: The sensor array includes multiple sets of image sensors for acquiring raw image data of the target scene and outputting differential data signals; The front-end FPGA module is connected to the sensor array and is used to receive the differential data signal and perform image preprocessing, and output the preprocessed image data through a high-speed serial interface; The back-end FPGA module is connected to the front-end FPGA module and is used to receive preprocessed image data, perform data parsing and image reconstruction, and output the processed image data via a QSFP+ interface in a fiber optic manner. On the software deployment side, it is connected to the back-end FPGA module through multiple fiber optic interfaces to receive multiple image data in parallel and perform synchronous fusion processing.

2. The high-speed area array camera system based on 40G network transmission according to claim 1, characterized in that: The sensor array includes multiple sets of image sensors, each set of image sensors outputting differential data signals, synchronization signals, pixel clock signals and data enable signals through an LVDS interface.

3. The high-speed area array camera system based on 40G network transmission according to claim 1, characterized in that: The front-end FPGA module includes: The first clock subsystem is used to generate the Aurora interface clock, DDR cache clock, and system clock. An image preprocessing unit, connected to the sensor array, is used to receive multi-channel differential data signals and perform image stitching, format conversion, noise suppression, and timing alignment operations. The first storage subsystem includes DDR3 memory and Flash memory. The DDR3 memory is connected to the image preprocessing unit and is used to cache image data during the preprocessing process. The Flash memory is used to store FPGA configuration programs. The SERDES interface is connected to the first storage subsystem and is used to read cached image data and forward it to the back-end FPGA module. The first clock subsystem provides clock signals to the image preprocessing unit, the first storage subsystem, and the SERDES interface, respectively.

4. The high-speed area array camera system based on 40G network transmission according to claim 1, characterized in that: The back-end FPGA module includes: The second clock subsystem is used to generate the system clock, Aurora interface clock, DDR cache clock, and 10 Gigabit Ethernet clock. The data processing unit is connected to the front-end FPGA module via a GTX high-speed serial transceiver and is used to receive preprocessed image data and perform data parsing and image reconstruction. The second storage subsystem includes multiple sets of DDR4 memory and Flash memory. The DDR4 memory is connected to the data processing unit and is used to cache the reconstructed image data. The Flash memory is used to store backend programs and configuration information. A trigger interface, connected to the data processing unit, includes a trigger input interface and a trigger output interface, used to receive external trigger signals and output synchronous trigger signals; A QSFP+ interface is connected to the second storage subsystem for reading cached image data and outputting it through an MPO / MTP-12 fiber optic cable. The second clock subsystem provides clock signals to the data processing unit, the second storage subsystem, and the QSFP+ interface, respectively.

5. A high-speed area array camera system based on 40G network transmission according to claim 1, characterized in that: The software deployment terminal includes an image acquisition card and a memory management module. The image acquisition card is equipped with multiple SFP+ interfaces, each with a transmission rate of 10Gbps. These multiple SFP+ interfaces are connected to the QSFP+ interfaces of the back-end FPGA module via LC-Duplex optical fibers. The memory management module allocates an independent memory buffer area for each SFP+ interface based on the real-time data reception volume of each SFP+ interface. The software deployment terminal reads image data from each memory buffer area in network port order and stitches and merges them into a complete image.

6. The high-speed area array camera system based on 40G network transmission according to claim 1, characterized in that: The software deployment end also includes an interface and power management board, which is used to provide multiple power supplies for the sensor array, the front-end FPGA module and the back-end FPGA module, and reserves system configuration ports, log export interfaces and remote debugging interfaces.

7. An image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission, applied to the system described in any one of claims 1-6, characterized in that, Includes the following steps: S1: Acquire raw image data of the target scene through a sensor array and transmit the raw image data in parallel to the front-end FPGA module in the form of differential signals; S2: The front-end FPGA module performs image preprocessing on the received multi-channel differential data signals. The image preprocessing includes image stitching, format conversion, noise suppression, and timing alignment. The preprocessed image data is then transmitted to the back-end FPGA module via a high-speed serial interface. S3: The back-end FPGA module performs data parsing and image reconstruction on the received image data, and outputs the processed image data in parallel via fiber optics through the QSFP+ interface. S4: The software deployment end receives image data in parallel through multiple fiber optic interfaces, dynamically allocates memory for the image acquisition card, and synchronously merges the image data from each channel into a complete image according to the network port order.

8. The image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission according to claim 7, characterized in that: In S1, the sensor array includes multiple sets of image sensors, and the differential data signal output by each set of image sensors includes LVDS differential data signal, synchronization signal, pixel clock signal and data enable signal.

9. The image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission according to claim 7, characterized in that: In S2, the format conversion converts the RAW raw data into MONO8 format data; the timing alignment includes synchronizing the synchronization signal, pixel clock signal and data enable signal.

10. The image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission according to claim 7, characterized in that: In S3, the back-end FPGA module receives an external trigger signal and controls the timing of data acquisition and data transmission according to the external trigger signal to achieve synchronous trigger control with external devices.

11. The image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission according to claim 7, characterized in that: In S4, the software deployment terminal receives image data in parallel through multiple SFP+ interfaces, and the dynamic allocation includes allocating a corresponding memory area for each network port based on the data peak value of each network port.

12. The image acquisition and transmission method for a high-speed area array camera system based on 40G network transmission according to claim 7, characterized in that: In S2, the front-end FPGA module caches the preprocessed image data in the DDR high-speed cache and transmits it to the back-end FPGA module via the Aurora protocol and the GTX high-speed serial transceiver at a transmission rate of 40Gbps.

Images