FPGA circuit for image transmission and industrial camera
By designing the collaborative operation of modules such as the Ethernet controller and image processing module in the FPGA circuit, the problem of increased GVSP data throughput in industrial cameras under high frame rate and high resolution scenarios was solved, achieving low latency and high throughput image data transmission, while balancing control flexibility and transmission efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- SHENZHEN SHUMA ELECTRONICS TECH
- Filing Date
- 2025-05-08
- Publication Date
- 2026-04-28
AI Technical Summary
In high frame rate and high resolution scenarios, the data throughput of existing industrial cameras increases exponentially with GVSP, leading to system real-time performance and resource contention issues. Traditional solutions cannot balance control flexibility and transmission efficiency.
Design an FPGA circuit that includes an Ethernet controller, an image processing module, a storage control module, an image transmission module, and a processing module. The interconnect module enables efficient connection and collaboration between the modules, reducing the copying of image data in the processing module of the FPGA circuit. The processing module is used for parameter setting and management, ensuring fast response of GVCP commands and stable transmission of GVSP data streams.
It achieves low-latency, high-throughput transmission of image data, balancing control flexibility and transmission efficiency, meeting the requirements of GVCP multi-instruction dynamic interaction, and ensuring the system's real-time performance and bandwidth utilization.
Smart Images
Figure CN224178213U_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of industrial camera data transmission, and in particular to an FPGA circuit for image transmission and an industrial camera. Background Technology
[0002] As the core equipment of machine vision systems, the performance of the communication protocol of industrial cameras directly affects image acquisition efficiency and system real-time performance. Currently, GigE Vision is a widely adopted standard protocol in the field of industrial cameras. Its protocol stack consists of multiple layers of network protocols: the physical layer and data link layer are based on Gigabit Ethernet (GIGE) to achieve high-speed transmission; the network layer and transport layer follow the IP protocol and UDP protocol respectively to adapt to general network architectures; and the application layer is divided into GVCP (GigE Vision Control Protocol) and GVSP (GigE Vision Streaming Protocol), which are used for device control and image data transmission respectively.
[0003] The GVCP protocol enables function settings through control command interaction between the host computer and the camera, such as reading the camera's XML description file, setting exposure time, and triggering mode. These commands are characterized by diverse parameter types and complex interaction logic, but the data volume per communication is relatively small. In contrast, the GVSP protocol is specifically designed for transmitting image data from the camera to the host computer. Its data packet format is fixed, but its bandwidth is extremely high, especially in high-speed, high-resolution scenarios, where a single camera's transmission bandwidth can reach hundreds of Mbps. With the increasing demands for frame rate and resolution in industrial inspection, GVSP data throughput is growing exponentially, posing a severe challenge to system real-time performance.
[0004] Traditional solutions typically employ two approaches. One involves using all hardware except the CPU to handle GVCP instruction processing and GVSP data transmission. However, GVCP instruction processing consumes significant logic resources, which is unsustainable when FPGA logic resources are limited. The other approach relies on the CPU for both GVCP control flow and GVSP data flow protocol parsing and data transmission scheduling. However, the CPU must simultaneously handle dynamically changing control instructions and massive image data transfers, making it highly susceptible to performance bottlenecks due to resource contention. Specifically, during frequent GVCP configurations, the CPU may struggle to guarantee stable GVSP data transmission due to interrupt handling delays. Conversely, the high bandwidth requirements of GVSP consume substantial CPU resources, leading to sluggish control instruction responses. Furthermore, image data requires multiple copies through CPU memory (e.g., sensor→CPU→DDR→network card), further exacerbating transmission latency and bandwidth waste.
[0005] To address the aforementioned contradictions, there is an urgent need for an architecture design that balances control flexibility and transmission efficiency, ensuring low latency and high throughput transmission capabilities of GVSP data streams while satisfying the dynamic interaction of multiple GVCP commands. Utility Model Content
[0006] This application provides an FPGA circuit and an industrial camera for image transmission, which balances control flexibility and transmission efficiency.
[0007] An FPGA circuit for image transmission includes:
[0008] Ethernet controller;
[0009] An image processing module is used to receive image data from an image acquisition device and transmit the image data;
[0010] The storage control module is connected to the image processing module and the memory respectively, and is used to transfer the image data output by the image processing module to the memory for storage, and output the stored image data.
[0011] An image transmission module is connected to the storage control module and the Ethernet controller, respectively, and is used to forward the image data to the Ethernet controller so that it can be sent to the Ethernet physical layer through the Ethernet controller;
[0012] The processing module is connected to the image processing module, the storage control module, and the image transmission module, respectively, and is used to configure the transmission of the image data in the image processing module, the storage control module, the image transmission module, and the Ethernet controller according to the setting parameters.
[0013] In one embodiment, the FPGA circuit for image transmission further includes:
[0014] An interconnect module is connected to the processing module, the image processing module, the storage control module, and the image transmission module, respectively, and is used to establish connections between the processing module and the image processing module, between the processing module and the image transmission module, between the processing module and the storage control module, between the storage control module and the image transmission module, and between the image processing module and the storage control module.
[0015] In one embodiment, the Ethernet controller is further configured to receive the setting parameters from the Ethernet physical layer; the FPGA circuit for image transmission further includes:
[0016] A parameter transmission module is connected to both the interconnect module and the Ethernet controller, and is used to transmit the setting parameters received by the Ethernet controller to the processing module through the interconnect module.
[0017] In one embodiment, the processing module is further configured to transmit and set the setting parameters to the parameter transmission module.
[0018] In one embodiment, the FPGA circuit for image transmission further includes:
[0019] The communication module is connected to both the image acquisition device and the image processing module, and is used to receive image data transmitted by the image acquisition device and transmit it to the image processing module.
[0020] In one embodiment, the FPGA circuit for image transmission further includes:
[0021] A buffer module is connected to both the communication module and the image processing module, and is used to buffer the image data before outputting it to the image processing module.
[0022] In one embodiment, the FPGA circuit for image transmission further includes:
[0023] The data packaging module is connected to both the communication module and the buffer module, and is used to package the image data and output it to the buffer module.
[0024] In one embodiment, the processing module is also connected to the communication module and the data packaging module respectively, for setting the transmission of the image data to the communication module and the data packaging module respectively.
[0025] In one embodiment, the FPGA circuit for image transmission further includes:
[0026] The IIC / IO interface circuit is connected to the processing module and the image acquisition device. The processing module is also used to set parameters of the image acquisition device through the IIC / IO interface circuit according to the setting parameters.
[0027] An industrial camera includes an image acquisition device, a memory, and the aforementioned FPGA circuitry for image transmission.
[0028] The aforementioned FPGA circuit and industrial camera used for image transmission can achieve high-speed transmission of image data through an image processing module, a storage control module, an image transmission module, and an Ethernet controller. The image data does not need to be copied through the processing module and then transmitted to the Ethernet physical layer, which reduces the burden on the processing module in the FPGA circuit. Furthermore, the processing module configures and manages other modules according to the setting parameters. The setting parameter communication data volume is small and the transmission speed is fast, which takes into account both control flexibility and transmission efficiency. While meeting the dynamic interaction of multiple commands for setting parameters, it ensures low latency and high throughput transmission capability of image data. Attached Figure Description
[0029] Figure 1 This is a structural block diagram of an FPGA circuit for image transmission according to an embodiment of this application;
[0030] Figure 2 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application;
[0031] Figure 3 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application;
[0032] Figure 4 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application;
[0033] Figure 5 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application;
[0034] Figure 6 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application;
[0035] Figure 7 This is a structural block diagram of an FPGA circuit for image transmission according to another embodiment of this application. Detailed Implementation
[0036] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.
[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0038] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly. The connection can be a direct connection or an indirect connection.
[0039] Furthermore, the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. Additionally, the technical solutions of the various embodiments can be combined with each other, but only on the basis of being achievable by those skilled in the art. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0040] This utility model embodiment provides an FPGA circuit 100 for image transmission, such as... Figure 1 As shown, the circuit includes: an Ethernet controller 110, an image processing module 120, a storage control module 130, an image transmission module 140, and a processing module 150. The image processing module 120 receives image data from the image acquisition device 103 and transmits the image data. The storage control module 130 is connected to the image processing module 120 and the memory 101, and transmits the image data output by the image processing module 120 to the memory 101 for storage, and outputs the stored image data. The image transmission module 140 is connected to the storage control module 130 and the Ethernet controller 110, and forwards the image data to the Ethernet controller 110 for transmission to the Ethernet physical layer 102 via the Ethernet controller 110. The processing module 150 is connected to the image processing module 120, the storage control module 130, the image transmission module 140, and the Ethernet controller 110 according to the set parameters.
[0041] It is understood that the image processing module 120 adopts the advanced AXI bus standard and is responsible for processing and DMA-transferring the image data obtained from the image acquisition device 103, thereby transmitting it at high speed to the storage control module 130. The storage control module 130 can be used to efficiently process the storage and retrieval of image data. It can first forward the data to the memory 101 for temporary storage, and then read it from the memory 101 and send it to the image transmission module 140. The memory 101 can be an off-chip DDR3 memory 101. The image transmission module 140 can transmit image data according to the communication protocol of the Ethernet controller 110. The standard protocol for industrial cameras is usually GIGE Vision, and the image data transmitted to the host computer is in GVSP format. Therefore, the image data read by the storage control module 130 can be transmitted to the Ethernet controller 110 through the image transmission module 140. The image transmission module 140 can transmit image data in the form of GVSP packets. The Ethernet controller 110 can be the Ethernet MAC in the FPGA circuit 100, which supports giant frame data transmission to ensure high-efficiency and stable network communication.
[0042] The processing module 150 can be the CPU in the FPGA circuit 100. As a central processing unit, it connects to the aforementioned modules, configures and manages each module via the AXI bus, and coordinates and controls the overall data flow. Specifically, the processing module 150 can receive setting parameters, which include relevant parameters for image data transmission from each module. The processing module 150 can then configure the image data transmission settings for each module based on these setting parameters. These setting parameters can be GVCP packets, transmitted from the host computer to the processing module 150. This method features small data volume, fast transmission speed, and low load on the processing module 150.
[0043] Thus, high-speed transmission of image data can be achieved sequentially through image processing module 120, storage control module 130, image transmission module 140, and Ethernet controller 110. Image data does not need to be copied through processing module 150 and then transmitted to Ethernet physical layer 102, reducing the burden on processing module 150 in FPGA circuit 100. Furthermore, processing module 150 configures and manages other modules according to setting parameters. The communication data volume of setting parameters is small and the transmission speed is fast, taking into account both control flexibility and transmission efficiency. While meeting the dynamic interaction of multiple instructions for setting parameters, it ensures low latency and high throughput transmission capability of image data.
[0044] In one embodiment, such as Figure 2As shown, the FPGA circuit 100 for image transmission also includes an interconnect module 160, which can be an interconnect unit in the FPGA circuit 100. The interconnect module 160 is connected to the processing module 150, the image processing module 120, the storage control module 130, and the image transmission module 140, respectively, to establish connections between the processing module 150 and the image processing module 120, between the processing module 150 and the image transmission module 140, between the processing module 150 and the storage control module 130, between the storage control module 130 and the image transmission module 140, and between the image processing module 120 and the storage control module 130.
[0045] Specifically, the connection between processing module 150 and image processing module 120 can be used to implement transmission settings from processing module 150 to image processing module 120, including the starting address and length of data transmission. Subsequently, image processing module 120 executes the data transmission task according to the configuration parameters. The connection between processing module 150 and image transmission module 140 can be used to implement transmission settings from processing module 150 to image transmission module 140; the connection between processing module 150 and storage control module 130 can be used to implement transmission settings from processing module 150 to storage control module 130; the connection between storage control module 130 and image transmission module 140 can be used to implement image data transmission from storage control module 130 to image transmission module 140; and the connection between image processing module 120 and storage control module 130 can be used to implement image data transmission from image processing module 120 to storage control module 130.
[0046] The interconnect module 160 is a bus switching device based on the AXI protocol, which can be used as the axi_interconnect in the FPGA circuit 100, supporting connections between multiple master controllers and multiple slave devices. Through the interconnect module 160, the processing module 150 can access multiple slave devices (image processing module 120, storage control module 130, image transmission module 140, etc.) for data transmission, configuration, or command sending, realizing parallel processing.
[0047] In this way, the interconnect module 160 can establish connections between different functional modules, enabling efficient and reliable data transmission and collaboration between modules, and providing efficient data exchange services for the entire system. Furthermore, according to changes in system design requirements, the configuration of the interconnect module 160 can be adjusted to easily add or remove functional modules, thereby allowing for timely adjustments to the system architecture and flexible system expansion.
[0048] In one embodiment, such as Figure 3As shown, the Ethernet controller 110 is also used to receive setting parameters from the Ethernet physical layer 102; the FPGA circuit 100 for image transmission also includes a parameter transmission module 170, which is connected to the interconnect module 160 and the Ethernet controller 110 respectively, and is used to transmit the setting parameters received by the Ethernet controller 110 to the processing module 150 through the interconnect module 160.
[0049] It is understood that the parameter transmission module 170 connects the Ethernet controller 110 and the interconnection module 160. After the Ethernet controller 110 receives the setting parameter packet (which may be a GVCP packet, etc.) from the Ethernet physical layer 102, it is then processed by the parameter transmission module 170 according to the protocol specification and sent to the processing module 150 through the interconnection module 160.
[0050] In one embodiment, the processing module 150 is also used to transmit and set the parameters of the parameter transmission module 170.
[0051] It is understood that the processing module 150 can establish a connection with the parameter transmission module 170 through the interconnection module 160. After receiving the setting parameters, the processing module 150 can set the transmission rate, bandwidth, flow control information, etc. of the parameter transmission module 170 during the setting parameter transmission process, so as to reliably transmit the setting parameter data to the processing module 150 and meet the transmission requirements in specific scenarios.
[0052] In one embodiment, such as Figure 4 As shown, the FPGA circuit 100 for image transmission also includes a communication module 180, which is connected to the image acquisition device 103 and the image processing module 120 respectively, and is used to receive image data transmitted by the image acquisition device 103 and transmit it to the image processing module 120.
[0053] The communication module 180 can be a MIPI RX interface module, used to directly receive image data streams of the MIPI transmission standard from the image acquisition device 103, convert them into AXI stream data streams, and then transmit them through the image processing module 120. The communication module 180 acts as a bridge, realizing the conversion between the digital video interface protocol and the FPGA internal data transmission protocol. The image data reception and conversion of the communication module 180 can be configured by the processing module 150.
[0054] In one embodiment, the FPGA circuit 100 for image transmission further includes a buffer module 190, such as... Figure 5 As shown, the buffer module 190 is connected to the communication module 180 and the image processing module 120 respectively, and is used to buffer the image data and output it to the image processing module 120.
[0055] It is understandable that by setting up the buffer module 190, the integrity of image data can be guaranteed without loss due to potential speed differences between real-time acquisition and processing. The buffer module 190 can be a FIFO buffer module in the FPGA circuit 100, which supports first-in-first-out data storage. When the data read and write speeds are mismatched, the FIFO acts as a data buffer, working in conjunction with a flow control mechanism to ensure the orderliness and efficiency of data inflow / outflow.
[0056] In one embodiment, the FPGA circuit 100 for image transmission further includes a data packetization module 210, such as... Figure 6 As shown, the data packing module 210 is connected to both the communication module 180 and the buffer module 190, and is used to pack image data and output it to the buffer module 190. The data packing module 210 can be the image_pack module in the FPGA circuit 100.
[0057] Before the communication module 180 transmits the image data to the buffer module 190, the image data can be processed by the data packaging module 210. For example, the Axi stream image data stream can be added with GVSP format, including the guide frame, data frame header, and tail frame. This is done by the data packaging module 210 instead of the processing module 150, which avoids affecting the speed of the processing module 150 and causing a sharp drop in the frame rate.
[0058] In one embodiment, such as Figure 6 As shown, the processing module 150 is also connected to the communication module 180 and the data packaging module 210 respectively, and is used to set up the transmission of image data for the communication module 180 and the data packaging module 210 respectively.
[0059] It is understood that the processing module 150 configures the image data transmission settings for the communication module 180, which may include, for example, the data stream conversion rate and transmission rate; the processing module 150 also configures the image data transmission settings for the data packaging module 210, which may include GVSP format settings, including parameters such as the guide frame, data frame header, and tail frame, thus allowing the image data transmission process to be adjusted as needed. The processing module 150 can set and update parameters through the AXI interface on the data packaging module 210.
[0060] In one embodiment, such as Figure 7 As shown, the FPGA circuit 100 for image transmission also includes an IIC / IO interface circuit 220. The IIC / IO interface circuit 220 is connected to the processing module 150 and the image acquisition device 103. The processing module 150 is also used to set parameters or read the status of the image acquisition device 103 through the IIC / IO interface circuit 220 according to the set parameters.
[0061] It is understood that the IIC / IO interface circuit 220 may include an IIC interface circuit and an IO interface circuit. The IIC interface circuit is used to process the serial data communication between the processing module 150 and the image acquisition device 103, and to configure various parameters of the sensor and the data obtained by related modules after processing. The IO interface circuit is used to process the switching signals of the processing module 150 directly controlling the image acquisition device 103, such as reset and emergency stop; in addition, the IIC / IO interface circuit can also read the status of the image acquisition device 103. By introducing the IIC / IO interface circuit 220 into the system, the processing module 150 can flexibly set various parameters of the image acquisition device 103 and send control signals, ensuring high-quality acquisition and real-time performance of image data, and realizing the efficient application of the FPGA circuit 100 for image transmission in different scenarios.
[0062] In one embodiment, the FPGA circuit 100 for image transmission further includes a ROM storage module 230 for storing camera feature description files; and a register 240 connected to both the ROM storage module 230 and the processing module 150. The register 240 can be used to temporarily store the camera feature description files and provide them to the processing module 150. The processing module 150 can transfer the camera feature description files to the DDR3 memory 101 during the initialization phase. In some embodiments, the FPGA circuit 100 for image transmission may also include a state management module 250 for managing the timing and behavior of the register 240. This state management module 250 may be a State module / Ctrl module in the FPGA circuit 100. In some embodiments, the camera feature description files may also be stored in external flash memory, and the register 240 can be used to provide the camera feature description files to the processing module 150.
[0063] In one embodiment, the FPGA circuit 100 for image transmission further includes a QSPI control module 260, which is connected to the processing module 150 and can control QSPI peripherals, such as external flash, based on the AXI bus protocol to embed programs.
[0064] In one embodiment, the FPGA circuit 100 for image transmission also includes a universal asynchronous transceiver 270 connected to the processing module 150. This transceiver may be a UART in the FPGA circuit 100, used to realize asynchronous serial communication between the FPGA and an external serial device through an AXI interface, so as to output debugging information of the FPGA.
[0065] In one embodiment, the FPGA circuit 100 for image transmission also includes a timer 280 connected to the processing module 150 for generating a timeout interrupt via the AXI interface to prevent data transmission and reception from stalling.
[0066] This utility model embodiment also provides an industrial camera, including an image acquisition device 103, a memory 101, and an FPGA circuit 100 for image transmission in any of the above embodiments.
[0067] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. An FPGA circuit for image transmission, characterized in that, include: Ethernet controller; An image processing module is used to receive image data from an image acquisition device and transmit the image data; The storage control module is connected to the image processing module and the memory respectively, and is used to transfer the image data output by the image processing module to the memory for storage, and output the stored image data. An image transmission module is connected to the storage control module and the Ethernet controller, respectively, and is used to forward the image data to the Ethernet controller so that it can be sent to the Ethernet physical layer through the Ethernet controller; The processing module is connected to the image processing module, the storage control module, and the image transmission module, respectively, and is used to configure the transmission of the image data in the image processing module, the storage control module, the image transmission module, and the Ethernet controller according to the setting parameters.
2. The FPGA circuit for image transmission according to claim 1, characterized in that, The FPGA circuit for image transmission also includes: An interconnect module is connected to the processing module, the image processing module, the storage control module, and the image transmission module, respectively, and is used to establish connections between the processing module and the image processing module, between the processing module and the image transmission module, between the processing module and the storage control module, between the storage control module and the image transmission module, and between the image processing module and the storage control module.
3. The FPGA circuit for image transmission according to claim 2, characterized in that, The Ethernet controller is also configured to receive the setting parameters from the Ethernet physical layer; the FPGA circuit for image transmission further includes: The parameter transmission module is connected to both the interconnection module and the Ethernet controller, and is used to transmit the setting parameters received by the Ethernet controller to the processing module through the interconnection module.
4. The FPGA circuit for image transmission according to claim 3, characterized in that, The processing module is also used to transmit and set the setting parameters to the parameter transmission module.
5. The FPGA circuit for image transmission according to claim 4, characterized in that, The FPGA circuit for image transmission also includes: The communication module is connected to both the image acquisition device and the image processing module, and is used to receive image data transmitted by the image acquisition device and transmit it to the image processing module.
6. The FPGA circuit for image transmission according to claim 5, characterized in that, The FPGA circuit for image transmission also includes: A buffer module is connected to both the communication module and the image processing module, and is used to buffer the image data before outputting it to the image processing module.
7. The FPGA circuit for image transmission according to claim 6, characterized in that, The FPGA circuit for image transmission also includes: The data packaging module is connected to both the communication module and the buffer module, and is used to package the image data and output it to the buffer module.
8. The FPGA circuit for image transmission according to claim 7, characterized in that, The processing module is also connected to the communication module and the data packaging module respectively, and is used to configure the transmission of the image data in the communication module and the data packaging module respectively.
9. The FPGA circuit for image transmission according to claim 1, characterized in that, The FPGA circuit for image transmission also includes: The IIC / IO interface circuit is connected to the processing module and the image acquisition device. The processing module is also used to set parameters or read the status of the image acquisition device through the IIC / IO interface circuit according to the setting parameters.
10. An industrial camera, characterized in that, It includes an image acquisition device, a memory, and an FPGA circuit for image transmission as described in any one of claims 1 to 9.