Target identification and image compression transmission system based on FPGA and ARM architecture

Abstract

The invention provides a target identification and image compression transmission system based on FPGA and ARM architecture, and belongs to the field of image processing. The problem of low reliability caused by certain loss of resolution and frame frequency when a single ARM processes a high-resolution image and transmits the image is solved. The system comprises an ARM processor which receives a template data updating instruction sent by a main control module and sends image data to the main control module; the FPGA chip is connected with the ARM through a PCIE (Peripheral Component Interface Express) structure, receives the control information sent by the ARM processor and sends template data and image data to the ARM processor; the camera is used for collecting image data and sending the image data to the FPGA chip; and the template injection module is used for receiving an injection template updating instruction sent by the FPGA chip, converting protocol data and sending the protocol data back to the FPGA chip. The method is mainly used in the fields of video image acquisition and display, target identification and image compression and transmission.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and in particular relates to a target recognition and image compression transmission system based on FPGA and ARM architecture. Background Art

[0002] In the field of video image target recognition and image transmission, single ARM is mostly used for processing. With the advancement of technology, the resolution of detectors is getting larger and larger. Therefore, in the field of target recognition and image transmission, there are more and more scenarios for using large-resolution detectors. Due to the performance limitations of a single ARM, there will be a certain loss of resolution and frame rate when processing and transmitting large-resolution images. At the same time, the collected raw data cannot be output and displayed in real time.

[0003] In summary, a target recognition and image compression transmission system based on FPGA and ARM architecture is proposed. Summary of the Invention

[0004] In view of this, the present invention aims to propose a target recognition and image compression transmission system based on FPGA and ARM architecture to solve the problem of low reliability due to certain loss of resolution and frame rate when a single ARM processes large-resolution images and transmits images.

[0005] To achieve the above objectives, the present invention adopts the following technical solutions to provide a target recognition and image compression transmission system based on FPGA and ARM architecture, including: The ARM processor receives the template data update instruction sent by the main control module (1) and sends the image data to the main control module (1); The FPGA chip is connected to the ARM processor through the PCIE structure, receives the control information sent by the ARM processor, and sends template data and image data to the ARM processor; The camera collects image data and sends it to the FPGA chip; The template injection module (7) receives the template injection update instruction sent by the FPGA chip, converts the protocol data, and sends it back to the FPGA chip.

[0006] Furthermore, the ARM processor includes an ARM control information receiving and sending module (2) and an ARM_PCIE read-write control module (3). The ARM control information receiving and sending module (2) receives the template data update instruction and decodes it, and sends the control information to the FPGA chip through the ARM_PCIE read-write control module (3).

[0007] Furthermore, the ARM processor further comprises a target recognition module (11), an image compression module (12) and a compressed data sending module (13), wherein the target recognition module (11) receives, recognizes and marks the digital image data sent by the ARM_PCIE read-write control module (3), and transmits the image data with the target mark to the image compression module (12), and the compressed data sending module (13) receives the image compressed by the image compression module (12), packages it and sends it to the main control module (1).

[0008] Furthermore, the FPGA chip includes an FPGA_PCIE read-write control module (4) connected to the ARM_PCIE read-write control module (3).

[0009] Furthermore, the FPGA chip further comprises an FPGA control information receiving and sending module (5), and the FPGA_PCIE read-write control module (4) receives the control instruction of the ARM_PCIE read-write control module (3) and sends it to the FPGA control information receiving and sending module (5).

[0010] Furthermore, the FPGA chip further includes a CSI-MIPI protocol decoding module (8) and a DDR3 read-write control module (9), and the FPGA control information receiving and sending module (5) decodes the control instruction and sends it to the CSI-MIPI protocol decoding module (8) and the DDR3 read-write control module (9).

[0011] Furthermore, the CSI-MIPI protocol decoding module (8) receives data from the camera and template injection module (7) and sends it to the DDR3 read-write control module (9).

[0012] Furthermore, the DDR3 read / write control module (9) stores the image data and the injected template data sent by the CSI-MIPI protocol decoding module (8) in different addresses of the DDR3 (10); the DDR3 read / write control module (9) reads the image data and the injected template data in the DDR3 (10) according to the sending timing of the FPGA_PCIE read / write control module (4) and sends the data to the FPGA_PCIE read / write control module (4); the DDR3 read / write control module (9) reads the image data in the DDR3 (10) according to the Cameralink output timing and outputs it to the Cameralink output display (14).

[0013] Furthermore, the FPGA_PCIE read-write control module (4) receives the digital image data sent by the DDR3 read-write control module (9) and sends it to the ARM_PCIE read-write control module (3) through the PCIE hardware; the FPGA_PCIE read-write control module (4) controls the sending of the injection template to the ARM_PCIE read-write control module (3) according to the received control instruction.

[0014] Furthermore, the camera is a 4K resolution visible light movement (6).

[0015] Beneficial effects: 1. Combining the parallel processing advantages of FPGA in image acquisition and transmission with the advantages of ARM in image algorithm processing, FPGA and ARM are interconnected through the PCIE interface to meet the acquisition, display, target recognition and image compression and transmission functions of 4K video images, improving the system's feasibility and reliability.

[0016] It reduces the system development cost, ensures the maintainability of the algorithm, and meets the various usage environment requirements of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which constitute part of the present invention, are provided to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are provided to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings: Figure 1 This is a diagram of a target recognition and image compression transmission system based on FPGA and ARM architecture according to the present invention; In the figure: main control module (1); ARM control information receiving and sending module (2); ARM_PCIE read and write control module (3); FPGA_PCIE read and write control module (4); FPGA control information receiving and sending module (5); 4K resolution visible light movement (6); template injection module (7); CSI-MIPI protocol decoding module (8); DDR3 read and write control module (9); DDR3 (10); target recognition module (11); image compression module (12); compressed data sending module (13); Cameralink output display (14). DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely explain the technical solutions in the embodiments of the present invention. It should be noted that the embodiments of the present invention and the features therein can be combined with each other in the absence of conflict, and the embodiments described are only part of the embodiments of the present invention, not all of the embodiments.

[0019] It should be noted that the descriptions of the present invention regarding directions such as "left", "right", "left side", "right side", "upper", "lower", "top", and "bottom" are all defined based on the relationship between the orientations or positions shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the structure must be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention. In the description of the present invention, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0020] In the description of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to direct connections, indirect connections through an intermediary, or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances. Specific implementation method one: Referring to the accompanying drawings, this embodiment provides a target recognition and image compression transmission system based on FPGA and ARM architecture, including: The ARM processor receives the template data update instruction sent by the main control module (1) and sends the image data to the main control module (1); the first main control module (1) sends the template data update instruction to the ARM control information receiving and sending module (2) according to user needs; the main control module (1) receives the compressed image data sent by the compressed data sending module (13) and performs image decoding and display according to the compression protocol.

[0022] The FPGA chip is connected to the ARM processor through the PCIE structure, receives the control information sent by the ARM processor, and sends template data and image data to the ARM processor; The camera collects image data and sends it to the FPGA chip; The template injection module (7) receives the template injection update instruction sent by the FPGA chip, converts the protocol data, and sends it back to the FPGA chip. The template injection module (7) receives the template injection instruction sent by the FPGA control information receiving and sending module (5) and converts the new template into 4-lane CSI-MIPI protocol data, and sends the data to the CSI-MIPI protocol decoding module (8).

[0023] In this embodiment, the ARM processor comprises an ARM control information receiving and sending module (2) and an ARM_PCIE read-write control module (3). The ARM control information receiving and sending module (2) receives an instruction to update template data and decodes it, and sends the control information to the FPGA chip through the ARM_PCIE read-write control module (3). The ARM control information receiving and sending module (2) decodes the control information sent by the main control module (1) and sends it to the FPGA_PCIE read-write control module (4) through the ARM_PCIE read-write control module (3); the ARM_PCIE read-write control module (3) receives the control instruction information sent by the ARM control information receiving and sending module (2), and sends the control instruction information to the FPGA_PCIE read-write control module (4); the ARM_PCIE read-write control module (3) determines whether to update the injected template data according to the control instruction, and if the control instruction is that the injected template needs to be updated, the template data sent by the FPGA_PCIE read-write control module (4) is received and the template data is updated; the ARM_PCIE read-write control module (3) receives the digital image data sent by the FPGA_PCIE read-write control module (4) and sends it to the target recognition module (11) for reception.

[0024] In this embodiment, the ARM processor further comprises a target recognition module (11), an image compression module (12) and a compressed data sending module (13). The target recognition module (11) receives, recognizes and marks the digital image data sent by the ARM_PCIE read / write control module (3), and transmits the image data with the target mark to the image compression module (12). The compressed data sending module (13) receives the image compressed by the image compression module (12), packages it and sends it to the main control module (1). The target recognition module (11) receives the image data transmitted by the ARM_PCIE read / write control module (3), and simultaneously performs target recognition and marking in combination with the existing injection template, and transmits the image data with the target mark to the image compression module (12); the image compression module (12) receives the image data with the target mark sent by the target recognition module (11) and compresses it according to the H.265 protocol, and sends the compressed image to the compressed data sending module (13); the compressed data sending module (13) performs UDP packaging on the compressed image data sent by the image compression module (12) and sends it to the main control module (1).

[0025] In this embodiment, the FPGA chip includes an FPGA_PCIE read-write control module (4) connected to the ARM_PCIE read-write control module (3). The FPGA_PCIE read-write control module (4) receives a control instruction sent by the ARM_PCIE read-write control module (3) and transmits the instruction to the FPGA control information receiving and transmitting module (5); the FPGA_PCIE read-write control module (4) receives digital image data sent by the DDR3 read-write control module (9) and transmits the digital image data to the ARM_PCIE read-write control module (3) through the PCIE hardware; and the FPGA_PCIE read-write control module (4) controls the sending of the injection template to the ARM_PCIE read-write control module (3) according to the received control instruction.

[0026] In this embodiment, the FPGA chip further includes an FPGA control information receiving and transmitting module (5), and the FPGA_PCIE read-write control module (4) receives the control instruction of the ARM_PCIE read-write control module (3) and sends it to the FPGA control information receiving and transmitting module (5). The FPGA control information receiving and transmitting module (5) receives the control instruction sent by the FPGA_PCIE read-write control module (4) and decodes it, and sends the decoded instruction to the CSI-MIPI protocol decoding module (8) and the DDR3 read-write control module (9).

[0027] In this embodiment, the FPGA chip further includes a CSI-MIPI protocol decoding module (8) and a DDR3 read-write control module (9), and the FPGA control information receiving and sending module (5) decodes the control instruction and sends it to the CSI-MIPI protocol decoding module (8) and the DDR3 read-write control module (9).

[0028] In this embodiment, the CSI-MIPI protocol decoding module (8) receives data from the camera and template injection module (7) and sends it to the DDR3 read-write control module (9). The CSI-MIPI protocol decoding module (8) receives image data sent by the 4K resolution visible light core (6) and sends the data to the DDR3 read-write control module (9); the CSI-MIPI protocol decoding module (8) receives the injection template instruction sent by the FPGA control information receiving and sending module (5), and receives the new injection template data from the template injection module (7), and sends the data to the DDR3 read-write control module (9) after decoding.

[0029] In this embodiment, the DDR3 read / write control module (9) stores the image data and injection template data sent by the CSI-MIPI protocol decoding module (8) in different addresses of the DDR3 (10); the DDR3 read / write control module (9) reads the image data and injection template data in the DDR3 (10) according to the sending timing of the FPGA_PCIE read / write control module (4) and sends the data to the FPGA_PCIE read / write control module (4); the DDR3 read / write control module (9) reads the image data in the DDR3 (10) according to the Cameralink output timing and outputs it to the Cameralink output display (14). The DDR3 read / write control module (9) stores the image data and injection template data sent by the CSI-MIPI protocol decoding module (8) in different addresses of (10); the DDR3 read / write control module (9) reads the image data and injection template data in (10) according to the sending timing of the FPGA_PCIE read / write control module (4) and sends the data to the FPGA_PCIE read / write control module (4); the DDR3 read / write control module (9) reads the image data in (10) according to the Cameralink output timing and outputs it to the Cameralink output display (14).

[0030] In this embodiment, the FPGA_PCIE read-write control module (4) receives the digital image data sent by the DDR3 read-write control module (9) and sends it to the ARM_PCIE read-write control module (3) through the PCIE hardware; the FPGA_PCIE read-write control module (4) controls the sending of the injection template to the ARM_PCIE read-write control module (3) according to the received control instruction.

[0031] In this embodiment, the camera is a 4K resolution visible light engine (6). The 4K resolution visible light engine (6) converts the 4K resolution image data of the CMOS into 4-lane CSI-MIPI protocol data and sends the data to the CSI-MIPI protocol decoding module (8).

[0032] Working principle: The main control module (1) sends the instruction of the template data to be updated according to the user's needs to the ARM control information receiving and sending module (2); the main control module (1) receives the compressed image data sent by the compressed data sending module (13) and decodes and displays the image according to the compression protocol; the ARM control information receiving and sending module (2) decodes the control information sent by the main control module (1) and sends it to the FPGA_PCIE read-write control module (4) through the ARM_PCIE read-write control module (3); the ARM_PCIE read-write control module (3) receives the control instruction information sent by the ARM control information receiving and sending module (2) and sends the control instruction information to the FPGA_PCIE read-write control module (4). The FPGA_PCIE read-write control module (4) determines whether to update the injected template data according to the control instruction. If the control instruction is that the injected template needs to be updated, the template data sent by the FPGA_PCIE read-write control module (4) is received and the template data is updated. The ARM_PCIE read-write control module (3) receives the digital image data sent by the FPGA_PCIE read-write control module (4) and sends it to the target recognition module (11). The FPGA_PCIE read-write control module (4) receives the control instruction sent by the ARM_PCIE read-write control module (3) and passes the instruction to the FPGA control information receiving module (11). Receiving and sending module (5); FPGA_PCIE read-write control module (4) receives digital image data sent by DDR3 read-write control module (9) and sends it to ARM_PCIE read-write control module (3) through PCIE hardware; FPGA_PCIE read-write control module (4) controls the sending of injection template to ARM_PCIE read-write control module (3) according to the received control instruction; FPGA control information receiving and sending module (5) receives the control instruction sent by FPGA_PCIE read-write control module (4) and decodes it, and sends the decoded instruction to CSI-MIPI protocol decoding module (8) and DDR3 read-write control module (9); 4K The high-resolution visible light core (6) converts the CMOS 4K resolution image data into 4-lane CSI-MIPI protocol data and sends the data to the CSI-MIPI protocol decoding module (8); the template injection module (7) receives the template injection instruction sent by the sending module (5) according to the received FPGA control information, converts the new template into 4-lane CSI-MIPI protocol data, and sends the data to the CSI-MIPI protocol decoding module (8); the CSI-MIPI protocol decoding module (8) receives the image data sent by the 4K resolution visible light core (6), and sends the data to the DDR3 read-write control module (9);The CSI-MIPI protocol decoding module (8) receives the injection template instruction sent by the FPGA control information receiving and sending module (5), and receives the new injection template data from the template injection module (7), and sends the data to the DDR3 read-write control module (9) after decoding; the DDR3 read-write control module (9) stores the image data and injection template data sent by the CSI-MIPI protocol decoding module (8) to different addresses of the DDR3 (10); the target recognition module (11) receives the image data transmitted by the ARM_PCIE read-write control module (3), and at the same time performs target recognition and marking in combination with the existing injection template, and passes the image data with the target mark to the image compression module (12). The image compression module (12) receives the image data with the target mark sent by the target recognition module (11) and compresses it according to the H.265 protocol, and sends the compressed image to the compression data sending module (13); the compression data sending module (13) packages the compressed image data sent by the image compression module (12) into UDP packets and sends them to the main control module (1); the Cameralink output display (14) receives the image data from the DDR3 read / write control module (9) and displays the image according to the timing. The FPGA chip can be xc7z100ffg900_2, and the ARM processor can be RK3588J.

[0033] The embodiments of the present invention disclosed above are intended only to illustrate the present invention. The embodiments do not describe all details in detail, nor do they limit the present invention to the specific embodiments described. Numerous modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention.

Claims

1. A target recognition and image compression transmission system based on FPGA and ARM architecture, characterized in that: include: The ARM processor receives the template data update instruction sent by the main control module (1) and sends the image data to the main control module (1); The FPGA chip is connected to the ARM processor through the PCIE structure, receives the control information sent by the ARM processor, and sends template data and image data to the ARM processor; The camera collects image data and sends it to the FPGA chip; The template injection module (7) receives the template injection update instruction sent by the FPGA chip, converts the protocol data, and sends it back to the FPGA chip.

2. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 1, characterized in that: The ARM processor comprises an ARM control information receiving and sending module (2) and an ARM_PCIE read-write control module (3). The ARM control information receiving and sending module (2) receives an instruction to update template data and decodes the instruction, and sends the control information to the FPGA chip via the ARM_PCIE read-write control module (3).

3. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 2, characterized in that: The ARM processor further comprises a target recognition module (11), an image compression module (12) and a compressed data sending module (13). The target recognition module (11) receives, recognizes and marks digital image data sent by the ARM_PCIE read / write control module (3), and transmits the image data with the target mark to the image compression module (12). The compressed data sending module (13) receives the image compressed by the image compression module (12), packages it and sends it to the main control module (1).

4. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 3, characterized in that: The FPGA chip comprises an FPGA_PCIE read-write control module (4) connected to the ARM_PCIE read-write control module (3).

5. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 4, characterized in that: The FPGA chip further comprises an FPGA control information receiving and sending module (5), and the FPGA_PCIE read-write control module (4) receives control instructions from the ARM_PCIE read-write control module (3) and sends the control instructions to the FPGA control information receiving and sending module (5).

6. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 5, characterized in that: The FPGA chip further comprises a CSI-MIPI protocol decoding module (8) and a DDR3 read-write control module (9); the FPGA control information receiving and transmitting module (5) decodes control instructions and transmits them to the CSI-MIPI protocol decoding module (8) and the DDR3 read-write control module (9).

7. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 6, characterized in that: The CSI-MIPI protocol decoding module (8) receives data from the camera and template injection module (7) and sends it to the DDR3 read-write control module (9).

8. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 7, characterized in that: The DDR3 read / write control module (9) stores the image data and injection template data sent by the CSI-MIPI protocol decoding module (8) in different addresses of the DDR3 (10); the DDR3 read / write control module (9) reads the image data and injection template data in the DDR3 (10) according to the sending timing of the FPGA_PCIE read / write control module (4) and sends the data to the FPGA_PCIE read / write control module (4); the DDR3 read / write control module (9) reads the image data in the DDR3 (10) according to the Cameralink output timing and outputs it to the Cameralink output display (14).

9. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 8, characterized in that: The FPGA_PCIE read-write control module (4) receives the digital image data sent by the DDR3 read-write control module (9) and sends it to the ARM_PCIE read-write control module (3) through the PCIE hardware; the FPGA_PCIE read-write control module (4) controls the sending of the injection template to the ARM_PCIE read-write control module (3) according to the received control instruction.

10. The target recognition and image compression transmission system based on FPGA and ARM architecture according to claim 1, characterized in that: The camera is a 4K resolution visible light camera (6).

Images