Deployment platform and test method of image algorithm hardware acceleration engine based on ZYNQ
By integrating multiple image data input methods and unified data stream processing on the ZYNQ platform, the problems of real-time and power consumption performance in image algorithm hardware acceleration are solved, rapid application and simplified deployment are achieved, and the real-time performance of image algorithms is improved. and efficiency.
Patent Information
- Application Number
- CN202411105069.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-29
AI Technical Summary
In image processing applications, it is difficult for the prior art to achieve hardware acceleration of real-time processing and low-power image algorithms, and the hardware-based algorithms require separate development of inputs and outputs, resulting in high deployment difficulty and waste of time.
A ZYNQ-based image algorithm hardware acceleration engine deployment platform is designed, integrating multiple image data input methods, formatting and preprocessing, and algorithm processing is carried out through a unified AXI-Stream data stream, providing multiple interactive channel output results. , reducing the difficulty of deploying hardware algorithms.
It realizes the rapid application of image algorithms, reduces code repetition, improves the real-time and power consumption performance of hardware algorithms, and simplifies the deployment and testing of single or multiple algorithm modules.
Smart Images

Figure CN119048329A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image algorithm hardware acceleration, and in particular to a deployment platform and a testing method of an image algorithm hardware acceleration engine based on ZYNQ. Background Art
[0002] In actual image processing applications, algorithm processing time and processing power consumption are important factors that limit the application of image algorithms. In actual applications, if the image algorithm cannot achieve real-time processing, the detection results of the algorithm are outdated, and the entire application is meaningless. Similarly, the higher the complexity of the algorithm, the more computing power the algorithm requires, and the more computing power the corresponding power consumption and weight are. In actual needs, computing power corresponding to excessive power consumption or weight is not suitable for application.
[0003] ZYNQ is a SoC (system-on-chip integrated circuit) product series based on ARM Cortex-A9 processor and FPGA launched by Xilinx. The ZYNQ series chips combine the advantages of traditional processors and programmable logic, and can provide high-performance, low-power, programmable and scalable processing capabilities. ZYNQ chips are also equipped with a variety of interfaces to facilitate the connection of various peripherals. ZYNQ chips are mainly used in the fields of high-performance, low-power, real-time computing and signal processing. The hardware acceleration of the algorithm is to transform the algorithm into a fixed hardware circuit. Therefore, the algorithm operation does not rely on the instruction set processing of the CPU architecture, and the level is low. The corresponding real-time performance and power consumption performance can be greatly improved. However, there is a problem with the circuit-level algorithm: it cannot be directly run at the system level like most conventional software algorithms, and the corresponding hardware input and output need to be configured. Therefore, the hardware-based algorithms need to implement data input and output separately. The input of the image algorithm is basically single-frame or multi-frame image data, and the output is mostly processed image data stream. If each hardware-based image algorithm needs to develop input and output separately, it will be a great waste of time. Therefore, how to design a ZYNQ-based hardware acceleration deployment platform for image algorithms based on the common IO of image algorithm acceleration engines to quickly apply hardware algorithms has become an important research topic for technical personnel in this field. Summary of the invention
[0004] The purpose of the present invention is to provide a deployment platform and testing method for an image algorithm hardware acceleration engine based on ZYNQ, which integrates various image data input methods, formats and preprocesses them, deploys multiple algorithms, and outputs algorithm processing results through different interactive channels after algorithm processing, thereby reducing the difficulty of hardware algorithm deployment, avoiding repeated code writing, and facilitating the rapid application of image algorithms.
[0005] To achieve the above object, the present invention provides the following solutions: A deployment platform for an image algorithm hardware acceleration engine based on ZYNQ, including: Multiple image data input modules, each of which includes an image data source interface, a driver corresponding to the image data source, and a corresponding data cache queue FIFO. The image data source interface is used to access an external image data source. The driver corresponding to the image data source is used to parse and obtain image data from the image data source and convert different protocol data into common data. The data cache queue FIFO is used to cache input image data and isolate the impact between modules. The image acquisition master driver is used to detect whether there is image data in the multiple data cache queues FIFO, and when it is detected that there is image data in one or more data cache queues FIFO, pre-process the image data and transfer the pre-processed data to the input data cache queue of the frame cache; The image processing unit is used to read out the image data when detecting that there is image data in the input data cache queue, format and pre-process the image data according to the input format requirements of the deployed algorithm, and transmit the formatted and pre-processed data to the image algorithm unit, and send a start timing signal to the timing module at the same time; The timing module is used to record the time when the algorithm starts executing and to perform timing; The image algorithm unit is deployed with several algorithm modules. Each algorithm module processes the received data according to its own deployed algorithm. After the algorithm is executed, the algorithm processing result is transmitted to the interaction module, and an end timing signal is sent to the timing module at the same time; The timing module is also used to record the time point when the algorithm ends execution, and to count the algorithm processing time and transmit it to the interaction module; The interactive module is used to output the algorithm processing results according to the preset interactive channel.
[0006] Further, the plurality of image data input modules include a DC data input module, a storage input module, and a camera input module; The DC data input module includes a DC data input interface, a DC drive and a corresponding data cache queue FIFO. The PC sends binary data to the DC input interface based on the DC data corresponding transmission protocol. The DC drive obtains image data through the DC data corresponding transmission protocol analysis and stores the image data in the corresponding data cache queue FIFO. The storage input module includes a storage input interface, a storage driver and a corresponding data cache queue FIFO. The image data is pre-written into the storage module, and then the storage module is connected to the storage input interface. The image data in the storage module is read out through the storage driver, and the image data is processed according to a preset format, and then stored in the corresponding data cache queue FIFO; The camera input module includes a camera input interface, a camera driver and a corresponding FIFO. The camera obtains image data through an image sensor and then outputs it to the camera input interface through a corresponding transmission protocol. The camera driver obtains image data of the camera's video stream according to the corresponding frame format and then stores it in the corresponding data cache queue FIFO.
[0007] Furthermore, the image acquisition master driver is also used for: When it is detected that there is image data in a single FIFO, the image data in the current FIFO is preprocessed; When it is detected that there is image data in multiple FIFOs, the image data in the corresponding FIFOs are read in sequence according to the preset priority for preprocessing.
[0008] Furthermore, the image acquisition master driver is used to pre-process the image data, specifically including: (1) Format the images in the image data in the current FIFO, including cropping, grayscale, and medianization; (2) Generate a 64-bit descriptor corresponding to the image in the image data. The descriptor includes the image length, width, color mode, and input channel; (3) Place the descriptor in the data frame header, followed by the image data, and output it to the input data cache queue of the frame buffer.
[0009] Furthermore, the image processing unit includes an image format detection module, an image formatting module and an image preprocessing module. The image format detection module is used to detect that there is data in the input data cache queue of the frame cache, read out the image data, and when obtaining the frame header of the image data, detect whether the format requirements of the deployed algorithm are met, and if not, input it to the image formatting module; An image formatting module, used to format image data according to the input format requirements of the deployed algorithm; The image preprocessing module is used to preprocess the formatted image data, including image enhancement processing. After processing, the preprocessed image data is output to the image algorithm unit through the AXI-Steam stream, and a start timing signal is sent to the timing module.
[0010] Furthermore, multiple algorithm modules use a unified AXI-Steam data stream as input and output format.
[0011] Further, the interaction module includes an output overall driving module and three preset interaction channels; The three preset interaction channels are: The first interactive channel is used to output a video stream processed by a single algorithm through the HDMI protocol; The second interactive channel is used to output the comparison video streams processed by multiple algorithms; The third interactive channel is used to output the position coordinate information of the target to be measured in the image data.
[0012] Furthermore, the interaction module also includes a sending data formatting module, which is used to format the algorithm processing results according to a preset output method to adapt to the output protocols corresponding to different preset interaction channels, and store them in an output data cache queue. The output general driver module reads the data in the output data cache queue.
[0013] Furthermore, the deployment platform also includes a DDR3 module, and the DDR3 module is used to store the data formatted and pre-processed by the image processing unit.
[0014] The present invention also provides a ZYNQ-based image algorithm hardware acceleration engine testing method, which is applied to the above-mentioned ZYNQ-based image algorithm hardware acceleration engine deployment platform, and includes the following steps: Obtain image data in multiple input formats provided by different image data sources, and store them in corresponding data cache queues FIFO respectively; Detect whether there is image data in multiple data buffer queues FIFOs, and when it is detected that there is image data in one or more FIFOs, pre-process the image data and transmit the pre-processed data to the input data buffer queue of the frame buffer; When it is detected that there is image data in the input data buffer queue of the frame buffer, the image data is read out, and the image data is formatted and preprocessed according to the input format requirements of the deployed algorithm; Process the received data according to the deployed algorithm, record the algorithm start processing time and algorithm execution end time, calculate the algorithm processing time, and obtain the algorithm processing result; Output the algorithm processing results according to the preset interactive channel.
[0015] According to the specific embodiment provided by the present invention, the deployment platform and testing method of the image algorithm hardware acceleration engine based on ZYNQ provided by the present invention disclose the following technical effects: (1) A variety of common input methods are integrated together and formatted by the overall driver, and then output using the standard high-bandwidth AXI-Stream. When algorithm deployment testing is required, the image data is input into the algorithm through the standard protocol, and after algorithm processing, it is output to the interactive module using AXI-Stream. The interactive module displays the algorithm results through a variety of preset interactive channels. Therefore, based on the common IO of the image algorithm acceleration engine, the present invention designs a hardware acceleration deployment platform for image algorithms based on ZYNQ to quickly apply hardware algorithms. (2) For a single image algorithm acceleration engine, a single algorithm module can be directly deployed on the platform to reduce the workload of external rewriting IO expansion; the algorithm hardware implementation plus this platform can be directly packaged into a product to speed up product formation; in addition, multiple algorithm modules can be deployed on the platform at the same time to implement the image algorithm acceleration engine, and the same image data stream on the same platform can be used for testing and comparison. For different image algorithm acceleration engines, the processing effects of the algorithms can be compared; for different acceleration structure implementations of the same algorithm, an effective acceleration structure can be selected to speed up the optimization of the algorithm hardware structure; (3) Build a deployment platform based on ZYNQ to provide a common operating environment for single or multiple image accelerators implemented through FPGA. The entire ZYNQ chip includes the PS side (ARM) and the PL side (FPGA). Since the ARM side contains the Linux system, it can collect and schedule data. Therefore, the entire deployment platform mainly collects image data from the IO input of the PS side (ARM), that is, the SD card, camera, and UART input are all on the PS side. Then convert it into a unified high-bandwidth and high-speed protocol AXI-Stream and flow it into the PL side (FPGA). Using the high parallelism and low latency characteristics of FPGA, it is responsible for the image preprocessing algorithm and the deployed image single or multiple algorithms. Among them, FPGA may use other additional resources connected to PL, including IO and DDR3, etc., to facilitate adjustment according to the resources and computing power scale required by the deployed algorithm. Finally, the result of the algorithm calculation in FPGA is transmitted to the PS side through AXI-Stream, and the final result is transmitted through the total output driver through different interactive protocols. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0017] Figure 1This is an architecture diagram of a deployment platform of an image algorithm hardware acceleration engine based on ZYNQ in the present invention; Figure 2 This is a data flow chart of the deployment platform of the ZYNQ-based image algorithm hardware acceleration engine; Figure 3 This is a peripheral IO protocol diagram of the deployment platform of the image algorithm hardware acceleration engine based on ZYNQ of the present invention; Figure 4 This is a board diagram of the deployment platform application case of the image algorithm hardware acceleration engine based on ZYNQ of the present invention, where 4-1 is the PC end through UART input, 4-2 is the SD card input, 4-3 is the camera input, 4-4 is the HDMI1 output, and 4-5 is the HDMI2 output. Figure 4 The middle board card is only a carrier for platform implementation and an implementation example. The carrier board card does not fall within the scope of protection. DETAILED DESCRIPTION
[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] The purpose of the present invention is to provide a deployment platform and test method for an image algorithm hardware acceleration engine based on ZYNQ, which integrates common data acquisition methods for image algorithms, including direct real-time input of image sequences based on cameras and storage and then input of image data based on SD cards. UART directly inputs binary image data to manually detect the detection results of the algorithm on familiar images.
[0020] The deployment platform used in the present invention is developed based on the Xilinx ZYNQ 7000 series, and multiple data input and output data channels will be scheduled by the CPU based on the Linux system. However, the algorithm part does not require the participation of the CPU and is a pure circuit. The platform reduces the difficulty of hardware algorithm deployment, avoids repeated code writing, and provides a paradigm for the rapid application of the algorithm.
[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0022] like Figure 1-Figure 4As shown, the deployment platform of the image algorithm hardware acceleration engine based on ZYNQ provided by the present invention includes multiple image data input modules, an image acquisition general driver, an image processing unit, a timing module, an image algorithm unit and an interaction module; (1) If Figure 3 and Figure 4 The multiple image data input modules shown include a DC data input module, a storage input module, and a camera input module; exemplarily, a UART input module, an SD card input module, and a camera input module; (1.1) For the input of DC data, it is used to test whether the system is operating normally. The data transmission using the UART protocol is an embodiment. The actual transmission protocol can also be Ethernet, VGA, HDMI, PCIe, etc. The UART input module includes a UART input interface, a UART driver, and a corresponding FIFO. The PC end sends binary data to the UART input interface through the UART with the serial port assistant. The UART driver obtains image data through UART protocol analysis and stores the image data in the corresponding FIFO; wherein, the UART protocol analysis includes: first de-framing the data, pre-processing the de-framing data according to the preset length and width format, and obtaining the image data; (1.2) The SD card input module includes an SD card input interface, an SD card driver and a corresponding FIFO. The data of the SD card is written to the SD card in advance through the FATFS system. Then the SD card is connected to the SD card input interface, and the image data in the SD card is read out through the SD card driver. The image data is processed according to the preset length and width format, and then stored in the corresponding FIFO; (1.3) The camera input module includes a camera input interface, a camera driver, and a corresponding FIFO. For example, the camera model is ov5640, DDR3 is used as a frame buffer, and the data transmission protocol is MIPI-CSI. The actual application of the present invention is not limited to the camera model and the corresponding transmission protocol. For example, the camera is ov7060, and the protocol is IIC, Cameralink, etc. The camera obtains image data through the image sensor, and then outputs it to the camera input interface through the corresponding transmission protocol. The camera driver obtains the image data of the camera's video stream according to the corresponding frame format, pre-processes the image data, and then flows into the corresponding FIFO. For example, the image data enters DDR3 through the camera ov5640 for frame buffering, and then flows into the corresponding driver. After simple pre-processing such as cropping and formatting, the image data flows into the corresponding FIFO.
[0023] In addition to the three input methods provided above, image data can also be input in a wireless manner, etc.
[0024] (2) The image acquisition master driver is used to detect whether there are image data in the multiple FIFOs. When it is detected that there is image data in a single FIFO, the image data in the current FIFO is preprocessed; when it is detected that there is image data in multiple FIFOs, the image data in the corresponding FIFOs are read in sequence according to the preset priority for preprocessing, and the preprocessed data is transferred to the input data cache queue DATA_FIFO of the frame buffer; while caching data, DATA_FIFO can also cache data width, image size, color mode (RGB888 / RGB565, etc.) and other interpretation fields.
[0025] The image acquisition master driver is used to pre-process the image data, specifically including: (2.1) Format the images in the image data in the current FIFO, including cropping, grayscale, median, etc.; (2.2) Generate a 64-bit descriptor corresponding to the image in the image data. The descriptor includes information such as the image length, width, color mode, and input channel; (2.3) Place the descriptor in the data frame header, followed by the image data, and output it to the input data cache queue DATA_FIFO of the frame buffer.
[0026] (3) An image processing unit, which is used to read out the image data when detecting that there is image data in the input data cache queue DATA_FIFO of the frame cache, and write the read data into DDR3 for frame cache, and format and pre-process the image data according to the input format requirements of the deployed algorithm, and transmit the formatted and pre-processed data to the image algorithm unit, and send a start timing signal to the timing module; specifically, the image processing unit includes an image format detection module, an image formatting module and an image pre-processing module. (3.1) The image format detection module is used to read out the image data when it detects that there is data in the input data cache queue DATA_FIFO of the frame cache. When obtaining the frame header of the image data, it detects whether the format requirements of the deployed algorithm are met. If not, it is input to the image formatting module.
[0027] (3.2) Image formatting module, which is used to format image data according to the input format requirements of the deployed algorithm. Formatting is to format the data stream. Different image input formats, such as SD card, camera and test channel UART, have different data bandwidths and rates. The input of the algorithm is designed according to the highest rate of the algorithm, so it is easy to have a mismatch in rate and bandwidth. Therefore, the input data is first formatted into a format that matches the algorithm.
[0028] (3.3) Image preprocessing module, used to preprocess the formatted image data, including image enhancement processing, etc. After processing, the preprocessed image data is output to the image algorithm unit through the AXI-Steam stream, and a start timing signal is sent to the timing module (pulling the timing module signal high). The timing module is used to record the time point when the algorithm starts executing and to perform timing. Preprocessing mainly normalizes the data before the algorithm is input so that the input image size, pixel mode and other conditions are the same. Preprocessing includes operations such as ISP algorithm and cropping.
[0029] (4) The image algorithm unit is deployed with several algorithm modules. Each algorithm module processes the received data according to its own deployed algorithm. After the algorithm is executed, the algorithm processing result is transmitted to the interaction module, and an end timing signal is sent to the timing module at the same time. The timing module is also used to record the time point when the algorithm ends execution, and to count the algorithm processing time and transmit it to the interaction module.
[0030] The deployed algorithm module can be one or more, but they all belong to the algorithm IP core, using the unified AXI-Steam data stream as input and output. The specific bit width of AXI-Steam can be set to 32bit, 64bit, 128bit, etc. The specific value is set by weighing the maximum parallel width of the comprehensive algorithm and the upper limit bit width of AXI-Steam 1024bit. After the data is processed by the algorithm, the timing signal is turned off at the same time, and the results and time of the algorithm processing are output to the sending data formatting module.
[0031] (5) Interaction module, used to output the algorithm processing results according to the preset interaction channel. When the interaction module detects that there is data in the FIFO, it will read the data and output it according to the data channel preset by the user.
[0032] The interaction module includes an output overall driving module and three preset interaction channels; The three preset interaction channels are: The first interactive channel is used to output a video stream processed by a single algorithm through the HDMI protocol; The second interactive channel is used to output the comparison video streams processed by multiple algorithms; The third interactive channel is used to output the position coordinate information of the target to be measured in the image data.
[0033] The interaction module also includes a sending data formatting module, which is used to format the algorithm processing results according to a preset output method to adapt to the output protocols corresponding to different preset interaction channels, and store them in an output data cache queue. The output general driver module reads the data in the output data cache queue.
[0034] The present invention also provides a ZYNQ-based image algorithm hardware acceleration engine testing method, which is applied to the above-mentioned ZYNQ-based image algorithm hardware acceleration engine deployment platform, and includes the following steps: Obtain image data in multiple input formats provided by different image data sources, and store them in corresponding data cache queues FIFO respectively; Detect whether there is image data in multiple data buffer queues FIFOs, and when it is detected that there is image data in one or more FIFOs, pre-process the image data and transmit the pre-processed data to the input data buffer queue of the frame buffer; When it is detected that there is image data in the input data buffer queue of the frame buffer, the image data is read out, and the image data is formatted and preprocessed according to the input format requirements of the deployed algorithm; Process the received data according to the deployed algorithm, record the algorithm start processing time and algorithm execution end time, calculate the algorithm processing time, and obtain the algorithm processing result; Output the algorithm processing results according to the preset interactive channel.
[0035] In a specific embodiment, the image algorithm hardware acceleration engine testing method based on ZYNQ includes the following steps: (1) The PC sends binary image data to the serial assistant through UART. The driver Derict_Driver obtains the image data through UART protocol analysis and stores it in FIFO.
[0036] (2) The PC writes the image data to be tested to the SD card through the FATSF file system. The SD card driver SD_Driver obtains the image data in the SD through the SDIO protocol and then stores the data in the FIFO.
[0037] (3) The camera obtains image data through the image sensor and then outputs it with the MIPI-CSI protocol. The camera driver Camera_Driver obtains the camera's video stream data according to the corresponding frame format and then stores it in the FIFO.
[0038] (4) When the image acquisition driver Collect_Driver detects that there is data in a single FIFO, it pre-processes the image data input by the single driver and outputs it to the unified format module Data_Format after processing.
[0039] (5) When the image acquisition driver Collect_Driver detects that there is data in multiple FIFOs, it will read the data channel with the highest priority first for preprocessing according to the preset priority, and then send it to the format unification module Data_Format after processing.
[0040] (6) Data formatting The Data_Format module is mainly used to format the input data of the image acquisition driver Collect_Driver according to the input format requirements of the algorithm to be deployed.
[0041] (7) After the format is completed, all data will flow into algorithm module 0, algorithm module 1, algorithm module i, ..., algorithm module N, and will also flow into DDR3 to save the original data. The timing will start to count the algorithm processing time.
[0042] (8) When the algorithm processing is completed, the Display_Contral module will first end the statistical time, and then format it according to the preset output method to adapt to the corresponding protocol, and then output it to the display modules HDMI0, HDMI1 and LCM, and output the corresponding target coordinates at the same time.
[0043] In summary, the present invention mainly includes three parts: image acquisition, image processing and user interaction. The specific image data processing flow is as follows: image data enters the image acquisition driver through an SD card, a camera or a UART (universal asynchronous receiver / transmitter) data channel. The image driver reads the corresponding data into the image processing unit according to a preset inflow channel. All the incoming image data need to undergo unified image preprocessing, and then pass through a unified format module, flow into multiple algorithm modules, and start recording the processing time of the algorithm at the same time. Finally, after the algorithm is processed, the recording time ends, and the results and processing time of multiple algorithm processing flow into the interaction module. The interaction module mainly has three parts: that is, outputting multiple real-time processed comparison streams and image data streams processed by a single algorithm through HDMI, and outputting the position information of the target to be detected in the picture, so as to provide a reference for subsequent interaction instructions.
[0044] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of the present invention. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A deployment platform for image algorithm hardware acceleration engine based on ZYNQ, It is characterized in that include: Multiple image data input modules, each of which includes an image data source interface, a driver corresponding to the image data source, and a corresponding data cache queue FIFO. The image data source interface is used to access an external image data source. The driver corresponding to the image data source is used to parse and obtain image data from the image data source and convert different protocol data into common data. The data cache queue FIFO is used to cache input image data and isolate the impact between modules. The image acquisition master driver is used to detect whether there is image data in the multiple data cache queues FIFO, and when it is detected that there is image data in one or more data cache queues FIFO, pre-process the image data and transfer the pre-processed data to the input data cache queue of the frame cache; The image processing unit is used to read out the image data when detecting that there is image data in the input data cache queue, format and pre-process the image data according to the input format requirements of the deployed algorithm, and transmit the formatted and pre-processed data to the image algorithm unit, and send a start timing signal to the timing module at the same time; The timing module is used to record the time when the algorithm starts executing and to perform timing; The image algorithm unit is deployed with several algorithm modules. Each algorithm module processes the received data according to its own deployed algorithm. After the algorithm is executed, the algorithm processing result is transmitted to the interaction module, and an end timing signal is sent to the timing module at the same time; The timing module is also used to record the time point when the algorithm ends execution, and to count the algorithm processing time and transmit it to the interaction module; The interactive module is used to output the algorithm processing results according to the preset interactive channel.
2. According to claim 1, the deployment platform of the image algorithm hardware acceleration engine based on ZYNQ, It is characterized in that The plurality of image data input modules include a DC data input module, a storage input module, and a camera input module; The DC data input module includes a DC data input interface, a DC drive and a corresponding data cache queue FIFO. The PC sends binary data to the DC input interface based on the DC data corresponding transmission protocol. The DC drive obtains image data through the DC data corresponding transmission protocol analysis and stores the image data in the corresponding data cache queue FIFO. The storage input module includes a storage input interface, a storage driver and a corresponding data cache queue FIFO. The image data is pre-written into the storage module, and then the storage module is connected to the storage input interface. The image data in the storage module is read out through the storage driver, and the image data is processed according to a preset format, and then stored in the corresponding data cache queue FIFO; The camera input module includes a camera input interface, a camera driver and a corresponding FIFO. The camera obtains image data through an image sensor and then outputs it to the camera input interface through a corresponding transmission protocol. The camera driver obtains image data of the camera's video stream according to the corresponding frame format and then stores it in the corresponding data cache queue FIFO.
3. According to the deployment platform of the ZYNQ-based image algorithm hardware acceleration engine of claim 1, It is characterized in that The image acquisition master driver is also used for: When it is detected that there is image data in a single FIFO, the image data in the current FIFO is preprocessed; When it is detected that there is image data in multiple FIFOs, the image data in the corresponding FIFOs are read in sequence according to the preset priority for preprocessing.
4. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 1, It is characterized in that The image acquisition master driver is used to pre-process the image data, specifically including: (1) Formatting the images in the image data in the current FIFO, including cropping, grayscale, and medianization; (2) Generate a 64-bit descriptor corresponding to the image in the image data. The descriptor includes the image length, width, color mode, and input channel; (3) The descriptor is placed in the data frame header, followed by the image data, and output to the input data cache queue of the frame buffer.
5. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 1, It is characterized in that The image processing unit includes an image format detection module, an image formatting module and an image preprocessing module. The image format detection module is used to detect that there is data in the input data cache queue of the frame cache, read out the image data, and when obtaining the frame header of the image data, detect whether the format requirements of the deployed algorithm are met, and if not, input it to the image formatting module; An image formatting module, used to format image data according to the input format requirements of the deployed algorithm; The image preprocessing module is used to preprocess the formatted image data, including image enhancement processing. After processing, the preprocessed image data is output to the image algorithm unit through the AXI-Steam stream, and a start timing signal is sent to the timing module.
6. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 1, It is characterized in that Multiple algorithm modules use the unified AXI-Steam data stream as input and output format.
7. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 1, It is characterized in that The interaction module includes an output overall driving module and three preset interaction channels; The three preset interaction channels are: The first interactive channel is used to output a video stream processed by a single algorithm through the HDMI protocol; The second interactive channel is used to output the comparison video streams processed by multiple algorithms; The third interactive channel is used to output the position coordinate information of the target to be measured in the image data.
8. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 7, It is characterized in that The interaction module also includes a sending data formatting module, which is used to format the algorithm processing results according to a preset output method to adapt to the output protocols corresponding to different preset interaction channels, and store them in an output data cache queue. The output general driver module reads the data in the output data cache queue.
9. The deployment platform of the image algorithm hardware acceleration engine based on ZYNQ according to claim 1, It is characterized in that The deployment platform also includes a DDR3 module, and the DDR3 module is used to store the data formatted and pre-processed by the image processing unit.
10. A method for testing an image algorithm hardware acceleration engine based on ZYNQ, applied to a deployment platform of an image algorithm hardware acceleration engine based on ZYNQ as claimed in any one of claims 1 to 9, It is characterized in that The following steps are involved: Obtain image data in multiple input formats provided by different image data sources, and store them in corresponding data cache queues FIFO respectively; Detect whether there is image data in multiple data buffer queues FIFOs, and when it is detected that there is image data in one or more FIFOs, pre-process the image data and transmit the pre-processed data to the input data buffer queue of the frame buffer; When it is detected that there is image data in the input data buffer queue of the frame buffer, the image data is read out, and the image data is formatted and preprocessed according to the input format requirements of the deployed algorithm; Process the received data according to the deployed algorithm, record the algorithm start processing time and algorithm execution end time, calculate the algorithm processing time, and obtain the algorithm processing result; Output the algorithm processing results according to the preset interactive channel.
