Multi-channel video processing system and method based on ZYNQ

Abstract

The invention discloses a multi-channel video processing system and method based on ZYNQ, and the system is characterized in that an image collection module is connected with a camera, a video synchronization module achieves the synchronization of multi-channel videos, and transmits an image to a format conversion module for format conversion after the synchronization is completed; the image data after format conversion is transmitted to the VDMA module and is transmitted to the PS unit, the PS unit stores multiple frames of image videos, the image fusion module reads the images in the cache and realizes image fusion, the spliced data integration module performs data integration, then the spliced video data is subjected to size adjustment and format conversion, and finally the spliced video data is transmitted to the HDMI display module. And displaying the video. According to the scheme, image processing and caching are carried out in the PS unit and the PL unit respectively, the cameras are initialized at the PL end, line-field synchronization, data effective signals and pixel data are subjected to framing and then are synchronized through FIFO buffering, and the problem that video data collected by multiple cameras together are not synchronized can be effectively solved.

Description

Technical Field

[0001] The invention belongs to the field of multi-channel video processing, and in particular relates to a multi-channel video processing system and method based on ZYNQ. Background Art

[0002] In response to the increasingly urgent need for satellite near-field target perception and detection, traditional single cameras cannot meet the needs of all-round monitoring of wide areas for near-field targets within a few kilometers to 40 kilometers around the satellite. In the field of space surveillance, space targets include reconnaissance satellites and large space debris. By integrating the video streams of multiple cameras, the panoramic stitching system can achieve coverage and capture of a wider area and provide a panoramic surveillance perspective.

[0003] Traditional DSPs have relatively weak parallel processing capabilities when processing multiple video streams, and it is difficult to efficiently process large amounts of data at the same time; the response time and latency of the CPU may not meet the requirements when processing real-time video; GPU programming usually involves a more complex programming model, which increases the difficulty of development and debugging. FPGAs perform real-time pipeline operations on images in units of behavior and use a parallel architecture to meet real-time requirements. The ZYNQ series SoC launched by Xilinx innovatively encapsulates the CPU and FPGA on the same chip, greatly reducing the power consumption and difficulty of communication between the CPU and FPGA. This article will combine the panoramic stitching system with a high-performance hardware platform, which guarantees real-time and operability, and can quickly and real-time process and stitch multiple video streams to meet the needs of real-time monitoring. Summary of the invention

[0004] In view of the above problems, the object of the present invention is to provide a multi-channel video processing system and method based on ZYNQ.

[0005] The specific technical solution for achieving the purpose of the present invention is:

[0006] A multi-channel video processing system based on ZYNQ comprises a PL unit and a PS unit, wherein the PL unit is used to realize multi-channel video acquisition and video splicing, format conversion, and output, and the PS unit is used to realize cache.

[0007] Further, the PL unit includes an image acquisition module, a video synchronization module, a format conversion module, a VDMA module, an image fusion module, a splicing data integration module, an image reduction module, a conversion output module and an HDMI display module;

[0008] The image acquisition module is connected to the camera and is used to acquire multiple video images and transmit the acquired images to the video synchronization module;

[0009] The video synchronization module is used to achieve synchronization between multiple channels of video, and after synchronization is completed, the image is transmitted to the format conversion module for format conversion;

[0010] The format-converted image data is transmitted to the VDMA module, and then transmitted to the PS unit via the VDMA module. The PS unit stores the multi-frame image video to achieve multi-frame image caching;

[0011] The image fusion module reads the image in the cache based on the VDMA module and realizes image fusion. The splicing data integration module performs data integration based on the fused image, and then adjusts the size and converts the format of the spliced ​​video data through the image reduction module and the conversion output module, and finally transmits it to the HDMI display module for video display.

[0012] Furthermore, the process of the video synchronization module performing multi-channel video synchronization is as follows:

[0013] First, multiple cameras are initialized based on the PL unit and configured under the same clock control;

[0014] Use the same number of synchronization FIFOs as the number of cameras to synchronize the multi-channel data collected by multiple cameras, that is:

[0015] Set the delay count time. After the camera is initialized, after the delay count time, the register configuration takes effect, and the field synchronization signal of the camera is assigned to the write enable signal of the FIFO to control the writing of the FIFO. When the FIFO write enable is valid, the camera output data is written into its corresponding synchronization FIFO, where each frame of data includes the line synchronization signal cmos_href, the field synchronization signal cmos_vsync, the data valid signal data_ce and the data data;

[0016] In addition, a configurable empty threshold of the synchronous FIFO is set. When the data in multiple synchronous FIFOs reaches the configurable empty threshold, the data in each synchronous FIFO is uniformly transmitted to the format conversion module.

[0017] Furthermore, the process of the image fusion module reading the image in the cache based on the VDMA module and realizing image fusion is as follows:

[0018] First, confirm the overlapping area between cameras based on camera calibration;

[0019] Read the overlapping area data from the buffer area of ​​the PS unit, and fuse the overlapping area data of the images collected by each camera in the image fusion module;

[0020] The fused data is transmitted back to the cache area of ​​the PS unit and replaces the original overlapping area data.

[0021] Furthermore, the address of the overlapping area data in the cache area is:

[0022] When allocating the VDMA frame buffer address, the address of each frame buffer overlap area is written into the lookup table. Assuming the VDMA frame buffer start address is Xaddr and the number of rows is i, the address of the overlap area in the frame buffer is:

[0023] (X addr +W*i) <d left <(X addr +d+W*i)

[0024] (X addr +W-d+W*i) <d right <(X addr +W+W*i)

[0025] Among them, d left Indicates the address of the left overlapping area in the frame buffer, d right Indicates the address of the right overlapping area in the frame buffer, and W indicates the image width.

[0026] Furthermore, the process of data integration performed by the splicing data integration module based on the fused image is as follows:

[0027] The data integration module uses RAM to read and integrate the data in the cache area of ​​the PS unit based on the state machine, that is, reads the corresponding address data in each image in turn, and splices the read data to complete the splicing of multiple video images.

[0028] The present invention also provides a multi-channel video processing method based on the above system, comprising the following steps:

[0029] Step 1: Perform camera calibration and determine the relative positions of each camera according to the parameters;

[0030] Step 2: Connect the camera to the PL unit, initialize the camera and configure the registers in the PL unit, and convert the video source signal into RGB888 format;

[0031] Step 3, sending the video data in RGB888 format to the video synchronization module to synchronize multiple channels of video data;

[0032] Step 4: Convert the synchronized image data into AXI4-Stream format suitable for internal transmission in FPGA, and convert the interface signals from HSYNC, HBLANK, VSYNC, VBLANK, and DATA into TVAID, TREADY, TDATA, TLAST, and TUSER;

[0033] Step 5: The VDMA module sends the AXI4-Stream format data stream to the DDR3 of the PS unit through the AXI-HP interface for buffering;

[0034] Step 6: read the image data in the frame buffer to the image fusion module through the VDMA module, and use the fade-in and fade-out algorithm to fuse the overlapping areas;

[0035] Step 7, write the fused image back to the original address, integrate it through the splicing data integration module and then read it out to the image reduction module, and use the bilinear interpolation algorithm to reduce the panorama to a size suitable for the HDMI display;

[0036] Step 8. Convert the image format to RGB888, output it to the HDMI display module, use the Video TimingController IP core to provide the video display timing required by the display, and send it to the HDMI display for display.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The solution of the present invention is based on ZYNQ, performs image processing and caching in the PS unit and the PL unit respectively, and transmits and processes image data through the AXI4-Stream protocol, thereby improving data transmission efficiency;

[0039] Among them, the camera is initialized at the PL end, and the line field synchronization, data valid signal and pixel data are framed and synchronized through the FIFO buffer, which can effectively solve the problem of asynchronous video data collected by multiple cameras. In addition, an overlapping area fusion module between multiple camera data is set. After the overlapping part is processed by the image stitching and fusion algorithm, the pixel data of different address segments in DDR3 are read into RAM through the state machine for stitching and integration, which can prevent image tearing.

[0040] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the architecture of the multi-channel video processing system based on ZYNQ of the present invention.

[0042] Figure 2 The figure is a schematic diagram of the data frame format written into FIFO in the video synchronization module in an embodiment of the present invention.

[0043] Figure 3 Schematic diagram of address allocation for a frame of image stored in DDR3 in an embodiment of the present invention.

[0044] Figure 4It is a schematic diagram of the composition of a panoramic image of image fusion in an embodiment of the present invention.

[0045] Figure 5 It is a schematic diagram of splicing data integration based on a state machine in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] Example

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0049] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0050] Combination Figure 1 A multi-channel video processing system based on ZYNQ includes a PL unit and a PS unit, wherein the PL unit corresponds to an FPGA resource for realizing multi-channel video acquisition and video splicing, format conversion, and output, and the PS unit corresponds to a CPU unit for realizing cache.

[0051] The PL unit includes an image acquisition module, a video synchronization module, a format conversion module, a VDMA module, an image fusion module, a splicing data integration module, an image reduction module, a conversion output module and an HDMI display module;

[0052] The image acquisition module is connected to the camera and is used to acquire multiple video images and transmit the acquired images to the video synchronization module;

[0053] The video synchronization module is used to achieve synchronization between multiple channels of video. After synchronization is completed, the image is transmitted to the format conversion module for format conversion. Specifically, in this embodiment, the RGB888 format is converted into the AXI4-Stream format suitable for internal transmission in ZYNQ.

[0054] The format-converted image data is transmitted to the VDMA module, and then to the PS unit via the VDMA module. The DDR3 in the PS unit stores multi-frame image videos to achieve multi-frame image caching.

[0055] The image fusion module reads the image in the cache based on the VDMA module and realizes image fusion. The splicing data integration module performs data integration based on the fused image, and then adjusts the size and converts the format of the spliced ​​video data through the image reduction module and the conversion output module, and finally transmits it to the HDMI display module for video display.

[0056] The process of the video synchronization module performing multi-channel video synchronization is as follows:

[0057] First, multiple cameras are initialized based on the PL unit and configured under the same clock control;

[0058] In this embodiment, the camera acquisition resolution and the display resolution are configured to 1280×720, and the number of cameras is 4. Since the startup time of the hardware of the four cameras is different, the output image data cannot be aligned. In order to synchronize the four cameras, first, the initialization of the four cameras is synchronized as much as possible. The signal generated by the PS end can reduce the occupation of PL end resources. The configuration is simple. The I2C peripheral of the PS end needs to be configured as an EMIO interface and connected to the SCCB interface of the camera. However, the instruction execution of the PS is sequential. When configuring two or more cameras, there must be a configuration order, which leads to a phase difference in the output images of the four cameras. The initialization of the camera on the PL end can be configured under the control of the same clock, so that the time to complete the camera configuration is roughly the same. Although the cameras after synchronous configuration are basically synchronized in the frame, the pixels in the image still have different degrees of asynchrony. Therefore, a synchronous FIFO with the same number of cameras is used to synchronize the multi-channel data collected based on multiple cameras. The synchronization process is as follows:

[0059] The effective time of the configuration register in the camera acquisition module is about 10 frames of data, so a delay counter is defined in the module to set the delay count time. After the camera is initialized, after the delay count time, the camera output data begins to be written into its corresponding synchronization FIFO, where each frame of data includes the line synchronization signal cmos_href, the field synchronization signal cmos_vsync, the data valid signal data_ce and the data data; after the synchronization register configuration module, the output pixel phase difference of the camera is not large, and only a few lines need to be cached to complete the synchronization, so the FIFO depth is set to 5120, that is, 4 lines are cached;

[0060] After the camera collects data, the data format output by the camera is RGB565, and the data output pin of the camera module is 8 bits. Therefore, it takes two clock cycles to output 16-bit RGB565 format data. The first clock outputs the high 8 bits, and the second clock outputs the low 8 bits. The format of the subsequent image processing algorithm is RGB888, so it is necessary to fill the low bits of the three color channels in the RGB565 format with 0 to form the RGB888 format. If the camera clock signal is output, the data is only updated once when the clock has output two cycle clocks, that is, the same data is output twice. Therefore, a data_ce signal is added, which is valid after the low 8 bits of RGB565 data are updated, and invalid when the high 8 bits. Therefore, the signals output by the camera acquisition module include the line synchronization signal cmos_href, the field synchronization signal cmos_vsync, the data valid signal data_ce, and the 24-bit data cmos_data.

[0061] Set the input bit width of the synchronous FIFO to 27 bits. When writing data, frame the horizontal synchronization signal cmos_href, the field synchronization signal cmos_vsync, and the data valid signal data_ce together and write them into the FIFO. The writing format is as follows: Figure 2 As shown;

[0062] In order to prevent the last FIFO from being read empty and causing the FIFO to stop reading, it is necessary to set a configurable empty threshold of the synchronous FIFO. In this embodiment, the configurable empty threshold is set to 100. When the data in multiple synchronous FIFOs reaches the configurable empty threshold, the configurable empty signal prog_empty is valid when the remaining data amount in the FIFO is 100. When the prog_empty signals of the four FIFOs are all pulled low, they start to read from the FIFO at the same time, and uniformly transmit the data of each synchronous FIFO to the format conversion module.

[0063] The synchronized pixel data is converted from RGB888 format to AXI4-Stream format suitable for internal circulation in FPGA through Video in to AXI4-Stream module, and then stored in DDR3 of PS unit through VDMA triple frame buffer.

[0064] After the data stream is stored in DDR3, the image fusion of the overlapping parts needs to be performed, and the image fusion module is implemented in the PL unit.

[0065] The process of the image fusion module reading the image in the cache based on the VDMA module and realizing image fusion is as follows:

[0066] First, confirm the overlapping area between cameras based on camera calibration;

[0067] Read the overlapping area data from the buffer area of ​​the PS unit, and fuse the overlapping area data of the images collected by each camera in the image fusion module;

[0068] The fused data is transmitted back to the cache area of ​​the PS unit and replaces the original overlapping area data.

[0069] Furthermore, the address of the overlapping area data in the cache area is:

[0070] When allocating the VDMA frame buffer address, the address of each frame buffer overlap area is written into the lookup table. Assuming the VDMA frame buffer start address is Xaddr and the number of rows is i, the address of the overlap area in the frame buffer is:

[0071] (X addr +1280*i) <d left <(X addr +d+1280*i)

[0072] (X addr +1280-d+1280*i) <d right <(X addr +1280+1280*i)

[0073] Among them, d left Indicates the address of the left overlapping area in the frame buffer, d right Indicates the address of the right overlapping area in the frame buffer. In this embodiment, four video images are spliced ​​together. Therefore, for the first video image, there is only a right overlapping area. For the second and third video images, there are left and right overlapping areas. For the fourth video image, there is only a left overlapping area.

[0074] In the AXI4-Stream protocol, tuser indicates the start of a frame, tlast indicates the end of a line, and whenever tlast is pulled high, i+1. The specific address allocation is as follows: Figure 3 shown.

[0075] When stitching, only the overlapping area data is read from DDR3 and sent to the FIFO in the image fusion module (Stitch module) for processing, and the processed data is written back to the corresponding position; after the processing is completed, the data is written back to the dright of frame 1, the dright of frame 2, and the dright of frame 3. Here, only the right overlapping area of ​​each image is changed to the fused overlapping area image, instead of replacing both the left overlapping area and the right overlapping area, which can save more computing resources.

[0076] The storage location of the fused four frames of data in DDR3 is still segmented, so it needs to be read out for integration;

[0077] The process of data integration performed by the splicing data integration module based on the fused image is as follows:

[0078] The data integration module uses RAM to read and integrate the data in the cache area of ​​the PS unit based on the state machine, that is, reads the corresponding address data in each image in turn, and splices the read data to complete the splicing of multiple video images, such as Figure 4 shown.

[0079] Combination Figure 5 , describing the process of completing image stitching based on the state machine in this embodiment:

[0080] Since the image needs to be reduced (resize_bilinear) after integration, and the image reduction module uses a bilinear interpolation algorithm, it requires two adjacent points in the horizontal direction and two adjacent points in the vertical direction, so a 3×3 sliding window is required for calculation. RAM has more address mapping than FIFO, so it is more reasonable to use RAM for integration here, that is, use the state machine to read the data in DDR3 into RAM for data integration. After the stitching is completed, one line of the target image has 3840-3d pixels. The depth of RAM must be able to store at least three lines, that is, 3*(3840-3d). Here, the RAM depth is set to 11520. Use a state machine to control the start and end of a line of data. The state machine design is as follows:

[0081] When there is no data coming or the RAM write enable is low, it remains in the idle state IDLE. When the RAM write enable ram_wr_en is pulled high, it enters the READ1 state and pulls the tready signal high at the same time, indicating that it is ready to receive the data read from DDR3.

[0082] When the RAM write enable is pulled high, it enters the READ1 state and reads the data in frame 1 into the RAM. In this state, DDR3 starts reading from the first pixel address Xaddr1 of frame 1. The upper limit of the column counter h_cnt_1 is 1280. The data in DDR3 frame 1 is written to the RAM in sequence, and the RAM address increases by 1 each time a data is written. When the 1280th data is read and the RAM write enable ram_wr_en is valid, h_cnt_1 is cleared to enter the READ2 state and read frame 2. If the RAM write enable ram_wr_en is invalid, tready is pulled low and fed back to DDR3 to stop reading data from DDR3.

[0083] In the READ2 state, DDR3 starts reading from Xaddr2+d+1280*i of frame 2. The upper limit of column counter h_cnt_2 is 1280-d. The data in DDR3 frame 2 is written into RAM in sequence. The RAM address is incremented by 1 for each data written. When the 1280-dth data is read and the RAM write enable is valid, h_cnt_2 is cleared and the state is entered into READ3 to read frame 3. If the RAM write enable ram_wr_en is invalid, tready is pulled low and fed back to DDR3 to stop reading data from DDR3.

[0084] In the READ3 state, DDR3 starts reading from Xaddr3+d+1280*i of frame 3. The upper limit of column counter h_cnt_3 is 1280-d. The data in DDR3 frame 3 is written into RAM in sequence. When the 1280-dth data is read and the RAM write enable is valid, h_cnt_2 is cleared to enter the READ4 state to read frame 4. If the RAM write enable ram_wr_en is invalid, tready is pulled low and fed back to DDR3 to stop reading from DDR3.

[0085] In the READ4 state, DDR3 starts reading from Xaddr4+d+1280*i of frame 4, and the upper limit of column counter h_cnt_4 is 1280-d, and the data in DDR3 frame 4 is written to RAM in sequence. If the write enable ram_wr_en of FIFO is invalid, tready is pulled low and fed back to DDR3 to stop reading from DDR3; when the 1280-dth data is read, the column counter h_cnt_4 is cleared, tlast is pulled high for one cycle, indicating the end of a row, and the row counter v_cnt is increased by 1. When the row counter v_cnt counts to 720, frame_vsycn is pulled high, indicating the end of a frame.

[0086] Finally, the data is converted from AXI4-Stream format to RGB888 format that complies with the display, and output to the HDMI display for display.

[0087] The present invention also provides a multi-channel video processing method based on the above system, comprising the following steps:

[0088] Step 1: Perform camera calibration and determine the relative positions of each camera according to the parameters;

[0089] Step 2: Connect the camera to the PL unit, initialize the camera and configure the registers in the PL unit, and convert the video source signal into RGB888 format;

[0090] Step 3, sending the video data in RGB888 format to the video synchronization module to synchronize multiple channels of video data;

[0091] Step 4: Convert the synchronized image data into AXI4-Stream format suitable for internal transmission in FPGA, and convert the interface signals from HSYNC, HBLANK, VSYNC, VBLANK, and DATA into TVAID, TREADY, TDATA, TLAST, and TUSER;

[0092] Step 5: The VDMA module sends the AXI4-Stream format data stream to the DDR3 of the PS unit through the AXI-HP interface for buffering;

[0093] Step 6: read the image data in the frame buffer to the image fusion module through the VDMA module, and use the fade-in and fade-out algorithm to fuse the overlapping areas;

[0094] Step 7, write the fused image back to the original address, integrate it through the splicing data integration module and then read it out to the image reduction module, and use the bilinear interpolation algorithm to reduce the panorama to a size suitable for the HDMI display;

[0095] Step 8. Convert the image format to RGB888, output it to the HDMI display module, use the Video TimingController IP core to provide the video display timing required by the display, and send it to the HDMI display for display.

[0096] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A multi-channel video processing system based on ZYNQ, characterized in that: It includes a PL unit and a PS unit, wherein the PL unit is used to realize multi-channel video acquisition and video splicing, format conversion, and output, and the PS unit is used to realize cache.

2. The ZYNQ-based multi-channel video processing system according to claim 1, characterized in that: The PL unit includes an image acquisition module, a video synchronization module, a format conversion module, a VDMA module, an image fusion module, a splicing data integration module, an image reduction module, a conversion output module and an HDMI display module; The image acquisition module is connected to the camera and is used to acquire multiple video images and transmit the acquired images to the video synchronization module; The video synchronization module is used to achieve synchronization between multiple channels of video, and after synchronization is completed, the image is transmitted to the format conversion module for format conversion; The format-converted image data is transmitted to the VDMA module, and then transmitted to the PS unit via the VDMA module. The PS unit stores the multi-frame image video to achieve multi-frame image caching; The image fusion module reads the image in the cache based on the VDMA module and realizes image fusion. The splicing data integration module performs data integration based on the fused image, and then adjusts the size and converts the format of the spliced ​​video data through the image reduction module and the conversion output module, and finally transmits it to the HDMI display module for video display.

3. The ZYNQ-based multi-channel video processing system according to claim 2, characterized in that: The process of the video synchronization module performing multi-channel video synchronization is as follows: First, multiple cameras are initialized based on the PL unit and configured under the same clock control; Use the same number of synchronization FIFOs as the number of cameras to synchronize the multi-channel data collected by multiple cameras, that is: Set the delay count time. After the camera is initialized, after the delay count time, the register configuration takes effect, and the field synchronization signal of the camera is assigned to the write enable signal of the FIFO to control the writing of the FIFO. When the FIFO write enable is valid, the camera output data is written into its corresponding synchronization FIFO, where each frame of data includes the line synchronization signal cmos_href, the field synchronization signal cmos_vsync, the data valid signal data_ce and the data data; In addition, a configurable empty threshold of the synchronous FIFO is set. When the data in multiple synchronous FIFOs reaches the configurable empty threshold, the data in each synchronous FIFO is uniformly transmitted to the format conversion module.

4. The ZYNQ-based multi-channel video processing system according to claim 2, characterized in that: The process of the image fusion module reading the image in the cache based on the VDMA module and realizing image fusion is as follows: First, confirm the overlapping area between cameras based on camera calibration; Read the overlapping area data from the buffer area of ​​the PS unit, and fuse the overlapping area data of the images collected by each camera in the image fusion module; The fused data is transmitted back to the cache area of ​​the PS unit and replaces the original overlapping area data.

5. The ZYNQ-based multi-channel video processing system according to claim 4, characterized in that: The address of the overlapping area data in the cache area is: When allocating the VDMA frame buffer address, the address of each frame buffer overlap area is written into the lookup table. Assuming the VDMA frame buffer start address is Xaddr and the number of rows is i, the address of the overlap area in the frame buffer is: (X addr +W*i)<d left <(X addr +d+W*i) (X addr +W-d+W*i)<d right <(X addr +W+W*i) Among them, d left Indicates the address of the left overlapping area in the frame buffer, d right Indicates the address of the right overlapping area in the frame buffer, and W indicates the image width.

6. The ZYNQ-based multi-channel video processing system according to claim 2, characterized in that: The process of data integration performed by the splicing data integration module based on the fused image is as follows: The data integration module uses RAM to read and integrate the data in the cache area of ​​the PS unit based on the state machine, that is, reads the corresponding address data in each image in turn, and splices the read data to complete the splicing of multiple video images.

7. A multi-channel video processing method based on any one of claims 1 to 6, characterized in that: The following steps are involved: Step 1: Perform camera calibration and determine the relative positions of each camera according to the parameters; Step 2: Connect the camera to the PL unit, initialize the camera and configure the registers in the PL unit, and convert the video source signal into RGB888 format; Step 3, sending the video data in RGB888 format to the video synchronization module to synchronize multiple channels of video data; Step 4: Convert the synchronized image data into AXI4-Stream format suitable for internal transmission in FPGA, and convert the interface signals from HSYNC, HBLANK, VSYNC, VBLANK, and DATA into TVAID, TREADY, TDATA, TLAST, and TUSER; Step 5: The VDMA module sends the AXI4-Stream format data stream to the DDR3 of the PS unit through the AXI-HP interface for buffering; Step 6: read the image data in the frame buffer to the image fusion module through the VDMA module, and use the fade-in and fade-out algorithm to fuse the overlapping areas; Step 7, write the fused image back to the original address, integrate it through the splicing data integration module and then read it out to the image reduction module, and use the bilinear interpolation algorithm to reduce the panorama to a size suitable for the HDMI display; Step 8. Convert the image format to RGB888, output it to the HDMI display module, use the Video Timing Controller IP core to provide the video display timing required by the display, and send it to the HDMI display for display.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to claim 7 are implemented.

9. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to claim 7 are implemented.

Images