Multi-channel video transmission system and method based on FPGA and RK3588

Abstract

The invention belongs to the field of video data processing, and discloses a multi-channel video transmission system and method based on an FPGA and RK3588, the system comprises an FPGA platform and an RK3588 platform, and the FPGA platform is connected with the RK3588 platform. The FPGA platform comprises a multi-mode LVDS decoding module, a data interface conversion module, a read-write cache component, a DDR memory and a PCIE bridging module. The RK3588 platform comprises a PCIE (Peripheral Component Interface Express) controller, a video buffer pool, a video compression coding module and a data sending processing module; and the PCIE controller is connected with the PCIE bridging module. The multi-channel video transmission system and method can be widely applied to display products, the system application flexibility is high, compression and transmission of multi-channel videos can be supported at the same time, more application scenes can be met, and high practical value and economic value are achieved.

Description

Technical Field

[0001] The invention belongs to the field of video data processing, and relates to a multi-channel video transmission system and method based on FPGA and RK3588. Background Art

[0002] With the rapid development of information technology, the demand for multimedia data processing and transmission is increasing, especially in the fields of surveillance, medical imaging, industrial automation, etc., where the requirements for video acquisition, compression and real-time transmission are particularly prominent. Traditional video processing systems often use a single or fixed hardware architecture, which is difficult to meet the diverse, high-performance, and low-latency video processing requirements.

[0003] At present, FPGA has become one of the effective means to solve the problem of video transmission due to its high flexibility and parallel processing capabilities. At the same time, with the popularity of mobile devices and embedded systems, the ARM architecture-based processor Rockchip RK3588 has become popular due to its powerful computing power and good energy efficiency. RK3588 not only supports multiple operating systems, but also has excellent multimedia processing capabilities, such as image processing, video encoding and decoding, and is also an ideal choice for achieving efficient video compression and streaming.

[0004] However, the design of video processing system combining FPGA and RK3588 still faces many challenges, such as how to effectively utilize the characteristics of both to optimize the video data acquisition, compression and transmission process; how to ensure the stability, reliability and application flexibility of the system when processing a large number of video streams; and how to reduce the overall power consumption of the system and improve the cost performance of the system. Summary of the invention

[0005] In order to realize reliable and effective transmission of video data through FPGA and RK3588 technology, the present invention discloses a multi-channel video transmission system based on FPGA and RK3588, wherein the system comprises an FPGA platform and an RK3588 platform, and the FPGA platform is connected to the RK3588 platform.

[0006] Among them, the FPGA platform includes a multi-mode LVDS decoding module, a data interface conversion module, a read-write cache component, a DDR memory and a PCIE bridge module. The multi-mode LVDS decoding module performs format conversion on multiple received video data to obtain multi-channel video data. The data interface conversion module converts the multi-channel video data into multi-channel AXI bus data. The read-write cache component converts the multi-channel AXI bus data into multiple single-channel AXI bus data and caches them in the DDR memory. The PCIE bridge module outputs the single-channel AXI bus data cached in the DDR memory to the RK3588 platform.

[0007] The RK3588 platform includes a PCIE controller, a video buffer pool, a video compression encoding module and a data sending processing module. The PCIE controller is connected to the PCIE bridge module, and reads the offset address of the single-channel AXI bus data cached in the DDR memory according to the interrupt control signal of the PCIE bridge module, obtains the single-channel AXI bus data according to the offset address and outputs it to the video buffer pool; the video compression encoding module extracts the single-channel AXI bus data from the video buffer pool, compresses it to obtain code stream data, and outputs the code stream data through the data sending processing module.

[0008] Furthermore, the multi-mode LVDS decoding module includes multiple LVDS video interfaces, decoding and encoding submodules and synchronization signal alignment submodules, each of the LVDS video interfaces is connected to a camera respectively, the decoding and encoding submodule decodes and formats the video data received by the LVDS video interface according to the format configuration parameters to obtain multiple channels of video data, and the synchronization signal alignment submodule performs serial data alignment on the multiple channels of video data and outputs them to the data interface conversion module.

[0009] Furthermore, the multi-channel video data includes RGB format video data, YUV422 format video data and YUV444 format video data.

[0010] Further, the read-write cache component includes a VDMA read-write control module, an AXI bus routing module and a DDR control module, the VDMA read-write control module outputs the multi-channel AXI bus data to the AXI bus routing module according to the read-write configuration information, the AXI bus routing module splits the multi-channel AXI bus data into a plurality of single-channel AXI bus data, and caches the single-channel AXI bus data into the DDR memory via the DDR control module;

[0011] The AXI bus routing module also reads the cached single-channel AXI bus data from the DDR memory via the DDR control module, and outputs the single-channel AXI bus data to the RK3588 platform via the PCIE bridge module.

[0012] Furthermore, the read-write cache component also includes a soft core control module, which is connected to the VDMA read-write control module and is used to set the read-write configuration information of the VDMA read-write control module, wherein the read-write configuration information includes the resolution, burst length, interval step and read-write control of the read-write video.

[0013] Furthermore, the PCIE bridge module includes a master PCIE bridge module and a slave PCIE bridge module, and each channel in the PCIE bridge module is provided with an interrupt signal control register, and the interrupt signal control register controls the on-off of the channel according to whether the single-channel AXI bus data ends caching.

[0014] Furthermore, the video buffer pool includes a plurality of buffer zones, and each of the buffer zones is provided with a plurality of buffer blocks.

[0015] Furthermore, the video compression encoding module compresses the single-channel AXI bus data extracted from the video buffer pool according to the compression ratio, video timing and bit width to obtain code stream data, and outputs the code stream data according to channel attributes, wherein the video compression encoding module includes a bit rate controller and a video encoder.

[0016] Furthermore, the data sending processing module is connected to the file system, the monitor and the encapsulator respectively.

[0017] The embodiment of the present invention further provides a multi-channel video transmission method based on FPGA and RK3588, the method comprising the following steps:

[0018] S1. On the FPGA platform, according to user needs, multiple video data collected are converted into formats through the configured video format conversion protocol and synchronization signal to obtain multi-channel video data, wherein the multi-channel video data includes any one of RGB format video data, YUV422 format video data and YUV444 format video data, and the synchronization signal includes a video field synchronization signal, a line synchronization signal and an enable synchronization signal;

[0019] S2, converting the multi-channel video data into multi-channel AXI bus data, converting the multi-channel AXI bus data into a plurality of single-channel AXI bus data according to the read-write configuration information, and then caching the data into a DDR memory;

[0020] S3, reading the offset address of the single-channel AXI bus data cached in the DDR memory according to the access request output by the RK3588 platform and the interrupt control signal;

[0021] S4. Read the cached single-channel AXI bus data through the offset address, compress the read single-channel AXI bus data according to the compression ratio, video timing and bit width to obtain code stream data, and output the code stream data according to the channel attribute.

[0022] An embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the processor implements any of the above-mentioned multi-channel video transmission methods based on FPGA and RK3588, so as to realize reliable and effective transmission of video data through FPGA and RK3588 technology.

[0023] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned multi-channel video transmission methods based on FPGA and RK3588, so as to realize reliable and effective transmission of video data through FPGA and RK3588 technology.

[0024] The multi-channel video transmission system based on FPGA and RK3588 provided by the present invention has the following advantages:

[0025] (1) It has a multi-mode LVDS decoding function, which can simultaneously perform multi-channel, such as 6-channel LVDS interface video decoding, and the number of LVDS data differential pair signals is configurable. At the same time, the data encoding method on the LVDS data differential pair can be customized through the module macro definition parameters. The high interface encoding flexibility can meet the application requirements of different scenarios.

[0026] (2) It can meet the compression requirements of different video data formats of the RK3588 platform. The LVDS decoded output video format supports three formats, namely RGB format, YUV422 format and YUV444 format.

[0027] (3) When converting the video data format, the alignment of each differential pair data can be automatically detected based on the encoding position of the video field synchronization signal (VSY), the line synchronization signal (HSY) and the enable signal (DEN), and the alignment can be automatically adjusted based on the offset value to solve the problem of abnormal decoded video caused by misaligned parallel differential data in LVDS decoding.

[0028] (4) By adding a video cache completion interrupt signal control register to each channel in the PCIE bridge module, the interrupt signal can inform the RK3588 platform that the current frame of a certain video has been completed in the DDR cache. The RK3588 platform can read the current frame data of a certain video through the PCIE bridge module. Compared with the multi-channel video round-robin reading data method, it can not only synchronize the video frame rate, but also improve the PCIE data transmission efficiency.

[0029] (5) A video buffer pool processing mode is designed in the RK3588 platform. After completing the data interrupt reception, the data can be read into the pre-allocated video buffer pool. The video buffer pool is used as a data container to provide a stable code stream input environment for the compressor, and at the same time, the basic form of the input pixel data can be pre-analyzed.

[0030] (6) When compressing a video, a video compression encoding module is formed by designing a bit rate controller and a video encoder. The bit rate (i.e., compression ratio) control can input different compression methods (fixed, variable) according to project requirements to control the bit rate of the compressed video. The channel attributes can control the video channel type group, compression method, etc. The encoder is configured to compress the resolution information and pixel information of the video through the video timing and bit width input.

[0031] (7) Three methods are provided to process the compressed bitstream data: using the system I / O interface to write the frame data into the file system of the hard disk; using network protocols such as UDP / RTSP to transmit the frame data to the host computer for decoding and playback; and completing the encapsulation and conversion of MP4, AVI and other formats through the FFMpeg library. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 This is an architecture diagram of a multi-channel video transmission system based on FPGA and RK3588 disclosed in an embodiment of the present invention;

[0034] Figure 2 The architecture diagram of the FPGA platform disclosed in the embodiment of the present invention;

[0035] Figure 3 This is an architecture diagram of the RK3588 platform disclosed in the embodiment of the present invention;

[0036] Figure 4 This is the LVDS encoding format of the RGB video in 4lane mode disclosed in the embodiment of the present invention. DETAILED DESCRIPTION

[0037] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] The following describes the implementation methods of the present application through specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and the features of the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0039] The embodiment of the present invention discloses a multi-channel video transmission system based on FPGA and RK3588, see Figure 1 As shown, the system includes an FPGA platform and an RK3588 platform, and the FPGA platform is connected to the RK3588 platform.

[0040] Among them, see Figure 2 As shown, the FPGA platform includes a multi-mode LVDS decoding module, a data interface conversion module, a read-write cache component, a DDR memory and a PCIE bridge module. The multi-mode LVDS decoding module performs format conversion on multiple received video data to obtain multi-channel video data. The data interface conversion module converts the multi-channel video data into multi-channel AXI bus data. The read-write cache component converts the multi-channel AXI bus data into multiple single-channel AXI bus data and caches them in the DDR memory. The PCIE bridge module outputs the single-channel AXI bus data cached in the DDR memory to the RK3588 platform.

[0041] Furthermore, the multi-mode LVDS decoding module includes multiple LVDS video interfaces, decoding and encoding submodules and synchronization signal alignment submodules, each of the LVDS video interfaces is connected to a camera respectively, the decoding and encoding submodule decodes and formats the video data received by the LVDS video interface according to the format configuration parameters to obtain multiple channels of video data, and the synchronization signal alignment submodule performs serial data alignment on the multiple channels of video data and outputs them to the data interface conversion module.

[0042] Specifically, the module can realize the decoding function of multiple (up to 6) LVDS interface videos, and the decoded output video data format supports three formats: RGB / YUV422 / YUV444. The multi-mode LVDS decoding function, in which the multi-mode is mainly reflected in: ① It can decode multiple LVDS interface videos, and the number of video channels can be configured, supporting up to 6 channels; ② The number of data channels (data differential pairs) of each LVDS channel can be configured, and the valid configuration numbers are 2, 3 and 4; ③ The data encoding format on the LVDS data channel can be customized through the module macro definition parameters; ④ The module decoding supports three output formats, namely RGB, YUV422 and YUV444; ⑤ The module can automatically realize the serial data alignment function according to the bit number of the customized video field synchronization signal, line synchronization signal and enable synchronization signal, to solve the problem of abnormal decoded video caused by the misalignment of parallel differential data in LVDS decoding. The configuration status of the macro definition parameters in the default state of the multi-mode LVDS decoding module is: support 6 channels of simultaneous decoding; the number of data channels for each channel is 4; the encoding format on the data channel is such as Figure 4 As shown; the LVDS decoding output format is RGB; the bit numbers of the field synchronization signal, the line synchronization signal and the enable signal are 15, 16 and 14 respectively. Among them, the macro definition of the multi-mode LVDS decoding module is shown in Table 1 below:

[0043] Table 1: LVDS decoding module macro definition description

[0044]

[0045]

[0046]

[0047]

[0048] Furthermore, the data interface conversion module converts the RGB / YUV422 / YUV444 format data decoded by the multi-mode LVDS decoding module into AXI bus data, and the AXI bus data is used for VDMA read and write control and DDR data read and write control bus. The data interface conversion module can simultaneously perform format conversion of multiple channels (such as 6 channels) of video data.

[0049] Further, the read-write cache component includes a VDMA read-write control module, an AXI bus routing module and a DDR control module, the VDMA read-write control module outputs the multi-channel AXI bus data to the AXI bus routing module according to the read-write configuration information, the AXI bus routing module splits the multi-channel AXI bus data into a plurality of single-channel AXI bus data, and caches the single-channel AXI bus data into the DDR memory via the DDR control module;

[0050] The AXI bus routing module also reads the cached single-channel AXI bus data from the DDR memory via the DDR control module, and outputs the single-channel AXI bus data to the RK3588 platform via the PCIE bridge module.

[0051] Specifically, the VDMA read / write control module can use the VDMA IP core officially provided by Xilinx to perform read / write operations on DDR data through the AXI bus. The AXI bus routing module implements a routing module that converts multiple AXI buses into a single AXI bus, and implements read / write transmission operations of multiple AXI bus data in the DDR memory. The DDR control module can use the DDR control IP core officially provided by Xilinx to perform read / write control on the external DDR memory, and implement cache of multiple video data.

[0052] Furthermore, the read-write cache component also includes a soft core control module, which is connected to the VDMA read-write control module and is used to set the read-write configuration information of the VDMA read-write control module, wherein the read-write configuration information includes the resolution, burst length, interval step and read-write control of the read-write video.

[0053] Specifically, the soft core control module is mainly used to read and write configurations of the read and write registers of the VDMA IP core. The configuration information mainly includes the resolution, burst length, interval step, read and write control, etc. of the read and write video, so as to realize the read and write operations of the video data.

[0054] Furthermore, the PCIE bridge module includes a master PCIE bridge module and a slave PCIE bridge module, and each channel in the PCIE bridge module is provided with an interrupt signal control register, and the interrupt signal control register controls the on-off of the channel according to whether the single-channel AXI bus data ends the cache. Wherein, the master PCIE bridge module is PCIE2.0 X4, and the slave PCIE bridge module is PCIE2.0 X2. In the specific implementation, PCIE2.0 X4 is used preferentially, and if the data bandwidth is insufficient, PCIE2.0 X2 is added for assistance.

[0055] In practical applications, the channel of the main PCIE bridge module is first used for data transmission. When the transmission bandwidth of the main PCIE bridge module is full and tight, the channel of the slave PCIE bridge module is added for data transmission.

[0056] At the same time, the PCIE bridge module sets an interrupt control signal. RK3588, as the master end of PCIE, accesses the interrupt signal control register of the FPGA slave end through PCIE to realize the interrupt transmission mode of video data and improve the efficiency of video transmission. There are 6 interrupt signal control registers: Intr1, Intr2...Intr6, which correspond to the interrupt control of 6 video transmissions respectively. Each interrupt control register has 8 bits, of which bit0 represents the interrupt control enable bit, 1 represents the interrupt mode on, and 0 represents the interrupt mode off; bit1 represents the interrupt indication signal, 1 represents the current frame has been cached and the video data can be read. When the RK3588 detects that the bit is 1, it reads the data in the cache area and sets the bit position to 0 after the reading is completed; bit2~bit3 represents the frame number of the cached video read by RK3588. Each video channel will perform 3 frames of cyclic frame storage in DDR. RK3588 reads the cached data at this position according to the cached video frame number; bit4~bit7 are reserved bits, and the default value is 0.

[0057] See also Figure 3 As shown, the RK3588 platform includes a PCIE controller, a video buffer pool, a video compression encoding module and a data sending processing module. The PCIE controller is connected to the PCIE bridge module, and reads the offset address of the single-channel AXI bus data cached in the DDR memory according to the interrupt control signal of the PCIE bridge module, obtains the single-channel AXI bus data according to the offset address and outputs it to the video buffer pool; the video compression encoding module extracts the single-channel AXI bus data from the video buffer pool, compresses it to obtain code stream data, and outputs the code stream data through the data sending processing module.

[0058] Specifically, as the master end of the PCIE platform, the PCIE controller mainly configures the BAR space driven by the PCIE, determines the offset address of the video channel (up to 6 video channels, so up to 6 channels) that needs to be read according to the interrupt control signal, and then obtains the RGB / YUV422 / YUV444 pixel data format from the memory of the FPGA module according to the offset address.

[0059] Furthermore, the video buffer pool includes a plurality of buffers, and each of the buffers is provided with a plurality of buffer blocks. Specifically, after the data interrupt reception is completed through the PCIE controller, the data is read into the pre-allocated video buffer pool. The video buffer pool, as a data container, can provide a stable code stream input environment for the compressor, and can pre-analyze the basic form of the input pixel data. Preferably, a buffer block can be provided for the buffer of each video channel, and the calculation method is: basic value × input frame rate / output frame rate, where the basic value is the resolution level of the video, the maximum is 4320P (the basic value is 8), and the minimum is 480P (the basic value is 1).

[0060] Furthermore, the video compression encoding module compresses the single-channel AXI bus data extracted from the video buffer pool according to the compression ratio, video timing and bit width to obtain code stream data, and outputs the code stream data according to channel attributes, wherein the video compression encoding module includes a bit rate controller and a video encoder.

[0061] Specifically, data is sent from the video buffer pool to the encoding channel of the video compression encoding module for video compression. The encoding channel is completed by the bit rate controller and the video encoder. The bit rate control can input different compression methods (fixed, variable) to control the bit rate of the compressed video according to application requirements. The channel attributes can control the video channel type group, compression method, etc. The encoder is configured through the video timing and bit width input to compress the resolution information and pixel information of the video.

[0062] Furthermore, the data transmission processing module is connected to the file system, the monitor and the encapsulator respectively. Specifically, in the data transmission link, when a complete frame of video is compressed, the data transmission processing module can output the code stream data in three ways, namely, the first is to use the system I / O interface to write the frame data into the file system of the hard disk; the second is to use the network protocol such as UDP / RTSP to transmit the frame data to the host computer (and the monitor) for real-time decoding and playback; the third is to complete the encapsulation conversion of MP4, AVI and other formats into the encapsulator through the FFMpeg library.

[0063] Based on the same inventive concept, an embodiment of the present invention further provides a multi-channel video transmission method based on FPGA and RK3588, which can be implemented by the multi-channel video transmission system based on FPGA and RK3588 in the above embodiment. The specific process of the multi-channel video transmission method is described below, and the method includes the following steps:

[0064] S1. On the FPGA platform, according to user needs, multiple video data collected are converted into formats through the configured video format conversion protocol and synchronization signal to obtain multi-channel video data, wherein the multi-channel video data includes any one of RGB format video data, YUV422 format video data and YUV444 format video data, and the synchronization signal includes a video field synchronization signal, a line synchronization signal and an enable synchronization signal;

[0065] S2, converting the multi-channel video data into multi-channel AXI bus data, converting the multi-channel AXI bus data into a plurality of single-channel AXI bus data according to the read-write configuration information, and then caching the data into a DDR memory;

[0066] S3, reading the offset address of the single-channel AXI bus data cached in the DDR memory according to the access request output by the RK3588 platform and the interrupt control signal;

[0067] S4. Read the cached single-channel AXI bus data through the offset address, compress the read single-channel AXI bus data according to the compression ratio, video timing and bit width to obtain code stream data, and output the code stream data according to the channel attribute.

[0068] The multi-channel video transmission system based on FPGA and RK3588 provided by the present invention has the following advantages:

[0069] (1) It has a multi-mode LVDS decoding function, which can simultaneously perform multi-channel, such as 6-channel LVDS interface video decoding, and the number of LVDS data differential pair signals is configurable. At the same time, the data encoding method on the LVDS data differential pair can be customized through the module macro definition parameters. The high interface encoding flexibility can meet the application requirements of different scenarios.

[0070] (2) It can meet the compression requirements of different video data formats of the RK3588 platform. The LVDS decoded output video format supports three formats, namely RGB format, YUV422 format and YUV444 format.

[0071] (3) When converting the video data format, the alignment of each differential pair data can be automatically detected based on the encoding position of the video field synchronization signal (VSY), the line synchronization signal (HSY) and the enable signal (DEN), and the alignment can be automatically adjusted based on the offset value to solve the problem of abnormal decoded video caused by misaligned parallel differential data in LVDS decoding.

[0072] (4) By adding a video cache completion interrupt signal control register to each channel in the PCIE bridge module, the interrupt signal can inform the RK3588 platform that the current frame of a certain video has been completed in the DDR cache. The RK3588 platform can read the current frame data of a certain video through the PCIE bridge module. Compared with the multi-channel video round-robin reading data method, it can not only synchronize the video frame rate, but also improve the PCIE data transmission efficiency.

[0073] (5) A video buffer pool processing mode is designed in the RK3588 platform. After completing the data interrupt reception, the data can be read into the pre-allocated video buffer pool. The video buffer pool is used as a data container to provide a stable code stream input environment for the compressor, and at the same time, the basic form of the input pixel data can be pre-analyzed.

[0074] (6) When compressing a video, a video compression encoding module is formed by designing a bit rate controller and a video encoder. The bit rate (i.e., compression ratio) control can input different compression methods (fixed, variable) according to project requirements to control the bit rate of the compressed video. The channel attributes can control the video channel type group, compression method, etc. The encoder is configured to compress the resolution information and pixel information of the video through the video timing and bit width input.

[0075] (7) Three methods are provided to process the compressed bitstream data: using the system I / O interface to write the frame data into the file system of the hard disk; using network protocols such as UDP / RTSP to transmit the frame data to the host computer for decoding and playback; and completing the encapsulation and conversion of MP4, AVI and other formats through the FFMpeg library.

[0076] In this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any of the above-mentioned multi-channel video transmission methods based on FPGA and RK3588 is implemented.

[0077] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0078] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned multi-channel video transmission methods based on FPGA and RK3588.

[0079] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0080] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-channel video transmission system based on FPGA and RK3588, characterized in that: It includes an FPGA platform and an RK3588 platform, wherein the FPGA platform is connected to the RK3588 platform; The FPGA platform includes a multi-mode LVDS decoding module, a data interface conversion module, a read-write cache component, a DDR memory and a PCIE bridge module. The multi-mode LVDS decoding module performs format conversion on multiple video data received to obtain multi-channel video data. The data interface conversion module converts the multi-channel video data into multi-channel AXI bus data. The read-write cache component converts the multi-channel AXI bus data into multiple single-channel AXI bus data and caches them in the DDR memory. The PCIE bridge module outputs the single-channel AXI bus data cached in the DDR memory to the RK3588 platform. The RK3588 platform includes a PCIE controller, a video buffer pool, a video compression encoding module and a data sending processing module. The PCIE controller is connected to the PCIE bridge module, and reads the offset address of the single-channel AXI bus data cached in the DDR memory according to the interrupt control signal of the PCIE bridge module, and obtains the single-channel AXI bus data according to the offset address and outputs it to the video buffer pool; The video compression encoding module extracts the single-channel AXI bus data from the video buffer pool and compresses it to obtain code stream data, and outputs the code stream data through the data sending processing module.

2. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The multi-mode LVDS decoding module includes multiple LVDS video interfaces, decoding and encoding submodules and synchronization signal alignment submodules. Each of the LVDS video interfaces is connected to a camera respectively. The decoding and encoding submodule decodes and converts the format of the video data received by the LVDS video interface according to the format configuration parameters to obtain multiple channels of video data. The synchronization signal alignment submodule performs serial data alignment on the multiple channels of video data and outputs them to the data interface conversion module.

3. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The multi-channel video data includes RGB format video data, YUV422 format video data and YUV444 format video data.

4. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The read-write cache component includes a VDMA read-write control module, an AXI bus routing module and a DDR control module. The VDMA read-write control module outputs the multi-channel AXI bus data to the AXI bus routing module according to the read-write configuration information. The AXI bus routing module splits the multi-channel AXI bus data into a plurality of single-channel AXI bus data, and caches the single-channel AXI bus data into the DDR memory via the DDR control module. The AXI bus routing module also reads the cached single-channel AXI bus data from the DDR memory via the DDR control module, and outputs the single-channel AXI bus data to the RK3588 platform via the PCIE bridge module.

5. The multi-channel video transmission system based on FPGA and RK3588 according to claim 4, characterized in that: The read-write cache component also includes a soft core control module, which is connected to the VDMA read-write control module and is used to set the read-write configuration information of the VDMA read-write control module, wherein the read-write configuration information includes the resolution, burst length, interval step and read-write control of the read-write video.

6. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The PCIE bridge module includes a master PCIE bridge module and a slave PCIE bridge module. Each channel in the PCIE bridge module is provided with an interrupt signal control register. The interrupt signal control register controls the on-off of the channel according to whether the single-channel AXI bus data ends the cache.

7. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The video buffer pool includes a plurality of buffer zones, and each of the buffer zones is provided with a plurality of buffer blocks.

8. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The video compression encoding module compresses the single-channel AXI bus data extracted from the video buffer pool according to the compression ratio, video timing and bit width to obtain code stream data, and outputs the code stream data according to the channel attributes, wherein the video compression encoding module includes a bit rate controller and a video encoder.

9. The multi-channel video transmission system based on FPGA and RK3588 according to claim 1, characterized in that: The data sending processing module is connected with the file system, the monitor and the encapsulator respectively.

10. A multi-channel video transmission method based on FPGA and RK3588, characterized in that: The method comprises: On the FPGA platform, according to user needs, multiple video data collected are converted into formats through the configured video format conversion protocol and synchronization signal to obtain multi-channel video data, wherein the multi-channel video data includes any one of RGB format video data, YUV422 format video data and YUV444 format video data, and the synchronization signal includes a video field synchronization signal, a line synchronization signal and an enable synchronization signal; Convert the multi-channel video data into multi-channel AXI bus data, convert the multi-channel AXI bus data into a plurality of single-channel AXI bus data according to the read-write configuration information, and then cache the data into a DDR memory; Through the access request output by the RK3588 platform, the offset address of the single-channel AXI bus data cached in the DDR memory is read according to the interrupt control signal; The cached single-channel AXI bus data is read through the offset address, the read single-channel AXI bus data is compressed according to the compression ratio, video timing and bit width to obtain code stream data, and the code stream data is output according to the channel attribute.

Images