Master-slave multi-core RK3588 architecture and method for realizing multichannel 4K video processing

Abstract

The invention relates to a master-slave multi-core RK3588 architecture and method for realizing multi-channel 4K video processing, and the method comprises the steps: powering on a multi-core RK3588, loading firmware, and completing the initialization of an interface; the main RK3588 in the multi-core RK3588 completes network port routing and control instruction forwarding, and receives a video according to a control instruction; the multi-core RK3588 runs video processing application software to process the received video; the multi-core RK3588 waits for a new control instruction sent by the outside; and if a new control instruction is received, analyzing the message and executing a corresponding action, otherwise, continuing to process the received video. A switch, an FPGA and other circuits do not need to be added, the design process can be simplified, and practicability is improved; the slave equipment RK3588 is physically isolated to the outside, so that the data security is improved; and each channel RK3588 can flexibly configure a video processing mode according to an instruction, so that the software application flexibility is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of video signal processing, especially a master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing. Background Art

[0002] Images and videos are important carriers of visual information, and their resolutions are advancing from standard definition (720P), full definition (1K), high definition (2K) to ultra-high definition (4K), and the industry scales related to display and control systems and other industries are constantly expanding. A display and control system refers to a display screen combined with audio and video encoding and decoding technologies, data transmission technologies, control technologies, etc. to realize functions such as meetings, trainings, monitoring, management, scheduling, linkage, and command. Common product forms include display consoles and the like. Among them, the video processing module is a necessary component of a 4K display console, which realizes compression encoding and network transmission of the screen images and videos of the display console, and at the same time supports decoding and display of multiple-channel monitoring videos and the screen images and videos of the display console.

[0003] A display console usually consists of multiple display screens, and common ones are dual-screen, triple-screen, quadruple-screen, etc. To meet the requirements of video processing for multiple display screens, a common approach is to use corresponding multiple video processing modules, or design multiple identical SOC circuits on one video processing module. The advantages of this design are that the circuits are repeated and easy to transplant. The disadvantages are: first, each SOC needs to lead out independent video network interfaces and control network interfaces, resulting in a high deployment cost and increasing the operation of users; second, the SOCs are not interconnected with each other, and the remaining SOCs cannot replace the functions of one SOC after a failure, which does not meet the development requirements of the display console of "hardware general mutual backup and software deployed as needed".

[0004] The RK3588 chip is an 8K flagship SOC chip, adopting the ARM architecture, supporting multiple video encoding and decoding protocols such as H.264 / H.265, integrating high-quality JPEG encoders / decoders, and having dedicated image preprocessors and postprocessors; the RK3588 integrates 1 HDMI2.0 RX interface. Therefore, for circuits that simultaneously perform video processing on multiple-channel 4K HDMI interfaces, multiple RK3588 chips need to be used to implement. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing, a master-slave multi-core RK3588 architecture design method for implementing multi-channel 4K video processing, which does not require adding circuits such as switches and FPGAs, can simplify the design process, increase practicality; the slave RK3588 is physically isolated externally, increasing data security; each channel RK3588 can flexibly configure the video processing mode according to instructions, increasing the flexibility of software applications.

[0006] The present invention solves its technical problems by adopting the following technical solutions:

[0007] A master-slave multi-core RK3588 architecture for realizing multi-channel 4K video processing, including a multi-core RK3588, LPDDR4X, EMMC, SPI Flash, a network card PHY, and a PCIE network card. Among them, LPDDR4X, EMMC, and SPI Flash are bidirectionally connected to the multi-core RK3588, the network card PHY is bidirectionally connected to the multi-core RK3588 through RMGII, and the PCIE network card is bidirectionally connected to the multi-core RK3588 through PCIE.

[0008] Moreover, the multi-core RK3588 adopts a one-master-multi-slave architecture. The multi-core RK3588 includes a master RK3588 and slave RK3588s. Among them, LPDDR4X, EMMC, and SPI Flash are bidirectionally connected to the master RK3588, the network card PHY is bidirectionally connected to the master RK3588 through RMGII, and the PCIE network card is bidirectionally connected to the master RK3588 through PCIE; LPDDR4X and EMMC are bidirectionally connected to the slave RK3588s, and the network card PHY is bidirectionally connected to the slave RK3588s through RMGII.

[0009] A working method for a master-slave multi-core RK3588 architecture for realizing multi-channel 4K video processing includes the following steps:

[0010] Step 1: The multi-core RK3588 is powered on to load the firmware and complete interface initialization;

[0011] Step 2: The master RK3588 in the multi-core RK3588 completes network port routing and control instruction forwarding, and receives video according to the control instructions;

[0012] Step 3: The multi-core RK3588 runs video processing application software to process the received video;

[0013] Step 4: The multi-core RK3588 waits for a new control instruction sent externally; if a new control instruction is received, the message is parsed and corresponding actions are executed, otherwise it returns to Step 3.

[0014] Moreover, the specific implementation method of Step 1 is as follows: The master RK3588 in the multi-core RK3588 loads the firmware from the SPI flash. The firmware process first initializes the PCIE interface, then initializes the ddr, and finally other loaders. The entire loading process does not exceed 100 ms; the slave RK3588s in the multi-core RK3588 load the firmware from the EMMC.

[0015] Moreover, the specific implementation method of step 2 is as follows: After the main RK3588 in the multi-core RK3588 enters the Linux system, it completes the loading of the PCIE interface and network interface drivers; according to the control instructions sent by the host through PCIE and the control network, it completes the instruction parsing and forwarding; for the video stream coming in from the external video network, it determines which specific channel RK3588 it belongs to according to the port number; and sends it to the corresponding RK3588.

[0016] Moreover, the specific implementation method of step 3 is as follows: The software is developed based on the media processing software platform. The video processing flow inside the software includes video input, video processing, video encoding, video decoding, and video output; the slave RK3588 in the multi-core RK3588 configures and calls software components according to the control instructions sent through the internal network to implement video input / output and video encoding / decoding.

[0017] Moreover, the specific implementation method of step 4 is as follows: The main RK3588 in the multi-core RK3588 enables the PCIE and control network interface interrupts. When a new control instruction comes in, it immediately completes the parsing and response in the interrupt response function; the slave RK3588 in the multi-core RK3588 enables the internal control network interface interrupt, and according to the new instruction, it completes the reconfiguration and call of the video processing application software to complete the instruction response.

[0018] The advantages and positive effects of the present invention are:

[0019] 1. In the present invention, the multi-core RK3588 powers on and loads the firmware to complete the interface initialization; the main RK3588 in the multi-core RK3588 completes the network port routing and control instruction forwarding, and receives videos according to the control instructions; the multi-core RK3588 runs the video processing application software to process the received videos; the multi-core RK3588 waits for new control instructions sent externally; if new control instructions are received, it parses the packets and executes corresponding actions, otherwise it continues to process the received videos. The present invention does not need to add circuits such as switches and FPGAs, which can simplify the design process and increase practicality; the slave device RK3588 is physically isolated externally, increasing data security; each channel RK3588 can flexibly configure the video processing mode according to the instructions, increasing the flexibility of software applications.

[0020] 2. All control interfaces in the present invention are led out by the main RK3588, and the slave RK3588 is flexibly configured according to the instructions, achieving the application effect of hardware general mutual backup and software deployment on demand.

[0021] 3. The present invention first proposes to use the method of network port forwarding to process network video streams, reducing the number of data memory copy processes, reducing latency, and the measured video encoding / decoding latency of four channels at 3840×2160@60fps is not higher than 100ms.

[0022] 4. The present invention uses two PCIE interfaces on the main RK3588. One is used as a PCIE slave device of the host to complete initialization with the host startup, and the other is used as a master device to expand multiple gigabit network interfaces for internal direct connection. This design eliminates the design of the PCIE bridge chip, simplifies the design, and reduces costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a schematic diagram of the architecture of the present invention;

[0024] Figure 2 is a flowchart of the present invention;

[0025] Figure 3 is a flowchart of the architecture data processing of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The present invention will be further described in detail below with reference to the accompanying drawings.

[0027] A master-slave multi-core RK3588 architecture for implementing multi-channel 4K video processing, as Figure 1 shown, includes a multi-core RK3588, LPDDR4X, EMMC, SPI Flash, a network card PHY, and a PCIE network card. Among them, LPDDR4X, EMMC, and SPI Flash are bidirectionally connected to the multi-core RK3588, the network card PHY is bidirectionally connected to the multi-core RK3588 through RMGII, and the PCIE network card is bidirectionally connected to the multi-core RK3588 through PCIE.

[0028] The multi-core RK3588 adopts a one-master-multi-slave architecture. The multi-core RK3588 includes a main RK3588 and slave RK3588s. Among them, LPDDR4X, EMMC, and SPI Flash are bidirectionally connected to the main RK3588, the network card PHY is bidirectionally connected to the main RK3588 through RMGII, and the PCIE network card is bidirectionally connected to the main RK3588 through PCIE; LPDDR4X and EMMC are bidirectionally connected to the slave RK3588s, and the network card PHY is bidirectionally connected to the slave RK3588s through RMGII.

[0029] A working method of a master-slave multi-core RK3588 architecture for implementing multi-channel 4K video processing, as Figure 2 shown, includes the following steps:

[0030] Step 1. The multi-core RK3588 is powered on to load the firmware and complete the interface initialization.

[0031] The specific implementation method of step 1 is as follows: The main RK3588 in the multi-core RK3588 loads the firmware from the SPI flash. The firmware process is to initialize the PCIE interface first, then initialize the ddr, and finally other loaders. The entire loading process takes no more than 100 ms. The slave RK3588 in the multi-core RK3588 loads the firmware from the EMMC.

[0032] Step 2: The main RK3588 in the multi-core RK3588 completes network port routing and control instruction forwarding, and receives video according to the control instructions.

[0033] The specific implementation method of step 2 is as follows: After the main RK3588 in the multi-core RK3588 enters the linux system, it completes the loading of the PCIE interface and network interface drivers; according to the control instructions sent by the host through PCIE and the control network, it completes instruction parsing and forwarding; for the video stream coming in from the external video network, for example, for four-channel 4K video processing, the agreed port numbers are 8551 - 8554. If the detected video stream is 8551, it transfers to the video processing application software branch of the main RK3588; if it is other port numbers, it captures the video stream to other network interfaces and transfers it to the corresponding slave RK3588 by setting the network port forwarding rules.

[0034] Step 3: The multi-core RK3588 runs the video processing application software to process the received video.

[0035] The video processing application software mainly realizes functions such as HDMI video input and output, video encoding and decoding, video scaling combination and overlay, etc. The software is developed based on the Media Process Platform (MPP for short). The internal video processing process of the software includes video input (VI), video processing (VPSS), video encoding (VENC), video decoding (VDEC) and video output (VO), etc. The slave RK3588 configures and calls the above software components according to the control instructions sent through the internal network to realize video input and output and video encoding and decoding.

[0036] Step 4: The multi-core RK3588 waits for new control instructions sent externally; if it receives new control instructions, it parses the message and executes the corresponding actions, otherwise it returns to step 3.

[0037] The main RK3588 in the multi-core RK3588 enables the PCIE and control network interface interrupts. When new control instructions come in, it immediately completes the parsing and response in the interrupt response function. The slave RK3588 in the multi-core RK3588 enables the internal control network interface interrupt, reconfigures and calls the video processing application software according to the new instructions, and completes the instruction response.

[0038] Such as Figure 3As shown in the figure, all services in the 1 master and 3 slave architecture pass through the master RK3588. For services that it is not responsible for, the network packets need to be forwarded inside the SoC to the next corresponding network port, and the slave SoC is responsible for implementing the corresponding encoding and decoding operations after receiving the packets. The specific operations are as follows:

[0039] (1) Adopt the bonding technology to implement a group of externally dual-redundant video networks on the master SoC;

[0040] (2) According to the technical requirements, there is only one IP netip for the video network externally, and 4 port numbers are allocated: 8551 - 8554, corresponding to 4 SoCs respectively;

[0041] (3) Inside the master SoC, adopt the network port forwarding technology (such as the iptables tool) to forward the packets of the video network to different network interfaces according to the port numbers;

[0042] (4) Each independent SoC parses the received packets and performs operations such as video decoding; for the encoding operations of each SoC, the data flow path is the same as above, but in the reverse order.

[0043] In the present invention, the multi-core RK3588 powers on and loads the firmware to complete the interface initialization; the master RK3588 in the multi-core RK3588 completes the network port routing and control instruction forwarding, and receives the video according to the control instructions; the multi-core RK3588 runs the video processing application software to process the received video; the multi-core RK3588 waits for new control instructions sent externally; if new control instructions are received, the packets are parsed and corresponding actions are executed, otherwise the received video is continued to be processed. The present invention does not need to add circuits such as switches and FPGAs, which can simplify the design process and increase the practicability; the slave device RK3588 is physically isolated externally, increasing data security; each channel RK3588 can flexibly configure the video processing mode according to the instructions, increasing the flexibility of software applications.

[0044] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manners. Any other implementation manners obtained by those skilled in the art according to the technical solutions of the present invention also belong to the protection scope of the present invention.

Claims

1. Master-slave multi-core RK3588 architecture for multi-channel 4K video processing, featuring: It includes multi-core RK3588, LPDDR4X, EMMC, SPIFlash, network card PHY and PCIE network card. Among them, LPDDR4X, EMMC and SPIFlash are bidirectionally connected to multi-core RK3588, network card PHY is bidirectionally connected to multi-core RK3588 through RMGII, and PCIE network card is bidirectionally connected to multi-core RK3588 through PCIE.

2. The master-slave multi-core RK3588 architecture for implementing multi-channel 4K video processing according to claim 1, characterized in that: The multi-core RK3588 adopts a one-master-multiple-slave architecture, and the multi-core RK3588 includes a master RK3588 and a slave RK3588, wherein LPDDR4X, EMMC and SPIFlash are bidirectionally connected to the master RK3588, the network card PHY is bidirectionally connected to the master RK3588 through RMGII, and the PCIE network card is bidirectionally connected to the master RK3588 through PCIE; LPDDR4X and EMMC are bidirectionally connected to the slave RK3588, and the network card PHY is bidirectionally connected to the slave RK3588 through RMGII.

3. A working method of a master-slave multi-core RK3588 architecture for implementing multi-channel 4K video processing according to any one of claims 1 to 2, characterized in that: The following steps are involved: Step 1: Power on the multi-core RK3588 and load the firmware to complete interface initialization; Step 2: The master RK3588 in the multi-core RK3588 completes the network port routing and control instruction forwarding, and receives the video according to the control instruction; Step 3: Multi-core RK3588 runs the video processing application software to process the received video; Step 4: Multi-core RK3588 waits for new control instructions to be sent from the outside; if a new control instruction is received, it parses the message and executes the corresponding action, otherwise it returns to step 3.

4. The master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing according to claim 1, characterized in that: The specific implementation method of step 1 is: the master RK3588 in the multi-core RK3588 loads the firmware from the SPI flash. The firmware process is to initialize the PCIE interface first, then initialize ddr, and finally other loaders. The entire loading process does not exceed 100ms; the slave RK3588 in the multi-core RK3588 loads the firmware from the EMMC.

5. The master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing according to claim 1, characterized in that: The specific implementation method of step 2 is as follows: after entering the Linux system, the master RK3588 in the multi-core RK3588 completes the loading of the PCIE interface and the network interface driver; according to the control instructions sent by the host through the PCIE and the control network, the instruction parsing and forwarding are completed; for the video stream coming in from the external video network, the specific channel RK3588 is determined according to the port number; and sent to the corresponding RK3588.

6. The master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing according to claim 1, characterized in that: The specific implementation method of step 3 is: the software is developed based on the media processing software platform, and the video processing flow within the software includes video input, video processing, video encoding, video decoding and video output; the slave RK3588 in the multi-core RK3588 configures and calls the software components according to the control instructions sent from the internal network to realize video input and output and video encoding and decoding.

7. The master-slave multi-core RK3588 architecture and method for implementing multi-channel 4K video processing according to claim 1, characterized in that: The specific implementation method of step 4 is: the master RK3588 in the multi-core RK3588 enables PCIE and controls the network interface interrupt, and when a new control instruction comes in, immediately completes the parsing response in the interrupt response function; the slave RK3588 in the multi-core RK3588 enables the internal control network interface interrupt, completes the reconfiguration call of the video processing application software according to the new instruction, and completes the instruction response.

Images