Six-path ultra-high-definition video live broadcast system based on double-64-bit system chip

Abstract

The invention discloses a six-path ultra-high-definition video live broadcast system based on a double-64-bit system chip, and relates to the technical field of video live broadcast director, during the operation of the system, the system comprises a double-RK3588 SoC collaborative architecture which respectively undertakes six-path 4K signal acquisition, NDI decoding, image format conversion, image scaling, special effect synthesis, preview synthesis and code output; high-speed data transmission is realized through HDMI, MIPI and PCIe, state synchronization is realized through custom TCP signaling, image format conversion and image scaling are performed by using RGA and GPU, transition, picture-in-picture and image matting processing are performed by using GPU, and preview and PGM output paths are separated to optimize performance. According to the invention, the cost of a multipath 4K director system can be reduced, complex special effect processing is realized, and the flexibility and processing efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of video live streaming technology, specifically a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips. Background Technology

[0002] Video live streaming technology is widely used in broadcasting, sports event broadcasting, conferences and exhibitions, and online live streaming. Its core objective is to achieve real-time switching, special effects compositing, and stable output between multiple video sources. As application scenarios evolve from traditional high-definition to immersive, low-latency, and multi-signal concurrency, live streaming systems are gradually evolving from processing single or limited high-definition signals to handling multiple 4K or even higher resolution video streams. Against this backdrop, how to achieve the acquisition, processing, compositing, and stable streaming of multiple 4Kp60 signals under limited hardware resources and power consumption has become a critical technical problem that urgently needs to be solved in the field of ultra-high-definition video live streaming systems.

[0003] Existing SoC-based ultra-high-definition video live streaming solutions generally suffer from severe competition for computing resources, data bus congestion, and redundant processing links when facing six concurrent 4Kp60 signal inputs. Especially when simultaneously performing video acquisition, network decoding, image scaling, special effects compositing, and encoding output, the CPU, GPU, and video processing unit often compete for resources, leading to decreased system stability. Furthermore, some systems rely on external FPGAs or independent broadcasting equipment, which often offer inferior performance and configuration flexibility in special effects processing and configuration, such as transitions and keying, compared to GPU-based methods. In addition, systems relying on very high-end discrete GPUs often have high costs and power consumption, hindering widespread adoption.

[0004] The aforementioned situation and shortcomings mainly stem from the inherent limitations of single-chip processing architecture in terms of bandwidth, parallel processing capabilities, and task isolation. When multiple 4K video streams are processed centrally within the same processing unit, it is highly susceptible to anomalies such as frame loss, increased processing latency, screen tearing, or desynchronized special effects. In network streaming scenarios, issues such as bitrate fluctuations, audio-visual asynchrony, or even streaming interruptions may also occur. These anomalies not only affect the professional quality of the live stream but also reduce the reliability of the system, limiting its application in demanding scenarios such as large-scale events and long-term continuous live streaming, thereby hindering the further promotion and implementation of ultra-high-definition video live streaming systems. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips, solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips, including an image acquisition module, an image decoding module, an image scaling module, an image format conversion module, an image processing module, a PCIe high-speed data transmission module, a screen preview and output module, a network signaling interaction module, and an image encoding module; The image acquisition module is used to receive 6 external 4Kp60 HDMI signals and convert the signals into MIPI CSI format through an HDMI to MIPI bridge chip before inputting them to two RK3588 SoCs. The image decoding module is used for hardware decoding of network streaming media signals (NDI protocol) and compressed video streams in local DDR cache; The image scaling module is deployed inside two RK3588 SoCs and is used to downsample the original 4K video source or the decoded video stream into a 540p resolution thumbnail for use in the preview interface. The image format conversion module is used to convert between color spaces such as YUV422 and RGBA, and RGBA and NV12, to adapt to the data format requirements of different processing units. The image processing module in RK3588-2 is used to perform transitions, picture-in-picture (PIP) and chroma keying operations on 4K original images, while in RK3588-1 it is used to overlay logos and CG. The PCIe high-speed data transmission module connects two RK3588 SoCs and is used for bidirectional transmission of raw 4K video frames and 540p preview images between RK3588-1 and RK3588-2. The network signaling interaction module is based on the RGMII physical layer interface and uses a custom TCP protocol to synchronize the broadcast switching status, user operation commands and system operating parameters between the two SoCs. The preview and output module stitches together all the preview images and displays them via HDMI or LCD preview. The image encoding output module is integrated into the RK3588-1 SoC. It is used to encode the final PGM signal after overlaying the logo and CG into H.264 / H.265 format and push it to a network streaming media server or record it to a USB.

[0007] Preferably, the video acquisition module includes 6 HDMI input interfaces and 6 LT6911UXE HDMI-to-MIPI bridge chips; Of the 6 HDMI signals, 2 are connected to the MIPI CSI interface of RK3588-2, 2 are connected to the MIPI CSI interface of RK3588-1, and the remaining 2 are split into two and connected to RK3588-1 and RK3588-2 respectively. In the end, RK3588-1 and RK3588-2 each receive 4 MIPI signals, including 2 overlapping signals.

[0008] Preferably, the image scaling module is executed using the RGA (Raster Graphics Acceleration) hardware unit built into the RK3588; The RK3588-2 uses RGA to perform bilinear interpolation downsampling to 960×540 resolution for two of the four HDMI signals it directly acquires that are different from the original 4K frames of the RK3588-1. The RK3588-2 also uses RGA to scale the 4K video frames output by NDI decoding to 540p small images; the RK3588-1 takes one copy of each of its four HDMI signals and scales it to 540p small images using RGA. All 540p thumbnails are transmitted via PCIe to the RK3588-1, where its GPU performs YUV to RGBA format conversion for use in multi-view preview (MV) compositing.

[0009] Preferably, the image format conversion module in the RK3588-2 converts the raw video data from HDMI, PCIe and decoder into RGBA format for use by GPU special effects processing; In this process, the raw data output by NDI decoding is first converted from RGA to RGBA and then sent to the GPU; after receiving the pure PGM signal output by RK3588-2 through HDMI, RK3588-1 is converted by the GPU to RGBA to overlay the logo and CG. Before the final PGM output, the RK3588-1 calls RGA to convert the composite image in RGBA format to NV12 for use by the encoder.

[0010] Preferably, the video effects processing module runs on the Mali-G610 GPU of the RK3588-2, and performs pixel-level parallel computing using Open GLES; The transparency gradient between the two sources is used to achieve the picture-in-picture function, which scales up the 4K image from the secondary source and embeds it into a specified area of ​​the main source. The coordinates and size are set by the user through the signaling module. Chroma keying is based on chroma keying, which generates a transparency mask for the green or blue background area of ​​the input source to preserve the foreground person or object.

[0011] Preferably, the PCIe high-speed data transmission module is configured in Gen3 x4 mode, with a unidirectional bandwidth of up to 30Gbps; RK3588-1 sends its two dedicated 4K raw frames to RK3588-2 via PCIe, and each frame is written to the reserved video memory area of ​​RK3588-2 in DMA mode. RK3588-2 sends 6 channels of 540p preview images to RK3588-1 via PCIe. Each image is organized in an independent memory block and is accompanied by a source ID identifier. PCIe communication employs a circular buffer and interrupt notification mechanism to ensure low latency in video frame transmission.

[0012] Preferably, the network signaling interaction module uses two RK3588 Gigabit Ethernet MAC controllers, which are connected to the same switching chip through the onboard RGMII interface; The signaling protocol is defined as a TCP-based binary structure, which includes operation type fields (such as switching sources, starting recording), parameter fields (such as PIP position, transition duration), and verification fields. RK3588-1 acts as a host node, listening for user UI events and generating signaling packets to send to RK3588-2. RK3588-2 parses the signaling packets, updates its internal state machine, and sends back an acknowledgment packet.

[0013] Preferably, the video encoding output module calls the built-in H.264 / H.265 hardware encoder in the RK3588-1; The encoder input is NV12 format PGM video, with a maximum resolution of 3840×2160 and a maximum frame rate of 60fps.

[0014] Preferably, the system adopts a hierarchical processing architecture. The RK3588-2 serves as the first-level processing unit, responsible for decoding all six 4K signals, compositing special effects, and generating clean PGM; the RK3588-1 serves as the second-level processing unit, responsible for preview image compositing, logo / CG overlay, final PGM output, recording, and streaming. The preview display path and PGM output path are processed separately: the preview uses a 540p small image, which is then composited by the GPU and output to the local LCD. The PGM output uses the original 4K image, which is processed through a series of steps and then sent to the HDMI display or to the encoder. This separation mechanism reduces the GPU load by 37% and avoids the impact of preview rendering on 4K compositing performance due to the use of texture units.

[0015] Preferably, the system also includes a power management unit and a heat dissipation structure; The power management unit provides 12V / 6A DC power to each of the two RK3588 chips, and stabilizes the core voltage at 0.8V±2% through an LDO regulator; The heat dissipation structure uses an aluminum fin heatsink in conjunction with heat pipes to cover the SoC surface. One end of the heat pipe is attached to the SoC's metal shield, and the other end extends to the chassis fan exhaust vent.

[0016] This invention provides a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips, which has the following beneficial effects: (1) During system operation, through unified planning and collaborative design of the image acquisition module, image decoding module, image scaling module, image format conversion module, image processing module, PCIe high-speed data transmission module, network signaling interaction module, image encoding module, and screen preview and output module, this system constructs a highly complete and closed-loop 6-channel 4Kp60 ultra-high-definition video live broadcast processing system. The modules are functionally interconnected and have clear hierarchical data flow, enabling multiple ultra-high-definition signals to be accessed, processed, synthesized, and output under a unified architecture. This design not only realizes the simultaneous management and scheduling of 6 channels of ultra-high-definition video signals, but also effectively avoids the data path chaos and processing redundancy problems caused by function stacking in traditional broadcast systems, thereby improving the overall controllability and reliability of the system.

[0017] (2) The system adopts a dual 64-bit SoC hierarchical processing mode, which concentrates high-load tasks such as decoding and special effects processing in the first-level processing unit, and distributes preview compositing, image and text overlay and encoding output tasks to the second-level processing unit. The high-bandwidth and low-latency interconnection between the two is achieved through the PCIe high-speed data transmission module. Compared with the existing single-chip centralized processing scheme, this architecture significantly alleviates the bottleneck of computing resources and memory bandwidth, enabling stable parallel processing of multiple 4K videos under high frame rate conditions.

[0018] (3) Leveraging the hardware acceleration capabilities of the image scaling and image format conversion modules, the system maintains high-quality output of the 4K main screen while generating low-resolution preview images for multi-screen monitoring, effectively balancing image quality and processing efficiency. The image processing module fully utilizes the parallel computing capabilities of the GPU to implement professional directing functions such as transitions, picture-in-picture, and chroma keying, ensuring continuous image quality, smooth edges, and consistent colors during special effects processing. Compared to traditional solutions relying on CPUs or software algorithms, this system significantly reduces the probability of abnormal phenomena such as frame loss, image jitter, and synchronization deviations during long-term operation and in high-complexity special effects scenarios. Compared to traditional solutions relying on FPGAs, this system offers more comprehensive and better image processing, while also providing more flexible configuration.

[0019] (4) By tightly integrating the image encoding module and the front-end processing module into the same system platform, this system achieves a highly integrated design from signal input to finished live stream output, reducing the latency and instability caused by external encoding equipment and multi-level switching. With a well-designed power management and heat dissipation structure, the system can maintain stable operation under high load and long-term continuous live streaming scenarios. Compared with existing ultra-high-definition video live streaming technologies, this solution has achieved significant improvements in integration, reliability, and functional expansion capabilities, providing effective support for the development of ultra-high-definition broadcasting systems towards miniaturization, specialization, and high cost-effectiveness. Attached Figure Description

[0020] Figure 1 This is a block diagram of the overall system architecture of the 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips of the present invention; Figure 2 This is a schematic diagram of the connection structure of the video acquisition module of the present invention; Figure 3 This is a schematic diagram of the image preview processing path of the present invention; Figure 4 This is a flowchart of the video effects processing of the present invention; Figure 5 This is a schematic diagram of the communication mechanism of the PCIe high-speed data transmission module of the present invention; Figure 6 This is a schematic diagram of the protocol stack and data flow of the network signaling interaction module of the present invention; Figure 7 This is a schematic diagram of the hierarchical processing architecture of the system of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1 This invention provides a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips. Please refer to [link / reference]. Figure 1 It includes an image acquisition module, an image decoding module, an image scaling module, an image format conversion module, an image processing module, a PCIe high-speed data transmission module, a network signaling interaction module, an image encoding module, and a screen preview and output module 17; The image acquisition module 1 and image acquisition module 10 are respectively connected to the MIPI CSI interfaces of the RK3588-2 SoC and the RK3588-1 SoC; the image decoding module 1 is used to process NDI streaming media signals and compressed video streams cached in local DDR; the image scaling module 5 and image scaling module 12 are composed of the RGA hardware unit built into the RK3588; the image format conversion module 3 connects the image decoding module 1, image acquisition module 2, PCIe high-speed transmission module and image processing module 5 to realize the conversion of YUV to RGBA; the image format conversion module 13 connects the image scaling module 12, PCIe high-speed transmission module 11 and image processing module 14 to realize the conversion of YUV to RGBA; the image processing module 5 is integrated into the Mali-G610 GPU of the RK3588-2 SoC and is used for special effects processing such as keying and picture-in-picture for 4K original images; the PCIe high-speed data transmission module 8 is used to realize the connection between the RK3588-1 SoC and the RK3588-2 SoC based on the PCIe bus. The bidirectional high-speed data transmission between the SoCs is connected to the PCIe controller pins of the two SoCs; the network signaling interaction module is connected through the Gigabit Ethernet MAC controller of the two SoCs; the image encoding module is integrated inside the RK3588-1 and connected to the network streaming unit.

[0023] In the accompanying drawings, PCIe high-speed data transmission module 8 and PCIe high-speed data transmission module 11 are used to illustrate the logical interface representation of the same PCIe high-speed data transmission unit on different SoC sides, which physically constitute a bidirectional PCIe data link.

[0024] Example 2 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figures 2 to 4Specifically: the image acquisition module includes a total of 6 HDMI input signals and 6 LT6911UXE HDMI-to-MIPI bridge chips; image acquisition module 2 includes 4 HDMI inputs: HDMI1, HDMI2, HDMI3, and HDMI4; image acquisition module 10 includes 4 HDMI inputs: HDMI3, HDMI4, HDMI5, and HDMI6, where HDMI3 and HDMI4 are shared by image acquisition modules 1 and 10; the 6 HDMI input interfaces 18, 19, 20, 21, 22, and 23 are respectively connected to the 4Kp60 output of external cameras or playback devices. HDMI signal; among them, HDMI input interfaces 18 and 19 are connected to RK3588-2 via LT6911UXE, HDMI input interfaces 22 and 23 are connected to RK3588-1 via LT6911UXE, and HDMI input interfaces 20 and 21 are connected to both RK3588-2 and RK3588-1 via LT6911UXE; LT6911UXE can realize 1-to-2 conversion of 4Kp60 HDMI-to-MIPI signal.

[0025] like Figure 3 As shown, the image scaling module 4 performs downsampling processing on the original 4K video source 24 in the RK3588-2 SoC; the original 4K video source 24 includes the first and second HDMI signals directly acquired by the RK3588-2 SoC, or the 4K frames decoded by NDI output from the image decoding module 1; each of the above 4K signals is sent to the RGA scaling unit 25, which performs a bilinear interpolation algorithm to scale each 4K frame to a 540p preview thumbnail 26 with a resolution of 960×540; the 540p preview thumbnail 26 is transmitted via DMA through the PCIe transmission channel 27 to the video memory area of ​​the PCIe output channel 28 of the RK3588-1 SoC; the RK3588-1 SoC also performs downsampling processing on the original 4K video source 29; the original 4K video source 24 includes the RK3588-1 The 3rd to 6th HDMI signals directly acquired by the SoC are sent to the RGA scaling unit 30 to generate corresponding 540p preview thumbnails 31. After format conversion, all 540p preview thumbnails are stitched together by the multi-screen preview synthesis unit 32 composed of GPUs in the RK3588-1 SoC to form a 2×3 layout preview screen, which is then output to the local LCD display.

[0026] like Figure 4As shown, image processing module 5 is deployed in the Mali-G610 GPU of the RK3588-2 SoC, and image processing module 14 is deployed in the Mali-G610 GPU of the RK3588-1 SoC. The main video source 33 and the auxiliary video source 34 are each from any two of the six 4K signals, specified by the user through signaling commands. The main video source 33 and the auxiliary video source 34 are sent to the transition mixing / picture-in-picture embedding / chroma keying unit 35. This unit performs corresponding processing according to the operation type in the signaling. If it is a transition, the brightness difference between two 4K frames is adjusted to achieve a transition effect. If it is a picture-in-picture function, the two signal sources are superimposed according to the coordinate position and size of the provided main / auxiliary signal sources. If it is a keying function, the auxiliary video source 34 generates a transparency mask for the corresponding pixel according to the input color value and superimposes the effect on the main video source 33. The 4K frame output after any or combined processing is a mixed image 36. This signal does not contain logo or CG information and is processed by the RK3588-2 SoC. The SoC's HDMI output interface transmits data to the RK3588-1 SoC.

[0027] Example 3 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figures 5-7 Specifically: such as Figure 5 As shown, the PCIe high-speed data transmission module 8 uses the PCIe Gen3 x4 physical layer to connect the RK3588-1 SoC and the RK3588-2 SoC. Through the PCIe high-speed data transmission module 8, the RK3588-1 SoC writes the 5th and 6th 4K raw frames into the RK3588-2 SoC's video memory via DMA. At the same time, the RK3588-2 SoC transmits a 540p preview thumbnail to the RK3588-1 SoC through the PCIe high-speed data transmission module 8. The RK3588-1 video memory reserves a contiguous physical memory area to store its exclusive 5th and 6th channel 4K raw frames; the RK3588-2 video memory reserves another contiguous physical memory area to receive this data; the DMA controller 40 is configured to read the 5th and 6th channel 4K frames from the RK3588-1 video memory and write them to a specified address in the RK3588-2 video memory circular buffer 41 via a PCIe link; during reverse transmission, the RK3588-2 SoC organizes the 6 channels of 540p preview images into independent memory blocks, with a 4-byte source ID identifier appended to the header of each memory block, and writes them to the RK3588-1 video memory circular buffer 38 via the DMA controller 37; after writing a frame, the DMA controller 37 triggers the interrupt signal line 39 to notify the RK3588-1 SoC to read the data; in a preferred embodiment, this mechanism ensures that the single frame transmission delay does not exceed 16 milliseconds.

[0028] like Figure 6As shown, in the network signaling interaction module 7, user UI events 42 are captured by the graphical interface running on the RK3588-1 SoC, including operations such as switching sources, starting recording, and setting PIP positions. The signaling generation unit 43 encapsulates the UI events 42 into a binary structure, which includes an operation type field (1 byte), a parameter field (variable length), and a CRC32 check field (4 bytes), and sends it to the signaling parsing and state machine 44 of the RK3588-2 SoC via the TCP protocol. After verifying the CRC32, the RK3588-2 signaling parsing and state machine 44 parses the operation type and parameters, updates the internal status register, and generates an acknowledgment return packet 45. The acknowledgment return packet 45 is replied to the RK3588-1 SoC via unicast TCP, completing one signaling interaction.

[0029] like Figure 7 As shown, the system adopts a hierarchical processing architecture. The RK3588-2 SoC, as the first-level processing unit 46, is responsible for 4K signal acquisition, NDI decoding, 540p scaling, GPU special effects compositing, and pure PGM generation. The RK3588-1 SoC, as the second-level processing unit 47, is responsible for receiving six 540p preview thumbnails and compositing them into multi-screen previews, while also receiving the pure PGM signal output by the RK3588-2 SoC 10. The RK3588-1 outputs through two paths: the preview rendering path 52 sends the 540p multi-screen composite result to the local LCD output 53; the PGM output path 49 sends the final image after the 4K signal passes through the Logo / CG overlay module 50 to the HDMI output or encoder encoding. The Logo / CG overlay module 50 is executed by the GPU of the RK3588-1 SoC, which overlays preset logos, subtitles, and other graphic elements onto the PGM image using an alpha blending method.

[0030] Example 4 This embodiment is an explanation based on Embodiment 1. Please refer to it. Figures 1 to 7Specifically: During actual operation, six external 4K camera signals are input to the system via HDMI input interfaces 18-23; signals 1 to 4 are converted by the first set of LT6911UXE HDMI-to-MIPI bridge chips and then input to the image acquisition module 2 of RK3588-2; signals 3 to 6 are converted by the second set of LT6911UXE HDMI-to-MIPI bridge chips and then input to the image acquisition module 10 of RK3588-1; RK3588-2 SoC receives the first to fourth raw 4K frames and obtains the fifth and sixth 4K frames from RK3588-1 SoC through PCIe high-speed data transmission module 8; simultaneously, RK3588-2 SoC receives NDI streaming media signals via the network, which are hardware decoded by video decoding module 1; the first and second HDMI 4K frames and the NDI decoded frames are processed by RK3588-2. The SoC generates a 540p preview thumbnail 26 via the RGA scaling unit 25 and sends it to the RK3588-1 SoC via the PCIe transmission channel 27. The RK3588-1 SoC synthesizes a 6-screen preview and outputs it to the local LCD. When the user selects the primary and secondary sources through the UI and triggers a switching command, the RK3588-1 SoC generates a TCP signaling packet and sends it to the RK3588-2 SoC 10. After parsing the command, the RK3588-2 SoC performs transition, PIP, or keying operations in the Mali-G610 GPU, generating a clean PGM signal 36, which is then output to the RK3588-1 SoC via HDMI. The RK3588-1 SoC converts this signal to RGBA format, overlays a logo / CG, converts it to NV12, and sends it to the HDMI output or hardware encoder. Throughout the process, the PCIe transmission uses DMA and a circular buffer 38 in conjunction with an interrupt mechanism to ensure frame synchronization. To enable those skilled in the art to fully understand and implement this invention, the specific implementation principles of this invention are further supplemented below with a specific application scenario.

[0031] Example 5 For a 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips, please refer to... Figure 2Specifically: When the director selects the second channel (HDMI input video interface 19) as the main video source 33 and the fifth channel (HDMI input video interface 22 or 23) as the auxiliary video source 34 in the graphical interface running on the RK3588-1 SoC and enables fade-in / fade-out transitions, UI event 42 is captured by signaling generation unit 43 and encapsulated into a binary structure containing the operation type field "0x03", the transition duration parameter "2000ms", and the CRC32 checksum; this structure is encapsulated via TCP protocol and sent to the signaling parsing and state machine 44 of the RK3588-2 SoC; after receiving the packet, the signaling parsing and state machine 44 in the RK3588-2 SoC 10 verifies the CRC32 correctness, parses out the main and auxiliary source IDs and transition parameters, and configures the Mali-G610. The GPU's internal transition mixing unit 35; the transition mixing unit 25 performs brightness changes on the second and fifth 4K frames according to time weight t / T within the next 2000 milliseconds, generating smooth transition frames; the 4K frames processed as above are used as pure PGM signals 36, and transmitted to the RK3588-1 SoC in YUV422 format through the HDMI output interface of the RK3588-2 SoC. This process is as follows. Figure 4 As shown, in a preferred embodiment, special effects processing is ensured to be completed entirely within the primary processing unit 38, avoiding latency accumulation caused by frequent cross-chip interactions.

[0032] After receiving the pure PGM signal 36, the RK3588-1 SoC uses its GPU to perform Alpha blending operations via the Logo / CG overlay module 50, overlaying the preset match logo and real-time score text onto the PGM screen in RGBA format. The final overlaid image is then converted from RGBA to NV12 format by the image format conversion module 15 and sent to the image encoding module 16. The image encoding module 16 activates the RK3588-1's built-in H.265 encoder, compressing the NV12 frames at 3840×2160@60fps using CBR 50 Mbps, GOP structure IBBP, and Main Profile. The encoded bitstream is pushed to the cloud streaming media server via the RTMP protocol. Simultaneously, the preview rendering path 52 independently outputs a 540p multi-screen composite result to the local LCD output 53. The two image acquisition paths are completely separated after the image acquisition module, as shown below. Figure 7 As shown, this ensures that the 4K PGM output is not affected by the preview compositing load.

[0033] Throughout the entire data transfer process, the PCIe high-speed data transmission module 8 adopts... Figure 5The DMA and circular buffer 38 coordination mechanism is as follows: When the RK3588-2 SoC sends 6 channels of 540p preview images to the RK3588-1 SoC, each image is organized as an independent memory block with a 4-byte source ID (such as 0x01~0x06) appended to the header. The DMA controller 37 sequentially writes the memory blocks into the RK3588-1 video memory circular buffer 38. An interrupt 39 is triggered after each frame is written by the DMA. This mechanism ensures that the preview image transmission delay is stable within 16 milliseconds, meeting the 60fps real-time requirement. Similarly, when the RK3588-1 SoC 9 transmits the 5th and 6th channels of 4K raw frames to the RK3588-2 SoC 10, it also directly writes them to the reserved physical address of the RK3588-2 video memory circular buffer 41 via DMA, without CPU intervention, ensuring lossless transmission of high-bandwidth raw video streams.

[0034] During system operation, in a preferred embodiment, the power management unit provides 12V / 6A power to the RK3588-1 SoC and RK3588-2 SoC respectively through two independent DC-DC conversion circuits. The LDO regulator accurately stabilizes the core voltage at 0.8V±2%, and suppresses ripple to below 10 mV to prevent voltage fluctuations from causing PLL lockout or DDR timing errors within the SoC. In the heat dissipation structure, the aluminum fin heat sink is tightly attached to the SoC metal shield with thermal grease, and the heat pipes conduct heat from the chip hotspots to the exhaust vent of the rear fan of the chassis. When the temperature control sensor detects that the SoC junction temperature exceeds 75°C, the chassis fan starts running at full speed to maintain the chip junction temperature below 85°C, ensuring that the Mali-G610 GPU and RGA unit operate stably under continuous high load and avoiding a decrease in video processing frame rate due to overheating and frequency reduction.

[0035] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A 6-channel ultra-high-definition video live streaming system based on dual 64-bit system chips, characterized in that: It includes an image acquisition module, an image decoding module, an image scaling module, an image format conversion module, an image processing module, a PCIe high-speed data transmission module, an image encoding module, a screen preview and output module, and a network signaling interaction module; The image acquisition module is deployed on two SoCs to receive six external 4Kp60 HDMI signals. These signals are then input to the RK3588-1 and RK3588-2 SoCs via six HDMI-to-MIPI bridge chips LT6911UXE. Each of the RK3588-1 and RK3588-2 receives four signals. The first and second signals are connected to the RK3588-2, as are the fifth and sixth signals. The third and fourth signals are then split into two MIPI signals after passing through the LT6911UXE and simultaneously connected to both the RK3588-1 and RK3588-2. The image decoding module is deployed on the RK3588-2 and is used to receive network streams (NDI protocol) and local video files, and call the hardware decoding unit to perform decoding; The image scaling module is deployed on two SoCs and is used to downsample the original 4K video or the decoded video stream to a small image with a resolution of 960×540. The image format conversion module employs two conversion methods, RGA conversion and GPU conversion, and is deployed on two SoCs. It mainly realizes the mutual conversion between YUV422 and RGBA and between RGBA and NV12 color spaces. The image processing module is deployed on two SoCs and mainly calls the GPU to perform transitions, picture-in-picture and chroma keying on the image on the RK3588-2, and overlay logo and CG on the image on the RK3588-1, and finally outputs an image in RGB format. The PCIe high-speed data transmission module enables bidirectional high-speed transmission of raw 4K video frames and thumbnails between two SoCs. The network signaling interaction module is based on the RGMII interface and uses the TCP protocol to synchronize the broadcast switching status, user operation commands and system operating parameters. The preview and output module is deployed on the RK3588-1, which mainly stitches together thumbnails into a complete MV screen and outputs it via HDMI preview. The image encoding module is integrated into the RK3588-1. It encodes the final PGM signal after overlaying the logo and CG with H.264 or H.265 and outputs it via network streaming media or by recording it to a USB.

2. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The video capture module includes 6 HDMI input interfaces and 6 LT6911UXE HDMI-to-MIPI bridge chips; Of the 6 HDMI input interfaces, the first and second are connected to two LT6911UXE HDMI-to-MIPI bridge chips, and their MIPI CSI outputs are connected to the RK3588-2 MIPI CSI interface. Channels 3 and 4 connect to two sets of LT6911UXE HDMI-to-MIPI bridge chips. Each LT6911UXE chip outputs two MIPI CSI signals, and each MIPI CSI output connects to RK3588-1 MIPI CSI and RK3588-2 MIPI CSI respectively. The LT6911UXE HDMI-to-MIPI bridge chip is configured to support 4Kp60 YUV422 8bit input and can output two sets of 4-Lane MIPI CSI signals, with a rate of no less than 2.5 Gbps per lane.

3. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The image scaling module is executed using the RGA hardware unit built into the RK3588; The RK3588-2 SoC uses RGA to perform bilinear interpolation downsampling to 960×540 resolution for the first and second HDMI signals it directly acquires and the 4K video frames it decodes and outputs. The RK3588-1 SoC uses RGA to scale the 3rd, 4th, 5th, and 6th HDMI signals it directly acquires to a small image with a resolution of 960×540. All 960×540 resolution thumbnails from the RK3588-2 are transmitted to the RK3588-1 SoC via the PCIe high-speed data transmission module. The GPU then performs YUV to RGBA format conversion for multi-screen preview compositing.

4. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The image format conversion module in the RK3588-2 SoC converts the raw video data from HDMI, PCIe and video decoding modules into RGBA format. The raw data output by NDI decoding is in NV12 format, which is first converted from RGA to RGBA and then sent to the GPU. After receiving the pure PGM signal in YUV422 format from the RK3588-2 SoC via HDMI, the GPU of the RK3588-1 SoC converts it to RGBA to overlay the logo and CG. Before the final PGM output, the RK3588-1 SoC calls RGA to convert the composite image in RGBA format to NV12 for use by the video encoding module.

5. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The video processing module runs on the Mali-G610 GPU of the RK3588-2 SoC, and performs pixel-level parallel computing using Open GLES; The transition function is achieved by setting the transparency gradient of two sources; The picture-in-picture function scales up the 4K image from the secondary source and embeds it into a designated area of ​​the main source. The coordinates and size are set by the user through the network signaling interaction module. Chroma keying is based on chroma keying, which generates transparency masks for the green or blue background areas of the input source.

6. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The PCIe high-speed data transmission module is configured in Gen3 x4 mode; The RK3588-1 SoC sends its exclusive 5th and 6th 4K raw frames to the RK3588-2 SoC via PCIe, and each frame is written to the reserved area of ​​the RK3588-2's memory in DMA mode. The RK3588-2 SoC sends six 960×540 resolution small images to the RK3588-1 SoC via PCIe. Each image is organized in an independent memory block and comes with a 4-byte source ID identifier. PCIe communication uses a ring buffer and an interrupt signal line notification mechanism.

7. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The network signaling interaction module uses the Gigabit Ethernet MAC controller of the RK3588-1 SoC and the RK3588-2 SoC, and connects through the onboard RGMII interface; The signaling protocol is defined as a custom binary structure based on TCP, which includes a 1-byte operation type field, a variable-length parameter field, and a 4-byte CRC32 check field. The RK3588-1 SoC acts as a host node, listening to user UI events and generating TCP signaling packets to send to the RK3588-2 SoC. After parsing the signaling, the RK3588-2 SoC updates its internal state machine and sends back an acknowledgment packet.

8. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The image encoding module calls the built-in H.264 / H.265 hardware encoder in the RK3588-1 SoC; The encoder input is the final PGM image in NV12 format, with a maximum resolution of 3840×2160 and a frame rate of 60 fps; The encoding parameters include a CBR bitrate of 50 Mbps, a GOP structure of IPPP, and Profile Main.

9. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The system adopts a hierarchical processing architecture, with the RK3588-2 SoC serving as the primary processing unit, responsible for decoding six channels of raw 4K signals, synthesizing special effects, and generating clean PGM signals. The RK3588-1 SoC, as a secondary processing unit, is responsible for small image preview compositing, logo / CG overlay, final PGM output, recording, and streaming. The preview rendering path and PGM output path are separated after the acquisition module of RK3588-2 SoC and RK3588-1 SoC: the preview uses a small image with a resolution of 960×540, which is then synthesized by the GPU and output to the local LCD. The PGM output uses the original 4K image, RK3588-2 special effects processing, and RK3588-1 logo / CG overlay processing before being sent to the encoded push stream output or HDMI output.

10. The 6-channel ultra-high-definition video live streaming system based on a dual 64-bit system chip according to claim 1, characterized in that: The system also includes a power management unit and a heat dissipation structure; The power management unit provides 12V / 6A DC power to both the RK3588-1 SoC and the RK3588-2 SoC, and stabilizes the core voltage at 0.8V±2% through an LDO regulator; The heat dissipation structure includes an aluminum fin heat sink and heat pipes. One end of the heat pipe is attached to the SoC metal shield, and the other end extends to the chassis fan exhaust vent.

Images