Picture-in-picture video implementation method and processing system based on RK3588

Abstract

The invention discloses an RK3588-based picture-in-picture video implementation method and a processing system, relates to a video processing technology, and aims to solve the problems of low efficiency, high delay and difficulty in balancing picture quality and bandwidth in an existing picture-in-picture processing scheme. The method sequentially comprises the steps of audio and video synchronous acquisition, VPU hardware decoding, zero-copy memory mapping transmission, picture-in-picture synthesis, VPU layered coding and MP4 packaging storage. The system comprises a video acquisition module, a VPU audio and video coding and decoding module, a picture-in-picture synthesis module, a real-time display module, a video recording module and an auxiliary memory and power consumption management unit. Through a hardware acceleration and optimization algorithm, picture-in-picture processing is low in delay, high in picture quality and low in system load, the method is suitable for scenes such as video conferences and real-time monitoring, the overall structure is simple and efficient, and various application requirements are met.

Description

Technical Field

[0001] This invention relates to the field of video processing technology, and in particular to a picture-in-picture video implementation method and processing system based on RK3588. Background Technology

[0002] With the rapid development of video communication technology, picture-in-picture video processing technology is being used more and more widely in scenarios such as portable talk hosts, video conferencing systems, and remote collaboration terminals. These scenarios generally require the simultaneous processing of multiple video streams, such as local camera footage and remote participant footage, or main scene footage and auxiliary information footage, and the real-time synthesis, display, and recording of multiple video streams. The core requirements are concentrated in four dimensions: low-latency interaction, low-power operation, multi-channel high-definition adaptation, and compatibility with domestic platforms.

[0003] Current picture-in-picture video processing solutions mainly fall into two categories, but both have significant technical shortcomings and are difficult to meet the actual needs of the above scenarios: The first type of solution relies on a general-purpose processor (CPU) for software encoding / decoding and video synthesis. This type of solution requires no dedicated hardware support and has lower development costs, but it suffers from three major problems: First, software encoding / decoding consumes a significant amount of CPU computing resources. When processing two or more channels of 1080P or 4K high-definition video, the CPU load often exceeds 80%, easily leading to system stuttering and frame rate drops. Second, the software processing flow involves numerous data transfer and algorithm calculation steps, resulting in end-to-end latency generally exceeding 100ms, which cannot meet the real-time interactive requirements of scenarios such as video conferencing and live interviews (typically requiring a latency of ≤50ms). Third, the high CPU load directly leads to a surge in device power consumption. For battery-powered portable devices, battery life can be reduced by more than 30%, severely limiting the user experience in portable scenarios.

[0004] The second type of solution uses dedicated video processing chips for hardware acceleration. While this can reduce CPU load to some extent, it still has two major shortcomings: First, most dedicated chips are designed for single or two standard definition / near-high definition video streams, lacking the ability to process multiple high-specification videos such as 4K@60fps in parallel. When the number of input video streams increases or the resolution is improved, decoding congestion and compositing stutters are likely to occur. Second, these dedicated chips are mostly designed by foreign manufacturers, and their software and hardware interfaces have poor compatibility with domestic operating systems (such as Kylin system) and domestic processor platforms. A large number of additional drivers and adaptation programs need to be developed, which not only increases development costs but may also lead to a decrease in system stability due to compatibility issues.

[0005] As a representative of domestically produced high-performance processors, the RK3588 chip boasts powerful hardware processing potential thanks to its built-in Video Processing Unit (VPU) and Graphics Processing Unit (GPU). The VPU supports encoding and decoding of multiple high-definition videos, while the GPU can efficiently perform graphics rendering. However, existing video processing technologies based on the RK3588 do not fully utilize the chip's hardware characteristics: functionally, they lack flexible support for dynamic picture-in-picture layouts (such as real-time adjustment of the main and secondary screen positions and scaling ratios); performance-wise, they lack specific optimizations for low-latency compositing, with the entire decoding-compositing-display process still lagging behind 60ms; and in terms of system collaboration, the resource scheduling mechanism between the VPU, CPU, and GPU is imperfect, resulting in low hardware resource utilization and failing to fully leverage the RK3588's low-power advantages.

[0006] In summary, existing picture-in-picture video processing solutions suffer from problems such as high CPU load, high latency, high power consumption, weak multi-channel HD adaptation capability, and poor compatibility with domestic platforms. Furthermore, they do not fully utilize the hardware potential of the RK3588 chip. There is an urgent need for a high-efficiency picture-in-picture video processing solution based on the RK3588 to meet the actual needs of scenarios such as portable talk hosts and video conferencing systems. Summary of the Invention

[0007] The purpose of this invention is to provide a picture-in-picture video implementation method and processing system based on RK3588, which meets the core requirements of portable talk hosts, video conferencing systems and other scenarios for real-time synthesis of multiple video streams, low-latency interaction, low-power operation and domestic adaptation.

[0008] To achieve the above objectives, this invention provides a picture-in-picture video implementation method based on RK3588, comprising the following steps: S1. Audio and video synchronous acquisition: Acquire multiple video streams through the multi-channel video input interface of the video acquisition module, and simultaneously acquire audio signals; S2. Audio and video hardware decoding: The VPU audio and video codec module of the RK3588 chip is enabled to perform hardware decoding on the acquired multi-channel video streams and output YUV format video frames; at the same time, the synchronously acquired audio signals are decoded. S3. Picture-in-Picture Compositing: The picture-in-picture compositing module combines the decoded main screen video frames and sub-screen video frames into a composite image. S4. Real-time display: The real-time display module outputs the composite video frames rendered by the GPU to the display interface, and adaptively adjusts the resolution of the output screen according to the display requirements. S5. Audio and video encoding: The synthesized picture is hardware encoded through the VPU audio and video encoding and decoding module, and the decoded audio signal is AAC encoded at the same time. S6. Audio and video encapsulation and recording: The video recording module encapsulates the encoded video data and encoded audio data into an MP4 file and stores it.

[0009] Preferably, in step S2, a multi-channel video time-slice scheduling algorithm is used to perform hardware decoding on multiple video streams. The input video stream is divided into time slices according to the frame interval, and different priority weights are assigned to the decoding tasks by configuring the hardware registers of the VPU, so that the decoding delay of the main screen video stream is no higher than 8ms and the decoding delay of the sub-screen video stream is no higher than 12ms.

[0010] Preferably, between step S2 and step S3, a memory optimization step is also included: using zero-copy memory mapping technology, the YUV video frames output by VPU hardware decoding are directly stored in the continuous memory area reserved by CMA, and the physical address is mapped to the texture memory of the GPU through ION memory driver for use by the picture-in-picture compositing module.

[0011] Preferably, in step S5, layered encoding control is adopted to identify the main screen area in the composite image as the ROI area, and the REGION_MAP register of the VPU encoding engine is configured to allocate a higher encoding bit rate to the ROI area than that to the non-ROI area, wherein the QP difference between the ROI area and the non-ROI area is not less than 10.

[0012] Preferably, the interrupt signal generated upon completion of VPU decoding and encoding is bound to the Cortex-A76 core of the RK3588 chip for processing, and the VPU decoding completion interrupt is handled using a threaded IRQ interrupt mechanism.

[0013] A picture-in-picture video processing system based on RK3588, comprising: The video capture module is used to acquire multiple video streams through multiple video input interfaces and simultaneously acquire audio signals; The VPU audio / video codec module, integrated within the RK3588 chip, employs a multi-core heterogeneous design and includes: a decoding engine cluster comprising at least two independent H.264 / H.265 hardware decoders (VDEC) and one VP9 decoder, each with built-in entropy decoding, motion compensation, and inverse transform dedicated circuitry, supporting 4K@60fps decoding; an encoding engine cluster integrating at least one H.264 / H.265 hardware encoder (VENC), with built-in rate-distortion optimization, intra-frame prediction, and dynamic bitrate control modules, supporting 4K@60fps encoding; and a shared memory management unit (MMU) that accesses system memory via a 32-bit DDR controller and the AXI4 bus protocol. The picture-in-picture compositing module is connected to the VPU audio and video codec module and is used to dynamically adjust the layout and compose the decoded main screen video frames and sub-screen video frames into a picture-in-picture format. The real-time display module, connected to the picture-in-picture compositing module, is used to output the composite image to the display interface; The video recording module, which is connected to the VPU audio and video encoding / decoding module and the picture-in-picture synthesis module, is used to encapsulate the encoded video data and audio data into an MP4 file and store it in the local ext4 file system.

[0014] Preferably, the system is configured to run a multi-channel video time-slice scheduling algorithm, dynamically configuring the priority weights of each decoder in the decoding engine cluster through the hardware registers of the VPU, and assigning a higher decoding priority to the main screen than to the sub-screens.

[0015] Preferably, the system further includes a memory management unit based on CMA and ION drivers, used to reserve a continuous physical memory area and directly map the YUV frames output by the VPU decoding engine cluster to the GPU texture memory, thereby achieving zero-copy data transmission from VPU to GPU.

[0016] Preferably, the VPU's encoding engine cluster supports Region of Interest (ROI) encoding technology. The system allocates the main screen area as an ROI area by configuring the encoder's REGION_MAP register, and allocates an encoding bit rate to the ROI area that is at least 3 times higher than that to the non-ROI area. The QP value of the ROI area is lower than that of the non-ROI area, and the difference in QP values ​​is not less than 10.

[0017] Preferably, the system further includes a power management unit, which is used to dynamically adjust the operating voltage and frequency of the VPU according to the currently processed video resolution, and trigger a resolution reduction encoding strategy when the power consumption exceeds the limit.

[0018] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: 1. Significantly improved encoding efficiency and enhanced high-specification video processing capabilities. Leveraging the dual-pipeline parallel design of the VPU encoding engine cluster, one pipeline processes intra-frame prediction for the current frame, while the other synchronously processes entropy coding of the previous frame. Precise synchronization is achieved through the ENC_OVERLAP flag, resulting in a significant improvement in encoding throughput compared to traditional single-pipeline solutions. Combined with hardware-implemented rate-distortion optimization and dynamic bitrate control modules, it stably supports high-frame-rate encoding of high-definition resolution video. Even when processing high-bitrate frames from multiple video streams, it maintains a stable encoding frame rate without frame drops or stuttering, fully meeting the real-time encoding requirements of high-definition picture-in-picture scenarios.

[0019] 2. A precise balance is achieved between image quality and bandwidth, optimizing resource utilization. By employing a layered encoding control strategy, the composite image is divided into Regions of Interest (ROI) and non-ROI regions. A higher encoding bitrate is allocated to the ROI region where the main image is located, while the quantization parameter differences between the two regions are configured through the VPU's REGION_MAP register. This design ensures both the clarity and detail reproduction of the main image and reduces bandwidth consumption in non-core areas, significantly saving overall video transmission or storage bandwidth. It achieves an optimal balance between image quality and cost within limited bandwidth or storage resources.

[0020] 3. System load and power consumption are significantly reduced, making it suitable for deployment in multiple scenarios. The RK3588 chip's built-in VPU enables end-to-end hardware encoding and decoding of audio and video, replacing traditional CPU-dependent software processing. This significantly reduces CPU resource consumption, freeing up the CPU for other core functions such as system control and interactive response. Simultaneously, the low-power characteristics of the VPU hardware encoding and decoding, combined with the efficient memory scheduling of the shared memory management unit, reduce power consumption during data transfer. The device's operating power consumption is significantly lower than software encoding and decoding solutions, making it suitable for both fixed devices powered by mains electricity and meeting the low-power battery life requirements of portable terminals.

[0021] 4. Improved system stability and compatibility, and reduced deployment costs. The VPU audio and video codec module is deeply adapted to the hardware architecture of the RK3588 chip. Its decoding engine supports multiple mainstream video formats, and its audio encoding supports AAC format, ensuring compatibility with video streams and audio signals from various sources without requiring additional format conversion modules. Simultaneously, hardware-level encoding and decoding processing and resource scheduling reduce the risk of software-level anomalies, resulting in stable and reliable long-term system operation. Its stability is significantly superior to traditional software solutions, reducing maintenance costs after equipment deployment.

[0022] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating an embodiment of a picture-in-picture video implementation method based on RK3588 according to the present invention. Figure 2This is a schematic diagram of an embodiment of a picture-in-picture video processing system based on RK3588 according to the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0026] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] Example 1 like Figure 1 As shown, a picture-in-picture video implementation method based on RK3588 includes the following steps: S1. Audio and video synchronous acquisition: Acquire multiple video streams through the multi-channel video input interface of the video acquisition module, and simultaneously acquire audio signals; S2. Audio and Video Hardware Decoding: The VPU audio and video codec module of the RK3588 chip is enabled to perform hardware decoding on the acquired multi-channel video streams and output YUV format video frames; at the same time, the synchronously acquired audio signals are decoded; a multi-channel video time slice scheduling algorithm is used to perform hardware decoding on the multi-channel video streams, dividing the input video stream into time slices of 10ms / slice according to the frame interval, and assigning different priority weights to the decoding tasks by configuring the hardware registers of the VPU, so that the decoding delay of the main screen video stream is no more than 8ms and the decoding delay of the sub-screen video stream is no more than 12ms.

[0028] Memory optimization steps: Zero-copy memory mapping technology is used to directly store the YUV video frames output by VPU hardware decoding into the reserved memory area of ​​CMA (Contiguous Memory Allocator), and the physical address is mapped to the texture memory of the GPU through ION memory driver for use by the picture-in-picture compositing module, avoiding CPU involvement in data transfer.

[0029] When a bitstream error is detected, the VPU hardware automatically enables the Reference Frame Copy function; the current frame is filled with data from the previous frame to ensure the continuity of the image.

[0030] S3. Picture-in-Picture Compositing: The picture-in-picture compositing module combines the decoded main screen video frames and sub-screen video frames into a composite image. S4. Real-time display: The real-time display module outputs the composite video frames rendered by the GPU to the display interface, and adaptively adjusts the resolution of the output screen according to the display requirements. S5. Audio and Video Encoding: The synthesized image is hardware encoded using the VPU audio and video encoding / decoding module, while the decoded audio signal is AAC encoded. Layered encoding control is employed, identifying the main image area in the synthesized image as a Region of Interest (ROI). The REGION_MAP register of the VPU encoding engine allocates a higher encoding bit rate to the ROI region than to the non-ROI region (ROI:Non-ROI = 3:1). The quantization parameters are dynamically configured through the VPU's REGION_MAP register to ensure that the QP difference between the ROI and non-ROI regions is ≥10.

[0031] The encoder employs a dual pipeline design. Pipeline A processes intra-frame prediction for the current frame, while pipeline B simultaneously processes entropy coding of the previous frame. Pipeline synchronization is controlled by the ENC_OVERLAP flag, improving coding throughput by 20%.

[0032] The interrupt signals generated upon completion of VPU decoding and encoding are bound to the Cortex-A76 core of the RK3588 chip for processing, and the VPU decoding completion interrupt is handled using a threaded IRQ interrupt mechanism.

[0033] S6. Audio and video encapsulation and recording: The video recording module encapsulates the encoded video data and encoded audio data into an MP4 file and stores it.

[0034] Example 2 A picture-in-picture video processing system based on RK3588, such as Figure 2 As shown, it includes: The video capture module is used to acquire multiple video streams through multiple video input interfaces, and simultaneously acquire audio signals. It supports multiple high-definition video input interfaces, including but not limited to HDMI, MIPI CSI, and USB cameras.

[0035] The VPU audio / video codec module, integrated into the RK3588 chip, employs a multi-core heterogeneous design and is deeply adapted to the V4L2 driver framework in the Kylin system, ensuring stable video capture. VPU hardware encoding and decoding reduces CPU load, resulting in an overall power consumption reduction of over 40%. Includes: The decoding engine cluster includes at least two independent H.264 / H.265 hardware decoders (VDEC) and one VP9 decoder. Each decoder has built-in dedicated circuits for entropy decoding (CABAC / CAVLC), motion compensation (MC), and inverse transform (IDCT), supporting 4K@60fps decoding; hardware-accelerated bitstream parsing and pixel reconstruction. The encoding engine cluster integrates at least one set of H.264 / H.265 hardware encoders (VENC), and has built-in rate-distortion optimization module (RDO), intra-prediction module and dynamic bitrate control module (RCU), supporting 4K@60fps encoding; The shared memory management unit (MMU) accesses system memory via a 32-bit DDR controller and the AXI4 bus protocol, achieving a peak transfer bandwidth of 12.8 GB / s. The system is configured to run a multi-channel video time-slice scheduling algorithm, dynamically configuring the priority weights of each decoder in the decoding engine cluster through the VPU's hardware registers, allocating higher decoding priority to the main screen than to sub-screens. By dynamically allocating decoder resources, the system ensures that the main screen decoding latency is ≤8ms and the sub-screen decoding latency is ≤12ms.

[0036] The VPU's encoding engine cluster supports Region of Interest (ROI) encoding technology. The system allocates the main screen area as an ROI area by configuring the encoder's REGION_MAP register, and assigns an encoding bit rate to the ROI area that is at least 3 times higher than that to the non-ROI area. The QP value of the ROI area is lower than that of the non-ROI area, and the difference in QP values ​​is not less than 10.

[0037] The picture-in-picture compositing module, connected to the VPU audio and video codec module, is used to dynamically adjust the layout and composite the decoded main screen video frames and sub-screen video frames. It enables dynamic layout adjustments between the main screen and sub-screens and supports operations such as position adjustment and scaling. The graphical interface uses GPU rendering and is compatible with the Kylin desktop environment.

[0038] The real-time display module, connected to the picture-in-picture compositing module, is used to output the composite image rendered by the GPU to the display interface (such as HDMI, LVDS, eDP), and supports adaptive resolution adjustment; the display latency is controlled within 50ms to ensure a real-time interactive experience.

[0039] The video recording module, connected to the VPU audio / video encoding / decoding module and the picture-in-picture compositing module, is used to encapsulate the encoded video and audio data into MP4 files and store them in the local ext4 file system.

[0040] The system also includes: The memory management unit, based on CMA and ION drivers, is used to reserve contiguous physical memory areas and directly map the YUV frames output by the VPU decoding engine cluster to the GPU texture memory, thereby achieving zero-copy data transfer from VPU to GPU.

[0041] The power management unit dynamically adjusts the operating voltage and frequency of the VPU according to the current video resolution being processed (1080p@0.8V / 600MHz, 4K@1.0V / 800MHz), and triggers a resolution reduction encoding strategy when power consumption exceeds the limit.

[0042] The remaining technical features in the above embodiments can be flexibly selected by those skilled in the art to meet different specific practical needs according to actual circumstances. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims. In the above description, numerous specific details have been set forth to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that these specific details are not necessary to implement the present invention. In other instances, to avoid obscuring the present invention, well-known techniques, such as specific construction details, operating conditions, and other technical conditions, have not been specifically described.

[0043] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for implementing picture-in-picture video based on RK3588, characterized in that, Includes the following steps: S1. Audio and video synchronous acquisition: Acquire multiple video streams through the multi-channel video input interface of the video acquisition module, and simultaneously acquire audio signals; S2. Audio and video hardware decoding: The VPU audio and video codec module of the RK3588 chip is enabled to perform hardware decoding on the acquired multi-channel video streams and output YUV format video frames; at the same time, the synchronously acquired audio signals are decoded. S3. Picture-in-Picture Combining: The picture-in-picture combining module combines the decoded main screen video frames and sub-screen video frames into a picture-in-picture composite image. S4. Real-time display: The real-time display module outputs the composite video frames rendered by the GPU to the display interface, and adaptively adjusts the resolution of the output screen according to the display requirements. S5. Audio and video encoding: The synthesized picture is hardware encoded through the VPU audio and video encoding and decoding module, and the decoded audio signal is AAC encoded at the same time. S6. Audio and video encapsulation and recording: The video recording module encapsulates the encoded video data and encoded audio data into an MP4 file and stores it.

2. The picture-in-picture video implementation method based on RK3588 according to claim 1, characterized in that: In step S2, a multi-channel video time-slice scheduling algorithm is used to perform hardware decoding on multiple video streams. The input video stream is divided into time slices according to the frame interval, and different priority weights are assigned to the decoding tasks by configuring the hardware registers of the VPU, so that the decoding delay of the main screen video stream is no more than 8ms and the decoding delay of the sub-screen video stream is no more than 12ms.

3. The picture-in-picture video implementation method based on RK3588 according to claim 2, characterized in that: Between steps S2 and S3, there is also a memory optimization step: using zero-copy memory mapping technology, the YUV video frames output by VPU hardware decoding are directly stored in the continuous memory area reserved by CMA, and the physical address is mapped to the texture memory of the GPU through the ION memory driver for use by the picture-in-picture compositing module.

4. The picture-in-picture video implementation method based on RK3588 according to claim 2, characterized in that: In step S5, layered encoding control is adopted to identify the main screen area in the composite image as the ROI area, and the REGION_MAP register of the VPU encoding engine is configured to allocate a higher encoding bit rate to the ROI area than that to the non-ROI area, wherein the QP difference between the ROI area and the non-ROI area is not less than 10.

5. A picture-in-picture video implementation method based on RK3588 according to claim 2, characterized in that: The interrupt signals generated upon completion of VPU decoding and encoding are bound to the Cortex-A76 core of the RK3588 chip for processing, and the VPU decoding completion interrupt is handled using a threaded IRQ interrupt mechanism.

6. A picture-in-picture video processing system based on RK3588, used to implement the picture-in-picture video implementation method based on RK3588 as described in any one of claims 1-5, characterized in that, include: The video capture module is used to acquire multiple video streams through multiple video input interfaces and simultaneously acquire audio signals; The VPU audio / video codec module, integrated within the RK3588 chip, employs a multi-core heterogeneous design and includes: a decoding engine cluster comprising at least two independent H.264 / H.265 hardware decoders and one VP9 decoder, each decoder incorporating entropy decoding, motion compensation, and inverse transform circuitry, supporting 4K@60fps decoding; an encoding engine cluster integrating at least one H.264 / H.265 hardware encoder, incorporating rate-distortion optimization, intra-frame prediction, and dynamic bitrate control modules, supporting 4K@60fps encoding; and a shared memory management unit that accesses system memory via a 32-bit DDR controller and the AXI4 bus protocol. The picture-in-picture compositing module is connected to the VPU audio and video codec module and is used to dynamically adjust the layout and composite the decoded main screen video frames and sub-screen video frames. The real-time display module, connected to the picture-in-picture compositing module, is used to output the composite image to the display interface; The video recording module, which is connected to the VPU audio and video encoding / decoding module and the picture-in-picture synthesis module, is used to encapsulate the encoded video data and audio data into an MP4 file and store it in the local ext4 file system.

7. A picture-in-picture video processing system based on RK3588 according to claim 6, characterized in that: The system is configured to run a multi-channel video time-slice scheduling algorithm, dynamically configuring the priority weights of each decoder in the decoding engine cluster through the hardware registers of the VPU, and assigning a higher decoding priority to the main screen than to the sub-screens.

8. A picture-in-picture video processing system based on RK3588 according to claim 6, characterized in that: The system also includes a memory management unit based on CMA and ION drivers, which is used to reserve a continuous physical memory area and directly map the YUV frames output by the VPU decoding engine cluster to the GPU texture memory, thereby achieving zero-copy data transmission from VPU to GPU.

9. A picture-in-picture video processing system based on RK3588 according to claim 6, characterized in that: The VPU's encoding engine cluster supports Region of Interest (ROI) encoding technology. The system allocates the main screen area as an ROI area by configuring the encoder's REGION_MAP register, and assigns an encoding bit rate to the ROI area that is at least 3 times higher than that to the non-ROI area. The QP value of the ROI area is lower than that of the non-ROI area, and the difference in QP values ​​is not less than 10.

10. A picture-in-picture video processing system based on RK3588 according to claim 6, characterized in that: The system also includes a power management unit, which dynamically adjusts the operating voltage and frequency of the VPU according to the current video resolution being processed, and triggers a resolution reduction encoding strategy when the power consumption exceeds the limit.

Images