Dual Video Stream GOP Layout for Low-Latency Live Entry
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In live streaming, when a viewer enters a room, starting from a GOP's I-frame causes significant delay due to large GOP lengths, and using two video streams with complete GOP structures increases coding resource consumption and delay, affecting viewer experience.
Innovation Solution
Generate a first video stream with consecutive GOPs and a second video stream with intra-coded and predictive-coded IP frame groups spaced apart by a specific number of frames, ensuring each GOP in the first stream corresponds to multiple IP frame groups in the second stream, with aligned decode timestamps and matching P-frame counts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If starting to push live streaming pictures from the first frame of a GOP when a viewer enters the room, then the terminal device can properly decode and play the pictures, but the live streaming delay increases significantly when GOP length is large
Solution Approach 1:
The patent divides the video stream into two separate streams: a first video stream with complete GOP structures for reliable decoding, and a second video stream with only IP frame groups (intra-coded and predictive-coded frames) spaced at intervals. This segmentation allows the system to provide both reliable playback and reduced delay by using the second stream's compressed frame intervals.
Solution Approach 2:
The patent extracts only the necessary IP frame groups from the complete GOP structure and creates a second video stream containing only these extracted frames spaced at intervals. This extraction removes unnecessary complete GOP structures while retaining the essential frames needed for playback, thereby reducing delay without sacrificing decoding reliability.
2Loss of time
If using two video streams with complete GOP structures to reduce delay, then live streaming pictures can be pushed faster, but coding resource consumption increases and video code rate increases
Solution Approach 1:
The patent applies partial action by creating a second video stream that includes only the necessary IP frame groups rather than complete GOP structures. This partial stream provides sufficient information for reduced delay playback without the excessive coding resources required for two complete streams, optimizing the balance between delay reduction and resource consumption.
3Reliability
If using two video streams with complete GOP structures, then decoding reliability is improved, but video code rate at delivery increases affecting viewer experience
Solution Approach 1:
The patent segments the video delivery into two streams with different characteristics: the first stream provides complete GOP structures for reliable decoding, while the second stream provides spaced IP frame groups at lower code rate. This segmentation allows the system to maintain decoding reliability through the first stream while reducing overall code rate through the second stream's efficient frame spacing.
Data Source
Figure 1
Figure 2~3
Figure 4a
AI summary
Embodiments of the present disclosure provide a video processing method and apparatus, an electronic device, and a storage medium. The method includes: generating a first video stream and a second video stream based on a video captured by a first terminal device, where the first video stream includes a plurality of consecutive GOPs, and the second video stream includes a plurality of intra coded picture and predictive coded picture IP frame groups spaced apart at an interval of a specific number of video frames; each GOP of the first video stream corresponds to one or more IP frame groups of the second video stream; each IP frame group includes an intra coded picture I-frame and a predictive coded picture P-frame after the I-frame; each video frame in each IP frame group has same picture content as a video frame in the first video stream that has a same decode time stamp DTS; and a number of the P-frames included in each IP frame group is the same as a number of reference frames for P-frames in the first video stream.