Video Overlay Encoding via Temporal Correlation Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for inserting images or advertisements into video streams during encoding are energy-intensive, complex, and require significant CPU resources, leading to synchronization challenges and increased bitrate.
Innovation Solution
A method for processing a current image that involves determining a target pixel area, obtaining a hidden image with a source pixel area, dividing the current image into blocks, and encoding the target pixel area using a motion vector pointing to the source pixel area, thereby eliminating pixel residuals and reducing computational overhead.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional substitution methods are used to insert images into video streams, then image insertion functionality is achieved, but energy consumption increases and device complexity increases
Solution Approach 1:
The patent extracts the image insertion functionality from the main video decoding path by using a separate overlay mechanism. The hidden image is decoded independently and overlaid onto the current image using alpha blending, avoiding the need to modify the main video decoding pipeline. This separation reduces the energy burden on the main decoder while achieving the desired image insertion effect.
Solution Approach 2:
The patent introduces an intermediary alpha channel mechanism to combine the current image and hidden image. Instead of directly substituting image blocks which requires complex synchronization and multiple decoders, the alpha channel acts as a mediator that smoothly blends the two images together, simplifying the processing requirements and reducing energy consumption.
2Ease of manufacture
If multiple video streams are decoded and displayed in parallel, then image insertion is achieved, but device complexity increases and energy consumption increases
Solution Approach 1:
The patent merges the hidden image decoding with the main video decoding process by using the same decoder for both streams. The decoder is configured to decode both the current image and hidden image from the bitstream, and the processing unit combines them using alpha blending. This merging approach avoids the need for separate dedicated decoders for each stream, reducing device complexity.
Solution Approach 2:
The patent makes the video decoder universal by enabling it to handle both main video and hidden image streams simultaneously. The same decoding hardware/software is used for multiple purposes - decoding the primary video content and the overlay image content - thereby reducing the need for multiple specialized decoders and simplifying the overall system architecture.
3Manufacturing precision
If fine frame-by-frame synchronization is implemented, then keying accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary synchronization setup by establishing temporal correspondence between the current image and hidden image at the decoding stage. The decoder is configured to maintain synchronization buffers and timing information in advance, allowing the processing unit to perform alpha blending without requiring complex real-time frame-by-frame alignment during the actual overlay operation.
Solution Approach 2:
The patent uses motion vector copying from the main video stream to the hidden image stream to maintain synchronization. Instead of implementing complex independent synchronization mechanisms, the timing and motion information from the primary video is copied and applied to the hidden image decoding, simplifying the synchronization process while maintaining keying accuracy.
4Ease of manufacture
If substitution of sub-parts of video stream is implemented, then image insertion is achieved, but bitrate increases
Solution Approach 1:
The patent applies local quality optimization by encoding the hidden image at a lower resolution and with reduced quality parameters compared to the main video stream. Since the hidden image is displayed as an overlay and does not require the same level of detail as the primary content, this localized quality reduction significantly decreases the bitrate contribution of the inserted image while maintaining acceptable visual quality.
Solution Approach 2:
The patent uses partial action by selectively applying alpha blending only to specific regions where the hidden image needs to be displayed, rather than processing the entire video frame. The overlay mechanism is activated only for the duration and spatial extent of the inserted image, reducing the overall processing load and associated bitrate overhead compared to full-frame substitution methods.
Data Source
AI summary
A method for processing a current image of a set of images is proposed, which comprises, at a video processing device comprising a processor: determining a target pixel area in the current image; obtaining a hidden image, comprising a source pixel area, of the set of images; dividing the current image into blocks for an encoding of the blocks according to an encoding sequence; and for a block of the target pixel area: determining a motion vector pointing to a block of the source pixel area in the hidden image corresponding to the block of the target pixel area, and based on the motion vector, encoding the block of the target pixel area according to a temporal correlation prediction mode pursuant to which no pixel residual representing a difference between the block of the target pixel area and the corresponding block of the source pixel area is generated.