Video Watermarking via Temporal Prediction Heat Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video watermarking methods face challenges in managing temporal and spatial propagation of watermarks in reference frames, leading to potential degradation of video quality and uncontrollable visual artefacts, especially with advanced entropy coding paradigms like context adaptation and arithmetic coding.
Innovation Solution
A method is proposed to determine a set of modifiable elements in a coded bit-stream by analyzing the bit stream to identify eligible locations for watermarking, constructing a heat map representing pixel usage for temporal prediction, and selecting candidate modifiable elements that fulfill robustness and fidelity criteria with controlled spatial propagation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reference frames are watermarked to increase embedding rate and robustness, then watermark robustness is improved, but temporal propagation causes uncontrollable visual artefacts and video quality degradation
Solution Approach 1:
The patent segments the video stream into reference and non-reference frames, and further segments reference frames into macroblocks. By identifying and isolating non-reference macroblocks (those not used for temporal prediction), the system can safely watermark only specific segments without causing temporal propagation artifacts. This segmentation allows selective watermarking that avoids harmful temporal effects while maintaining robustness.
Solution Approach 2:
The patent applies local quality by computing compression influence values for each macroblock individually. Instead of uniformly watermarking entire frames, the system evaluates each macroblock's influence on temporal prediction and applies watermarking only to regions with acceptable influence levels. This localized approach ensures that watermarking does not propagate uncontrollably through temporal prediction while maintaining embedding rate in safe regions.
2Loss of information
If bit stream modifications are made to insert watermarks, then watermark embedding is achieved, but video quality deteriorates due to uncontrolled temporal propagation
Solution Approach 1:
The patent performs preliminary analysis of the bit stream to identify non-reference macroblocks before watermarking. By pre-computing compression influence values and determining which macroblocks are safe for modification (those not used for temporal prediction), the system avoids subsequent quality degradation. This preliminary identification ensures that watermark modifications are made only in locations that will not propagate errors through the decoding process.
Solution Approach 2:
The patent converts the harmful effect of temporal prediction into a beneficial selection criterion. Instead of treating temporal propagation as merely a problem to be avoided, the system uses the temporal prediction structure to identify safe watermarking locations. Non-reference macroblocks, which by definition do not participate in temporal prediction, become the ideal candidates for watermarking. This approach transforms the temporal prediction mechanism from a source of artifacts into a guide for safe watermark insertion.
3Measurement precision
If compression influence values are computed for each macroblock to identify watermarking locations, then watermark placement accuracy is improved, but computational complexity increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for watermarking decisions from the full bit stream analysis. Instead of computing comprehensive compression influence values for all possible scenarios, the system focuses on identifying whether each macroblock is used for temporal prediction in reference frames. This extraction approach reduces computational complexity by taking out only the critical path information needed to distinguish reference from non-reference macroblocks.
Solution Approach 2:
The patent uses a simplified, computationally inexpensive method to determine macroblock reference status rather than performing expensive full compression influence analysis. By leveraging the existing decode process and motion compensation information already available in the codec, the system obtains sufficient precision for watermarking decisions at minimal additional computational cost. This disposable approach uses readily available data structures to achieve the needed precision without heavy computation.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A method for determining a set of modifiable elements in a group of pictures of a coded bit-stream representative of an audio video content is disclosed. The method comprises determining (S12) a set of a candidate modifiable elements wherein a candidate modifiable element comprises a modified value of the coded bit-stream and a spatial propagation map associated with the modified value, a spatial propagation map (SP6) comprising pixels whose decoding is impacted when the modified value is placed in the coded bit stream ; determining (S14) a heat map for each reference frame, the heat map comprising, for each pixel of the reference frame, an information representative of the usage of said pixel for temporal prediction during the decoding of said part of the coded bit-stream coding a group of pictures; obtaining (S16) a set of modifiable elements among candidate modifiable elements, a modifiable element having a spatial propagation map that does not overlap with corresponding heat map.