Special Effect Predictive Frames for Interactive Video Streaming

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional video streaming technologies face challenges in achieving special effects like freezing and sweeping over the internet due to limited bandwidth and processing overhead, and existing adaptive streaming approaches do not typically provide these effects.

Innovation Solution

The method involves a processor and memory system that decodes video frames using special effect predictive frames from multiple camera streams, allowing for efficient rendering of special effects like frozen moments and sweeping by accessing and decoding frames from a camera array, with primary and secondary special effect predictive frames enabling seamless transitions between video streams.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of energy

If conventional coding schemes are used to compress video information, then bandwidth usage is reduced, but processing overhead increases significantly

Engineering Contradiction:

Improvebandwidth usageVSAvoidprocessing overhead

Core Design Contradiction:

Loss of energyVSDevice complexity

Solution Approach 1:

The patent segments video frames into different types (I-frames, P-frames, SP-frames) with different encoding and decoding complexities. By using P-frames that reference previous frames and SP-frames for special effects that reference both temporal and spatial data, the system reduces overall processing overhead while maintaining compression efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary encoding actions by pre-processing video data into multiple streams with different frame types before transmission. This allows the receiving end to decode only necessary frames based on playback requirements, reducing real-time processing overhead during special effects playback.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If traditional video streaming protocols are used, then compatibility with existing infrastructure is maintained, but special effects like freezing and sweeping cannot be achieved

Engineering Contradiction:

Improvespecial effects capabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent creates a multi-functional video streaming system that can handle both conventional video playback and special effects (freezing, sweeping, bullet time) using the same HTTP-based infrastructure. The system achieves this by encoding multiple video streams with different frame types that can serve different playback needs.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces SP-frames (special effect predictive frames) as an intermediary element between conventional P-frames and special effect rendering. These SP-frames contain predictive information from multiple camera angles and time points, enabling complex special effects without requiring complete re-encoding of the entire video stream.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Manufacturing precision

If multiple camera streams are decoded for special effects, then effect quality is improved, but processing resources are consumed

Engineering Contradiction:

Improveeffect qualityVSAvoidprocessing resources

Core Design Contradiction:

Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent merges multiple camera streams into a unified decoding framework where SP-frames reference data from multiple sources. By combining temporal prediction (from previous frames) and spatial prediction (from other camera angles) into a single decoding process, the system achieves high-quality multi-camera special effects without proportionally increasing processing resources.

Inventive Principle:

Principle #5Merging (Combining)

4Ease of operation

If I-frames are used for independent decoding, then random access is enabled, but bandwidth consumption increases

Engineering Contradiction:

Improverandom access capabilityVSAvoidbandwidth consumption

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The patent applies different frame quality characteristics to different parts of the video stream. I-frames provide complete image data for random access points, while P-frames and SP-frames provide differential data for continuous playback. This local differentiation allows random access capability without transmitting full I-frame data at every access point.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS9667885B2Systems and methods to achieve interactive special effects

Publication Date: 2017.05.30 FUTUREWEI TECHNOLOGIES INC

AI summary

Methods and systems facilitate efficient and effective special effects in video streaming. In one embodiment, an apparatus enables special effects by decoding video frames in accordance with a decoding process. The decoding process includes: accessing a plurality of streams of video (wherein each stream originates from a respective camera of a camera array) and decoding frames from the plurality of streams of video. A first stream from a first camera comprises: a first primary special effect predictive (SP) frame that is coded based upon a predictive (P) frame in the first stream; and a first secondary special effect predictive (SP) frame that is coded based upon a primary special effect predictive (SP) frame included in a second stream from a second camera. The method further comprises rendering a special effect using the primary and secondary special effect predictive (SP) frames.