Picture-in-Picture Bitstream Encoding With Aligned Independent Subpictures

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing multimedia systems lack efficient methods for encoding, decoding, and displaying picture-in-picture content, particularly in handling independently encoded subpictures with consistent resolution and alignment.

Innovation Solution

An apparatus and method for generating an encapsulated file with separate tracks for independently encoded subpictures, including information on the picture-in-picture relationship and data unit replacements, using unique group IDs and extract and merge sample groups to align and combine subpictures.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If independently encoded subpictures are used for picture-in-picture, then encoding efficiency and flexibility are improved, but alignment and resolution consistency between subpictures become difficult to maintain

Engineering Contradiction:

Improveencoding efficiencyVSAvoidsubpicture alignment precision

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The video picture is divided into independently encoded subpictures, each with its own bitstream. This segmentation allows independent encoding and processing of different picture regions, improving encoding efficiency and flexibility while enabling picture-in-picture functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Sample group descriptions are generated in advance that specify the spatial location, size, and timing information of each subpicture. This preliminary action establishes the alignment framework before actual picture display, ensuring consistent resolution and positioning without requiring complex real-time adjustment.

Inventive Principle:

Principle #10Preliminary action

2Adaptability or versatility

If separate tracks are used for different subpictures, then independent processing and encoding are enabled, but complexity of managing and synchronizing multiple streams increases

Engineering Contradiction:

Improveindependent processing capabilityVSAvoidstream management complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Multiple independently encoded subpicture bitstreams are merged into a single encapsulated file with a single media track. This combining approach maintains the independence of each subpicture for processing purposes while simplifying the overall stream management by using a single container and synchronization mechanism.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

Sample group descriptions act as intermediary metadata that bridge the independently encoded subpictures. These descriptions provide the necessary timing, spatial, and synchronization information that allows the media player to correctly assemble and display the separate subpicture streams without complex coordination.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If subpictures with different resolutions are encoded, then content flexibility is improved, but alignment and visual consistency in picture-in-picture display become problematic

Engineering Contradiction:

Improvecontent flexibilityVSAvoidvisual consistency

Core Design Contradiction:

Adaptability or versatilityVSStability of the object's composition

Solution Approach 1:

Each subpicture is encoded with its own resolution and quality parameters suited to its specific content requirements. This local quality approach allows different subpictures to have different resolutions while maintaining flexibility in content representation.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

Sample group descriptions are prepared in advance to specify the exact spatial location, size, and timing of each subpicture within the overall picture. This preliminary specification ensures that even when subpictures have different resolutions, they align correctly during display through precise positioning information.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250254403A1Method and apparatus for encoding, decoding, or displaying picture-in-picture

Publication Date: 2025.08.07 NOKIA TECHNOLOGIES OY

AI summary

Various embodiments provide example apparatus, method, and computer program product. An example apparatus includes: receiving or generating a first encoded bitstream comprising independently encoded subpictures; receiving or generating a second encoded bitstream comprising independently encoded subpictures; wherein resolution of subpictures in the second encoded bitstream is same or substantially same as resolution of corresponding subpictures in the first encoded bitstream; generating an encapsulated file with a first track and second track, the first track comprises the first encoded bitstream comprising the independently encoded subpicture, the second track comprises the second encoded bitstream comprising the independently encoded subpictures; and wherein generating the encapsulated file comprises including following information in the encapsulated file: a picture-in-picture relationship between the first track and second track; and data units independently encoded subpictures in the first encoded bitstream that are to be replaced by data units of the independently encoded subpictures of the second encoded bitstream.