Scene Track Snapshots and Deltas for Fast 3D Media Playback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing media file formats struggle with efficient storage, streaming, and processing of time-based media, particularly in handling different content types like audio, images, and scene descriptions, especially in three-dimensional scenes, where resource consumption and discontinuous playback are challenges.

Innovation Solution

A media data format that includes model snapshots and deltas, where snapshots provide self-contained definitions and deltas describe changes differentially, allowing for efficient processing and reduced resource consumption, especially through parallel rendering and merging of samples in separate threads.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If model snapshots are used to represent scene state, then discontinuous playback and fast progression are supported, but resource consumption increases during processing

Engineering Contradiction:

Improveplayback speedVSAvoidprocessing resource consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The scene representation is segmented into two types of entries: snapshots that provide complete scene state for fast seeking and discontinuous playback, and deltas that provide differential updates for efficient continuous processing. This segmentation allows the system to choose the appropriate representation based on the playback scenario.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system dynamically switches between snapshot and delta representations based on playback requirements. During continuous playback, deltas are used to minimize processing resources. During seeking or discontinuous playback, snapshots are used to enable fast progression. This dynamic adaptation resolves the contradiction between playback speed and resource consumption.

Inventive Principle:

Principle #15Dynamics

2Use of energy by moving object

If model deltas are used to describe scene changes, then processing efficiency and resource consumption are improved, but discontinuous playback capability is reduced

Engineering Contradiction:

Improveprocessing resource consumptionVSAvoiddiscontinuous playback capability

Core Design Contradiction:

Use of energy by moving objectVSAdaptability or versatility

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The media data format is designed to be universal by supporting both snapshot and delta entries within the same track. This multi-functionality allows the same track to serve both continuous playback (using deltas for efficiency) and discontinuous playback (using snapshots for fast seeking), resolving the contradiction between resource consumption and adaptability.

Inventive Principle:

Principle #6Universality (Multi-functionality)

3Reliability

If complete scene snapshots are stored for every time interval, then accurate scene representation is achieved, but data storage and processing complexity increases

Engineering Contradiction:

Improvescene representation accuracyVSAvoiddata processing complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

Instead of storing complete scene snapshots for every time interval, the system extracts only the changes from the previous scene state and stores them as deltas. This extraction approach maintains accurate scene representation while significantly reducing data storage requirements and processing complexity, as deltas contain only the minimal information needed to update the scene.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs preliminary action by maintaining a reference to the previous scene state (either a full snapshot or a delta chain). This preliminary establishment of baseline state allows subsequent deltas to be stored compactly, reducing overall data complexity while preserving complete scene representation capability when needed.

Inventive Principle:

Principle #10Preliminary action

4Quantity of substance

If differential scene updates are used instead of complete snapshots, then storage efficiency is improved, but processing reliability for seeking operations decreases

Engineering Contradiction:

Improvedata storage sizeVSAvoidseeking operation reliability

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system changes the representation parameter from complete snapshots to differential deltas for time intervals where continuous playback occurs, reducing storage size. For time intervals where seeking or discontinuous playback may occur, the system maintains snapshot parameters to ensure seeking reliability. This parameter change strategy resolves the contradiction between storage efficiency and seeking reliability.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12573136B2Scene tracks for representing media assets

Publication Date: 2026.03.10 APPLE INC

AI summary

Improved media processing techniques includes media data formats with a “convenience” sample having both a snapshot and a delta description of a scene at the same media time. In other examples, improved media processing techniques include creating a temporary media sample having a merged snapshot or a merged delta in a context of a pre-render thread to improve performance of rendering performed in a context of a separate rendering thread.