Occlusion-Aware Image Data Signals for Dynamic Scene Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently generating and processing image data signals for dynamic real-world scenes, particularly in extended reality applications, due to high data rates and redundancy, which limits user interaction and immersion.
Innovation Solution
An apparatus and method for generating an image data signal that segments objects in multiple images, selects subsets based on occlusion measures, and generates an image data signal comprising these segments, reducing redundant data while maintaining high image quality and flexibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all captured view images are transmitted to provide complete scene representation, then image quality and completeness are improved, but data rate increases substantially
Solution Approach 1:
The patent segments the scene into multiple object instances, each with its own image data and metadata. This allows selective transmission of only necessary object segments rather than complete images, reducing data rate while maintaining scene representation quality.
Solution Approach 2:
The patent extracts and transmits only essential object information (image data, bounding box, depth, occlusion status) rather than complete images. This extraction approach removes redundant data while preserving critical scene elements needed for viewpoint rendering.
2Adaptability or versatility
If multiple images from different capture positions are provided to enable viewpoint changes, then viewer interaction and immersion are improved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary processing during capture to organize image data by object instances and assign metadata (bounding boxes, depth values, occlusion status). This pre-organization enables efficient runtime rendering without complex real-time processing of multiple complete images.
Solution Approach 2:
The patent applies different processing and representation methods to different objects based on their properties. Each object instance is handled individually with its own image data and metadata, allowing optimized processing for each object rather than uniform processing of complete images.
3Loss of information
If object image segments from multiple views are included to represent occluded objects, then scene completeness is improved, but data rate increases
Solution Approach 1:
The patent includes occlusion status information for objects that may be partially or fully occluded. This partial information approach provides sufficient data for viewpoint rendering without including complete image data for all possible object states, balancing completeness with data efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus comprises a receiver (201) receiving images of a scene from different view poses of the scene. A segmenter (203) determines object image segments in the images which represent an object. A selector (205) selects a subset of object image segments and a generator (207) generates the image data signal to comprise the subset of object image segments. An occlusion determiner (211) determines an occlusion measure for the object image segments which indicates a degree of occlusion of an object represented by the object image segment and the selector (205) selects object image segments for the subset dependent on the occlusion measures for the set of object image segments. A rendering apparatus may receive the image data signal and generate views of the scene from the image data signal.