XR Shadow Rendering Using Encoded Shadow Data for Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing graphics processing systems face challenges in accurately rendering shadows cast by computer-generated objects onto real-world objects in extended reality environments due to significant latency and interplay between virtual and real-world geometry, leading to inconsistent shadow depiction.
Innovation Solution
A system is implemented where shadow information is generated and encoded into a video data stream, transmitted over a network, and decoded by a second device for rendering and display, utilizing techniques like MPEG transport streams to manage latency and ensure accurate shadow representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If shadow information is generated and transmitted over a network, then shadow rendering accuracy is improved, but latency increases
Solution Approach 1:
Shadow information is generated and encoded into a video data stream in advance before the actual rendering occurs. This preliminary generation allows the shadow data to be prepared and transmitted over the network beforehand, enabling accurate shadow rendering when needed while managing latency through advance preparation
Solution Approach 2:
A video data stream serves as an intermediary carrier to transport shadow information from the generation device to the rendering device. This intermediary approach allows structured transmission of shadow data with proper timing and synchronization, balancing accuracy requirements with network transmission constraints
2Stability of the object's composition
If shadow information is encoded into video data stream and transmitted over network, then shadow consistency between virtual and real objects is improved, but device complexity increases
Solution Approach 1:
The video data stream infrastructure is utilized for dual purposes: transmitting both standard video content and shadow information through the same network channel. This multi-functionality approach maintains shadow consistency without requiring separate dedicated transmission systems, thereby limiting the increase in device complexity
Solution Approach 2:
Shadow information is encoded by modifying parameters within the existing video data stream structure rather than creating a completely separate data transmission system. This parameter-based encoding approach integrates shadow data into the video stream efficiently, maintaining consistency while avoiding excessive system complexity
Data Source
Figure 1
Figure 2
Figure 3
AI summary
For a scene comprising real-world objects and generated (CG) objects, the scene characterized by a scene geometry of a first time, one or more processors can generate, based on the scene geometry, shadow information for each of one or more shadows cast by a CG object of the scene on one or more real-world objects. The processor(s) can render and display a frame of the scene as a function of the shadow information and a scene geometry at a time of the rendering later than the first time. In some examples, the shadow information includes shadow factors and identifiers. Each factor can describe a light attenuation effect of a cast shadow on a real world object surface and can be incorporated into a shading atlas of the scene. Each identifier identifies a real-world object surface of the scene affected by a factor.