Video See-Through Reprojection with Generative AI Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video see-through (VST) systems in extended reality (XR) devices face a trade-off between field of view and angular resolution, where a wider field of view results in lower angular resolution, and high-resolution cameras require significant sensor power and computing resources.
Innovation Solution
The system combines image sensor data from a camera system with generative image content generated by an image generation model, allowing the camera system to capture image sensor data with high angular resolution and using the generative content to fill in the peripheral areas, thereby providing a wider perceived field of view without reducing angular resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of moving object
If a wider field of view is used on VST cameras, then an immersive experience is provided, but angular resolution decreases
Solution Approach 1:
The patent segments the field of view into a central region captured by the camera and peripheral regions generated by AI. The camera captures high-resolution central content while AI generates peripheral content, allowing the system to provide a wide field of view without sacrificing central angular resolution.
Solution Approach 2:
The patent changes the parameter composition of the final image by combining real camera data with AI-generated content. This allows the system to maintain high angular resolution in captured regions while expanding the overall field of view through generated content.
2Measurement precision
If high-resolution cameras are used, then angular resolution is maintained, but sensor power and computing resources increase
Solution Approach 1:
The patent applies partial action by using the camera to capture only the central region at high resolution, while using AI to generate the peripheral regions. This reduces the burden on the camera sensor and computing resources compared to capturing the entire wide field of view at high resolution.
Solution Approach 2:
The patent introduces AI-generated content as an intermediary to supplement camera footage. This intermediary provides peripheral visual information without requiring additional camera hardware or increasing sensor power requirements.
3Measurement precision
If high-resolution cameras are used, then angular resolution is maintained, but computing resources increase
Solution Approach 1:
The patent segments the processing workload by having the camera handle only central region capture and using AI to handle peripheral region generation. This segmentation reduces the computing resources required for processing the entire field of view at high resolution.
Solution Approach 2:
The patent uses AI to generate copies of peripheral visual content based on central region data. This copying approach requires fewer computing resources than capturing and processing original high-resolution data for the entire wide field of view.
Data Source
AI summary
A computing device may receive image sensor data from a camera system on the computing device, transmit input data to an image generation model, where the input data includes the image sensor data, receive generative image content from the image generation model, and generate display content by combining the image sensor data and the generative image content.