Mediated Reality Viewpoint Synthesis via Camera Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current communication technologies fail to provide immersive, real-time, and interactive experiences of real-world events, limiting users to superficial, computer-generated environments that detach them from meaningful interactions with the physical world.
Innovation Solution
The system creates immersive mediated reality environments by using live and computer-produced data to allow virtual participants to control their viewpoint and interact with real-world venues and events in real-time, using data collectors and server systems to generate virtual viewpoints even without corresponding real-world data collectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple expensive fixed position cameras are used to capture real-world events, then the viewer can experience multiple viewpoints, but the system complexity and cost increase significantly
Solution Approach 1:
The patent creates virtual copies of real-world viewpoints by processing images from fewer physical cameras. Virtual viewpoints are generated through computational methods that simulate perspectives without requiring physical cameras at every location, thereby reducing hardware complexity while maintaining viewpoint versatility.
Solution Approach 2:
The system introduces an intermediary computational processing layer between the physical cameras and the viewer. This intermediary uses image processing and virtual viewpoint generation to translate limited physical camera data into multiple virtual perspectives, eliminating the need for expensive fixed-position camera arrays.
2Adaptability or versatility
If post production work is used to create virtual viewpoints, then multiple viewpoints can be provided, but real-time interaction and control are lost
Solution Approach 1:
The system performs preliminary actions by continuously capturing and pre-processing images from physical cameras in real-time. This ongoing preparation ensures that when a viewer requests a virtual viewpoint, the computational work has already been partially done, enabling immediate delivery without post-production delays.
Solution Approach 2:
The patent implements dynamic viewpoint generation where virtual perspectives are created on-demand based on viewer requests. The system dynamically selects and processes appropriate physical camera images to generate the requested virtual viewpoint, allowing real-time interaction and control rather than static pre-produced content.
3Loss of information
If data collectors are placed at every possible viewpoint, then complete coverage is achieved, but the system complexity and cost increase
Solution Approach 1:
The system creates virtual copies of viewpoint data through computational processing. Instead of placing physical data collectors at every location, the system generates virtual viewpoint information by processing images from fewer physical collectors, maintaining complete viewpoint coverage while reducing hardware complexity.
Solution Approach 2:
The patent makes each physical data collector multi-functional by having it serve multiple virtual viewpoints simultaneously. A single physical camera's images are processed to generate multiple virtual perspectives, allowing one data collector to fulfill the role of many, thereby reducing the overall network complexity.
4Ease of operation
If computer-generated virtual environments are used, then viewpoint control is improved, but realism and connection to real-world events are reduced
Solution Approach 1:
The patent merges real-world captured images with virtual viewpoint generation techniques. The system combines authentic photographic data from physical cameras with computational virtual perspective methods, creating experiences that maintain real-world realism while providing computer-controlled viewpoint flexibility.
Solution Approach 2:
The system introduces an intermediary processing layer that transforms real-world images into virtual perspectives without completely generating content from scratch. This intermediary process maintains the authenticity of real-world events while adding viewpoint control, bridging the gap between realism and operability.
Data Source
AI summary
The invention relates to creating real-time, immersive mediated reality environments using real data collected from a physical event or venue. The invention provides a virtual participant with the ability to control their viewpoint and freely explore the venue, in real time by synthesizing virtual data corresponding to a requested virtual viewpoint using real images obtained from data collectors or sources at the venue. By tracking and correlating real and virtual viewpoints of virtual participants, physical objects, and data sources, systems and methods of the invention can create photo-realistic images for perspective views for which there is not physically present data source. Systems and methods of the invention also relate to applying effect objects to enhance the immersive experience including virtual guides, docents, text or audio information, expressive auras, tracking effects, and audio.