Virtual Viewpoint Synthesis for Live Event Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video broadcasts of live events are limited by the fixed viewpoints of cameras, which can miss important action due to occlusions and fail to provide the most critical perspectives, hindering viewer understanding and engagement.
Innovation Solution
A system and method that allows users to view live events from different virtual viewpoints by receiving video data and auxiliary data identifying suitable camera images, enabling manipulation of virtual viewpoints and displaying textured 3D models combined with camera-capture images to depict these viewpoints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cameras are positioned to capture action from desired viewpoints, then image quality and coverage are improved, but safety and practical constraints prevent optimal positioning
Solution Approach 1:
The patent creates virtual copies of camera viewpoints by synthesizing images from multiple real camera feeds. Instead of positioning physical cameras in dangerous or restricted locations, the system generates virtual camera positions that can view from any angle by combining and processing existing camera images, thus achieving desired viewpoints without exposing personnel to hazards.
Solution Approach 2:
The system introduces computer-generated virtual viewpoints as an intermediary between the physical camera system and the desired viewing perspective. This virtual intermediary layer allows the system to overcome the physical limitations of real cameras by mathematically reconstructing perspectives that would be impossible or dangerous to capture directly.
2Reliability
If cameras are positioned in safe locations, then operator safety is improved, but important action may be occluded or missed
Solution Approach 1:
The system segments the viewing problem by using multiple cameras positioned in safe locations, each capturing a specific portion of the event. By processing these segmented views through virtual viewpoint synthesis, the system reconstructs complete information about occluded actions without requiring any single camera to be in a dangerous position to capture everything directly.
Solution Approach 2:
The patent creates virtual copies of the event scene from multiple perspectives by synthesizing images from safe camera positions. These virtual copies reveal occluded actions and provide complete information about the event from any viewpoint, overcoming the information loss that would occur with single-camera perspectives while maintaining operator safety.
3Adaptability or versatility
If multiple camera images are processed to create virtual viewpoints, then comprehensive viewing is improved, but system complexity increases
Solution Approach 1:
The system implements a universal processing framework that handles multiple camera images and generates various virtual viewpoints using the same core algorithms. This multi-functional approach allows the system to adapt to different camera configurations, event types, and viewing requirements without requiring separate specialized processing systems for each scenario, thus managing complexity through generalization.
Solution Approach 2:
The patent manages system complexity by parameterizing the virtual viewpoint generation process. Instead of creating entirely new processing pipelines for each viewpoint, the system adjusts parameters such as camera position, angle, and synthesis strength to generate different virtual perspectives from the same base camera images, reducing overall system complexity while maintaining high adaptability.
Data Source
AI summary
A user such as a television viewer is provided with the capability to view an event from different virtual viewpoints which differ from the viewpoint of a camera. In one aspect, the user is informed of particular camera images from which a virtual viewpoint can be viewed. For example, a menu interface may provide thumbnail preview images of the one or more particular camera images. In another aspect, the use enters commands to manipulate a virtual viewpoint, such as by rotating around a point, moving a lookout point and zooming in or out. In response, a display is provided which includes a textured 3d model of the event combined with at least one textured object in the event. For instance, the event may be a sporting event and the at least one textured object may be a participant in the sporting event.