Textured 3D Model Updates for Virtual Viewpoint Animations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video broadcasts of live events are limited by camera positions, often failing to capture the most important actions due to occlusions, such as one player blocking another, which restricts the viewer's perspective and understanding of the event.
Innovation Solution
A system and method that updates a textured 3D model of an event using multiple camera images to create virtual viewpoints not achievable by a single camera, allowing for the depiction of actions from different angles and removing occlusions by detecting and repairing occluded lines and objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If cameras are positioned to capture important actions, then the action can be captured, but safety and practical considerations prevent optimal positioning
Solution Approach 1:
The patent creates a virtual copy of the camera viewpoint by generating a textured 3D model from camera images. This virtual model allows the system to simulate viewpoints that would be unsafe or impractical to achieve with physical cameras, while maintaining the ability to capture action quality.
Solution Approach 2:
The patent transitions from 2D camera images to a 3D textured model, adding the dimension of depth and spatial understanding. This enables the system to reconstruct scenes from multiple virtual viewpoints, including those that would be inaccessible to physical cameras due to safety or practical constraints.
2Adaptability or versatility
If multiple camera images are used to create virtual viewpoints, then comprehensive viewing is achieved, but processing complexity increases
Solution Approach 1:
The patent performs preliminary processing by creating a textured 3D model from multiple camera images before generating virtual viewpoints. This pre-processing step consolidates the complexity into a single model structure, allowing multiple viewpoints to be generated efficiently from the pre-computed 3D representation rather than processing raw images for each viewpoint.
Solution Approach 2:
The textured 3D model serves multiple functions: it represents the scene geometry, stores texture information from multiple camera views, and enables generation of multiple virtual viewpoints. This multi-functionality reduces the need for separate processing pipelines for each viewpoint, managing complexity while achieving versatility.
3Loss of information
If occluded objects are detected and repaired, then complete action understanding is achieved, but detection and processing time increases
Solution Approach 1:
The textured 3D model acts as an intermediary that stores complete scene information including occluded objects. By pre-computing the 3D model with all objects properly positioned and textured, the system can quickly retrieve and display complete action information without performing time-consuming detection and repair operations for each virtual viewpoint generation.
Data Source
AI summary
Image data of an event is provided by updating a textured 3d model of the event. For example, in a sporting event, a model of a stadium can be periodically updated to reflect changes over time in lighting, advertisements, number of spectators in the stands and so forth. Different virtual viewpoints of the event can be depicted in an animation using the textured 3d model and image data from objects at the event such as participants in the sporting event. The same image from which object data is obtained can also be used to update the textured 3d model so that the model is current in the animation, resulting in greater realism. The updating can be based on an operator command or automatic detection of a specified event, such as change in lighting or passage of time. The animation can be provided in a broadcast television signal.