3D Skeletal Model Rendering from Video Feeds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion capture systems for creating photorealistic 3D animations are limited by the need for extensive processing power, high costs, and complex logistics, especially when dealing with large numbers of cameras required for capturing complex scenes, making them slow and expensive for live event production.
Innovation Solution
A method and system for generating and rendering 3D skeletal models from video feeds, allowing for real-time or post-event rendering of 3D photorealistic graphics from any viewpoint, using a processing circuitry and memory to generate skeletal models, determine 3D rigged models, and wrap them with corresponding 3D rigged models, enabling efficient and cost-effective animation of non-rigid objects without the need for extensive camera setups.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If a large number of cameras are used to capture complex scenes for photorealistic 3D animations, then the quality and completeness of the 3D model is improved, but the processing power required, cost, and system complexity increase significantly
Solution Approach 1:
The patent uses a single camera to capture 2D video footage, then creates a digital copy or representation of the scene through skeletal model generation and 3D rendering. Instead of using multiple cameras to physically capture the scene from multiple angles, the system creates a computational copy of the 3D scene from 2D input, significantly reducing hardware complexity while maintaining 3D model quality
Solution Approach 2:
The patent replaces the mechanical system of multiple physical cameras with a computational system consisting of image processing algorithms, skeletal model generation, and 3D rendering software. The mechanical complexity of camera arrays is substituted with digital processing pipelines that generate 3D animations from 2D video feeds
2Measurement precision
If many sensors or markers are placed on objects for motion capture, then the accuracy of motion tracking is improved, but the cost and ease of operation deteriorate
Solution Approach 1:
The patent extracts motion information directly from 2D video footage without requiring physical markers or sensors to be attached to objects. The system extracts skeletal models and motion data through image processing algorithms, removing the need for intrusive marking equipment and simplifying the setup process while maintaining motion tracking accuracy
Solution Approach 2:
The system enables objects to be tracked through their visual appearance in the video feed without requiring them to carry additional equipment. The skeletal model generation algorithm automatically identifies and tracks objects based on their visual characteristics in the 2D video, making the system self-sufficient and eliminating the need for external markers or sensors
3Manufacturing precision
If extensive processing power is allocated to extract dynamic 3D models from multiple cameras, then the quality of photorealistic animation is improved, but the speed and cost of production worsen
Solution Approach 1:
The patent segments the complex task of 3D model extraction into distinct processing stages: 2D skeletal model generation from video frames, 3D rigged model determination, and final 3D animation rendering. This segmentation allows each stage to be optimized independently and enables parallel processing, improving production speed while maintaining animation quality through systematic processing
Data Source
AI summary
A system and method for creating 3D graphics representations from video. The method includes: generating a skeletal model for each of at least one non-rigid object shown in a video feed, wherein the video feed illustrates a sports event in which at least one of the non-rigid objects is moving; determining at least one 3D rigged model for the at least one skeletal model; and rendering the at least one skeletal model as a 3D representation of the sports event, wherein rendering the 3D skeletal model further comprises wrapping each of at least one 3D skeletal model with one of the at least one 3D rigged model, each 3D skeletal model corresponding to one of the at least one skeletal model, wherein each 3D rigged model is moved according to the movement of the respective skeletal model when the 3D skeletal model is wrapped with the 3D rigged model.