Live 3D Scene Reconstruction With Parallel Gaussian Splatting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for live 3D reconstruction of dynamic scenes in sports and events face challenges with real-time processing, heavy resource consumption, and motion-induced artifacts, limiting interactive, photorealistic 3D experiences in broadcasting.

Innovation Solution

A two-level parallel computation strategy is employed, processing multi-view video streams across distributed GPUs, distinguishing between dynamic and static elements, and using tailored reconstruction methods for efficient, real-time 3D reconstruction and rendering.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If standard NeRF is used for novel-view synthesis, then visual quality is improved, but rendering speed becomes too slow for real-time use

Engineering Contradiction:

Improvevisual qualityVSAvoidrendering speed

Core Design Contradiction:

Manufacturing precisionVSSpeed

Solution Approach 1:

The patent changes the fundamental parameters of the representation model from NeRF's implicit neural fields to explicit 3D Gaussian splats with closed-form rendering. This parameter transformation enables real-time rendering by replacing iterative ray-marching with direct Gaussian composition, achieving both photorealism and interactivity

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent substitutes the complex iterative optimization mechanism of NeRF with a simpler explicit Gaussian splatting approach. The rendering mechanism changes from computing neural network outputs along rays to composing pre-computed Gaussian primitives, dramatically reducing computational complexity while maintaining visual quality

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Speed

If 3DGS is used for dynamic scenes, then rendering speed is improved, but memory and computation resources are inflated

Engineering Contradiction:

Improverendering speedVSAvoidcomputation resource consumption

Core Design Contradiction:

SpeedVSUse of energy by moving object

Solution Approach 1:

The patent segments the scene into dynamic elements (people, objects) and static background regions, applying different reconstruction and rendering strategies to each. Dynamic elements use full 4D Gaussian modeling while static regions use optimized representations, reducing overall computational burden while maintaining performance where needed

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies full 3DGS reconstruction only to necessary dynamic regions rather than the entire scene. By identifying and isolating moving elements, the system performs intensive computation only where required, avoiding redundant processing of static background areas and reducing overall resource consumption

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If 3DGS is used without explicit motion priors, then reconstruction flexibility is improved, but optimization stalls and quality degrades during abrupt motion

Engineering Contradiction:

Improvereconstruction flexibilityVSAvoidoptimization stability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements feedback mechanisms that monitor optimization progress and adjust reconstruction parameters in real-time. When abrupt motion is detected, the system adapts the optimization strategy by adjusting Gaussian splat parameters, merging rates, and computational priorities to maintain stable optimization while preserving reconstruction flexibility

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent makes the reconstruction system dynamic by allowing adaptive adjustment of modeling parameters based on scene conditions. The system transitions between different reconstruction modes (e.g., geometric accuracy vs. temporal consistency) depending on motion characteristics, enabling reliable performance across varying dynamic conditions

Inventive Principle:

Principle #15Dynamics

4Productivity

If multi-view video streams are processed in real-time, then interactivity is improved, but processing latency increases

Engineering Contradiction:

Improvereal-time processing capabilityVSAvoidprocessing latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent performs preliminary actions by pre-processing and pre-optimizing 3D Gaussian splats during off-peak times or using cached data from previous frames. The system maintains a library of pre-computed geometric and appearance data that can be quickly assembled for new viewpoints, reducing real-time computation requirements and latency

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250384627A1System and method for real-time three-dimensional reconstruction and streaming of sports events and concerts

Publication Date: 2025.12.18 CARNEGIE MELLON UNIV

AI summary

The invention comprises embodiments of a system and method for real-time, three-dimensional reconstruction of dynamic, human-centered scenes from multi-view video streams, leveraging a two-level parallel computation strategy to efficiently reconstruct multiple frames and multiple dynamic elements simultaneously.