3D Media Streaming System Using Key Frame Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies are unable to stream and broadcast all moments of an event live in three dimensions due to the high processing volume required, limiting the ability to view events remotely in full 360-degree detail with current processor powers.
Innovation Solution
A three-dimensional media streaming and broadcasting system utilizing a camera set comprising depth-sensing and color cameras, a computing module, and a cloud-based server that captures and processes depth and color images to create point clouds, allowing for synchronized transmission and rendering of 3D key frames, enabling live 3D viewing of events via virtual reality goggles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If all moments of an event are captured and processed in three dimensions using high-volume processors, then complete 3D live streaming is achieved, but processing power requirements become prohibitively high
Solution Approach 1:
The patent segments the continuous event into discrete key frames that capture important moments. Instead of processing every moment in full 3D, the system identifies and processes only significant events (e.g., goals in sports, key moments in concerts) as key frames, dramatically reducing the total processing volume while maintaining 3D quality for the most important content.
Solution Approach 2:
The patent applies partial action by processing only a subset of moments in full 3D (the key frames) rather than all moments. Between key frames, the system uses 2D video or simplified representations, providing just enough 3D processing to capture essential events without the excessive computational burden of continuous full 3D processing.
2Manufacturing precision
If dozens of 5K 2D cameras are used to capture 360-degree images for photogrammetry, then 3D images of important moments are obtained, but the system complexity and cost increase significantly
Solution Approach 1:
The patent segments the camera system into a modular array of synchronized cameras with defined geometric relationships. Rather than using dozens of complex 5K cameras, the system uses a structured arrangement of cameras where each camera's position and orientation are precisely known, enabling simplified photogrammetric processing through the defined geometric model.
Solution Approach 2:
The patent performs preliminary calibration and setup of the camera array before the event, establishing precise geometric relationships and synchronization parameters in advance. This preliminary action creates a predetermined geometric model that simplifies real-time processing during the event, eliminating the need for complex on-the-fly calibration.
3Manufacturing precision
If point cloud processing is performed for every frame to create 3D images, then complete 3D coverage is achieved, but processing time and computational load become unmanageable
Solution Approach 1:
The patent extracts only the essential 3D information from point cloud data during key moments rather than processing complete point clouds for every frame. The system identifies key frames, extracts critical 3D elements from the camera array during these moments, and uses this extracted information to construct 3D representations only when necessary, dramatically reducing processing time.
Solution Approach 2:
The patent applies partial processing by performing computationally intensive point cloud operations only for key frames rather than all frames. For non-key frames, the system uses lighter processing methods or reuses previous 3D data, providing just enough 3D coverage to maintain immersion without the excessive processing burden of continuous full point cloud reconstruction.
Data Source
AI summary
A three-dimensional media streaming and broadcasting system and method, which enables different media, such as concerts and sports events, to be transferred in a live and three-dimensional manner, and the transferred image to be broadcast and viewed with devices such as virtual reality goggles.