Video Transmission Background Model Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video transmission systems face bandwidth limitations and resolution constraints, particularly in telepresence applications, due to the inability to efficiently transmit high-resolution videos using conventional camera hardware, leading to increased costs and reliability issues with mechanical PTZ systems and high bandwidth demands.
Innovation Solution
The method involves initializing a background model from the video scene, encoding it independently, and updating it incrementally, transmitting it at a lower bitrate than the video, and merging it with the video at the receiver to produce enhanced high-resolution videos, while simulating PTZ operations without mechanical constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution video is transmitted using conventional camera hardware, then video quality is improved, but bandwidth requirements increase significantly
Solution Approach 1:
The video is segmented into two independent components: a background model transmitted at low bitrate and foreground updates transmitted at higher bitrate. The background model is initialized from the video scene and updated incrementally, allowing the majority of the video content (background) to be transmitted efficiently at lower quality while preserving the ability to transmit high-resolution foreground details when needed.
Solution Approach 2:
Instead of transmitting the complete high-resolution video stream, the system transmits a compressed background model that replicates the static or slowly changing portions of the scene. This background model is then used to reconstruct the video at the receiver, reducing the amount of data that needs to be transmitted while maintaining acceptable video quality.
2Measurement precision
If mechanical PTZ systems are used to simulate human eye fixation, then resolution limitations are partially overcome, but system cost and complexity increase
Solution Approach 1:
The patent replaces mechanical PTZ systems with a software-based background model that simulates the effect of fixation. Instead of physically moving cameras or lenses to focus on specific areas, the system uses algorithmic processing to create and update a background model, achieving the same functional outcome without mechanical complexity.
Solution Approach 2:
The system creates a virtual copy of the background scene through the background model, which can be dynamically updated and transmitted. This virtual representation replaces the need for physical camera repositioning, allowing multiple users to access different views simultaneously without mechanical constraints.
3Measurement precision
If panoramic cameras with high sensor resolution are used, then resolution requirements are met, but network capacity requirements become impractical
Solution Approach 1:
The system segments the video content into background and foreground components, transmitting the background at low resolution and only transmitting high-resolution foreground updates when necessary. This segmentation allows the system to achieve high effective resolution without requiring continuous transmission of high-resolution panoramic video streams.
Solution Approach 2:
The background model is dynamically updated based on scene changes, allowing the system to adapt to varying content requirements. When the scene is static, only low-bitrate background updates are transmitted. When dynamic content appears, the system can allocate more bandwidth to transmit high-resolution updates, optimizing the balance between resolution and network capacity.
Data Source
AI summary
Systems and methods are provided for alleviating bandwidth limitations of video transmission, enhancing the quality of videos at a receiver, and improving the VR/AR experience. In particular, an improved video transmission and rendering system is provided for generating high-resolution videos. The systems have therein a transmitter and a VR/AR receiver; the transmitter includes an outer encoder and a core encoder, while the receiver includes a core decoder and an outer decoder. The outer encoder is adapted to receive the video from a source and separately output a salient video and an encoded three-dimensional background, and the outer decoder is adapted to merge the background with the salient video thereby producing an augmented video. Also provided is a system that simulates pan-tilt-zoom (PTZ) operations without PTZ hardware. Further provided are methods for video transmission whereby a three-dimensional background model is generated, a background independently encoded, updated incrementally, and the background and the updates transmitted independently from the video.