Video Background Segmentation for Virtual Replacement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing videoconference systems face issues with unpleasant background components in video frames, including repetitive background objects, geometric distortion, and visible body parts of other participants, which detract from the viewing experience.
Innovation Solution
The implementation of image segmentation algorithms by endpoint devices to identify and separate background segments from foreground subjects in video frames, followed by the modification of these background segments by blurring or replacing them with virtual backgrounds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If image segmentation is applied to remove background segments, then the viewing experience is improved by removing distracting elements, but the processing time and computational complexity increase
Solution Approach 1:
The patent applies image segmentation to divide the video frame into distinct segments: foreground subjects and background elements. This segmentation enables selective processing where only the background segments are removed or blurred while preserving the foreground, thereby reducing the effective processing area and time compared to processing the entire frame.
Solution Approach 2:
The patent extracts and removes unwanted background segments from the video frames. By identifying and extracting only the harmful background elements rather than processing the entire image, the system reduces computational time while maintaining the improvement in viewing experience.
2Object-affected harmful factors
If background segments are replaced with virtual backgrounds, then the viewing experience is improved, but the device complexity increases
Solution Approach 1:
The patent uses virtual backgrounds that are pre-generated copies or templates. Instead of creating complex real-time backgrounds, the system replaces detected background segments with pre-prepared virtual background images, thereby reducing the computational complexity required during live video processing.
Solution Approach 2:
The patent introduces virtual backgrounds as an intermediary element between the original background and the desired output. This intermediary approach simplifies the system by using pre-processed background replacements rather than complex real-time background generation or manipulation algorithms.
3Manufacturing precision
If image segmentation algorithms are implemented, then background separation is achieved, but the manufacturing complexity of the system increases
Solution Approach 1:
The patent employs image segmentation algorithms that operate by changing parameters such as color thresholds, edge detection sensitivity, and region-based classification criteria. By adjusting these parameters rather than using complex multi-stage algorithms, the system achieves accurate background separation with moderate computational complexity.
Data Source
AI summary
A videoconference system acquires images during a videoconference. A still image may be analyzed to identify persons participating in the videoconference. A first frame is generated for each person and represents a subsection of an image that focuses on the face of that person. These first frames may include unpleasant components. Background segmentation is performed on each first frame to remove or diminish such unpleasant components. For example, a second frame for a person may be generated by identifying the background segments in the first frame and blurring or replacing such background segments with a virtual background. When the second frames are put together, the differences in the first frames are obscured making it more pleasing to view. A composite stream of the second frames is generated, per layout rules, and sent to the far side for improved viewing of the participants at the far side.