3D Video Communication System Using Depth-Based Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video communication technologies, such as the chroma key method, are limited in creating a realistic 3D environment for video communication, as they require specific background colors, cannot differentiate multiple layers, and lack depth perception, resulting in a lack of immersion and realism.
Innovation Solution
A method and system for video communication that acquires and synthesizes depth values of local and remote scene contents to create a 3D video mode, allowing for the segmentation of targets and backgrounds, enabling the display of multi-viewpoint images to enhance the sense of reality and depth perception during communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If chroma key method is used to extract foreground target, then foreground target can be segmented from background, but background color requirements are strict and cannot accommodate diverse scenes
Solution Approach 1:
The patent changes the segmentation parameter from color-based (chroma key) to depth-based (depth value thresholding). By using depth information instead of color information, the system can segment foreground targets from backgrounds with any color, eliminating the color restriction while maintaining segmentation accuracy.
Solution Approach 2:
The patent introduces depth value as an intermediary parameter between the foreground target and background. Instead of directly comparing colors, the system uses depth values as a mediator to differentiate foreground objects from background elements, enabling color-independent segmentation.
2Measurement precision
If chroma key method is used for target extraction, then foreground target can be separated from background, but scene content cannot be divided into multiple layers
Solution Approach 1:
The patent segments the scene into multiple layers based on depth values. By thresholding depth information, the system divides the scene content into foreground objects at different distances from the camera, creating a multi-layer structure that preserves spatial relationships and enables selective replacement of specific layers.
Solution Approach 2:
The patent adds the depth dimension to the traditional 2D image processing. By incorporating depth value information, the system transitions from planar 2D segmentation to 3D layered segmentation, enabling differentiation of objects at various depths and creating a hierarchical scene structure.
3Ease of manufacture
If 2D video content replacement is used, then video content can be replaced, but user cannot experience depth of scene and sense of reality is lacking
Solution Approach 1:
The patent transitions from 2D video replacement to 3D video synthesis by incorporating depth information. The system maintains the ease of content replacement while adding the depth dimension, allowing users to perceive spatial relationships and experience the three-dimensional structure of the scene, thereby enhancing the sense of reality.
Solution Approach 2:
The patent applies different quality characteristics to different layers of the scene based on their depth values. Foreground objects at different distances are processed and synthesized with different depth characteristics, creating local variations in quality that correspond to the actual spatial structure, thus enhancing the overall sense of reality.
4Reliability
If stereo video technology is adopted to improve sense of reality, then depth perception can be achieved, but application scope is limited by environment arrangement requirements
Solution Approach 1:
The patent changes the fundamental parameter from environmental arrangement (requiring identical settings for both parties) to depth value information. By using depth-based segmentation and synthesis, the system achieves sense of reality without constraining the physical environment arrangement, allowing flexible application across different settings.
Solution Approach 2:
The patent creates a virtual copy of the remote scene's depth structure and synthesizes it with the local environment. Instead of requiring physical duplication of the environment, the system copies and transfers depth information, enabling the sense of reality to be achieved through computational synthesis rather than physical arrangement.
Data Source
AI summary
A video communication method, device, and system are provided, which relate to the filed of video communications, so as to solve problems that currently scenes of two parties of communication need special arrangement to improve sense of reality and scene contents cannot be displayed in a 3D video mode. Therefore, scenes of two parties of communication do not need special arrangement to improve the sense of reality of users, and the scene contents can be displayed in a 3D video mode. The video communication method, device, and system are applicable to video communication such as common video chat, video telephony, and video conference.