Video Telephony System Using 3D Depth Cameras for Dynamic Virtual Backgrounds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video telephony systems face challenges in protecting user privacy and providing a realistic video call experience, especially when the user's location changes, as they struggle with accurate background separation and maintaining a sense of face-to-face interaction due to limitations in 2D and 3D camera technologies.
Innovation Solution
A video telephony system using 3D cameras to capture depth information, allowing for the separation of users from their backgrounds and the generation of virtual backgrounds that change based on the user's location, enhancing the realism of the video call experience by applying parallax effects and depth information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a 2D camera is used to extract user image through color information with a single color background, then user appearance can be extracted, but the user's appearance including background is extracted when illumination condition changes
Solution Approach 1:
The patent replaces the 2D camera's color-based background separation method with a 3D depth camera that uses depth information for separation. This substitution eliminates the reliance on color information that is sensitive to illumination changes, providing reliable and accurate background separation under varying lighting conditions.
2Productivity
If background is learned in advance through differential image and previous frame, then background can be separated, but the background needs to be learned every time it changes according to user's location
Solution Approach 1:
The patent replaces the iterative background learning process with direct depth-based segmentation using a 3D depth camera. This allows the system to immediately and accurately separate user from background without requiring time-consuming learning or re-learning when the user moves to a different location.
3Ease of manufacture
If a 3D depth camera is used to separate user and background by extracting depth information, then background separation is easier, but the contour of the user is not clearly separated
Solution Approach 1:
The patent merges the advantages of both 2D and 3D camera systems by combining color information from a 2D camera with depth information from a 3D depth camera. This fusion allows for easy background separation while achieving precise user contour separation that neither camera type could achieve alone.
4Object-affected harmful factors
If the separated image of the other user and virtual image are combined and displayed, then privacy is protected, but the background of the other user is fixed even if the user moves making it difficult to feel face to face interaction
Solution Approach 1:
The patent makes the virtual background dynamic by adjusting it according to the user's location changes detected by the 3D depth camera. This allows the background to move and change perspective when the user moves, creating a more realistic and immersive face-to-face interaction experience while maintaining privacy protection.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system effectively protects user privacy by substituting real backgrounds with virtual ones and enhances the realism of video calls by dynamically adjusting the background perspective, providing a more immersive face-to-face interaction experience.
Implementation Method 1
as a three-dimensional (3D) depth camera has been introduced in recent years, the background can be easily separated by extracting depth information
Implementation Method 2
The first image display apparatus displays the second image in which a point of view of the background is changed according to a change in the location of the user
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
A video telephony system includes a first image display apparatus and a second image display apparatus which makes a video call with the first image display apparatus. The first image display apparatus transmits a first image including a photographed image of a first user to the second image display apparatus, and receives a second image in which a background of a photographed image of a second user is substituted with a virtual background image, from the second image display apparatus. The second image display apparatus changes the virtual background image of the second image according to a change in a location of the first user and transmits the second image.