Multiple Viewpoint Image Generation for Video Conference Presence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conference technologies face challenges in aligning participants' lines of sight, require high data transfer speeds, and are costly due to the need for multiple stereoscopic displays, making it difficult to achieve realistic presence and efficient data communication.
Innovation Solution
An information processing device that generates and displays multiple viewpoint images based on a three-dimensional model, reducing data communication volume and cost by using a communicator, multiple viewpoint image generator, and display controller, which adjusts rotation angle information to align perspectives and synthesize background and viewpoint images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple stereoscopic displays are used to enable two-way data communication and improve presence, then the sense of presence and line of sight alignment are improved, but costs increase and device complexity increases
Solution Approach 1:
The patent uses a camera to capture the other party's image and generates a three-dimensional model, creating a virtual copy of the remote participant. This copied image is then displayed on a single display device, eliminating the need for multiple stereoscopic displays while maintaining the ability to align lines of sight through virtual avatar rotation and positioning.
Solution Approach 2:
The patent replaces the mechanical system of multiple physical stereoscopic displays with a computational approach using image processing, three-dimensional modeling, and software-based line of sight alignment. The mechanical complexity of multiple displays is substituted with algorithms that rotate and position virtual avatars to match the other party's gaze direction.
2Reliability
If video is sent between remote conference rooms with high data transfer speed to improve image quality and presence, then the sense of presence is improved, but communication costs increase
Solution Approach 1:
The patent extracts only the essential visual information needed for presence - a single video feed from the other party - and processes it locally through three-dimensional modeling. This eliminates the need to transmit multiple high-bandwidth video streams from multiple cameras, reducing communication costs while maintaining image quality through sophisticated local rendering.
Solution Approach 2:
The patent changes the parameter of data transmission from multiple high-resolution video streams to a single video stream combined with three-dimensional model data. This parameter change reduces the total data volume requiring high-speed communication while maintaining the visual quality and presence effect through client-side rendering of the three-dimensional model.
3Reliability
If the camera is set according to the eye height of the subject to align lines of sight, then line of sight alignment is improved, but adaptability to various situations decreases
Solution Approach 1:
The patent implements dynamic adjustment of the three-dimensional model's orientation and position based on the detected line of sight direction. Instead of fixing the camera at a specific height, the system continuously adapts the virtual avatar's gaze direction to match the other party's eye level, regardless of the camera's physical position. This dynamic adjustment maintains line of sight alignment across various situations including different heights and standing or sitting positions.
Data Source
AI summary
Technology to display images with presence on a display device while reducing data communication volume. An information processing device includes: a communicator; a multiple viewpoint image generator configured to, based on a three-dimensional model of a subject the communicator receives from an external device, generate a display image including a viewpoint image corresponding to the perspective of multiple viewpoints; and a display controller configured to display the display image on a multiple viewpoint display device that displays images corresponding to multiple viewpoints.