Light Field Telepresence Pose Detection and Image Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conferencing technologies, including telepresence systems, lack immersive experiences that replicate the trust and empathy of face-to-face interactions, often resulting in suboptimal communication due to limitations in capturing and displaying life-like images with real-time responsiveness and accurate geometry.
Innovation Solution
The development of telepresence devices equipped with a light field camera array and a controller that determines the pose of one device and captures images through a display device, allowing for the selection and transmission of image data to another device to create a more immersive experience by preserving eye gaze and motion parallax, and providing dynamic, life-like geometry.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional video conferencing systems are used, then device complexity is low, but immersion and life-like interaction experience are insufficient
Solution Approach 1:
The system segments the imaging function into multiple cameras arranged in an array, with each camera capturing images from a different viewpoint through the display device. This segmentation enables light field capture and immersive experience while managing complexity through modular camera units
Solution Approach 2:
The patent transitions from conventional 2D video conferencing to 4D light field imaging by adding two spatial dimensions through multiple viewpoint cameras. This dimensional expansion creates immersive experiences with motion parallax and accurate geometry without proportionally increasing overall system complexity
2Measurement precision
If a single camera is used, then device complexity is low, but accuracy in capturing life-like images with real-time responsiveness is insufficient
Solution Approach 1:
The imaging system is segmented into multiple cameras, each capturing images from different viewpoints. This segmentation improves measurement precision by providing multiple perspectives for accurate geometry and motion parallax while maintaining manageable complexity through modular design
Solution Approach 2:
Each camera in the array serves multiple functions: capturing images from different viewpoints, enabling light field reconstruction, providing motion parallax, and supporting real-time responsiveness. This multi-functionality improves image capture accuracy without proportionally increasing device complexity
3Adaptability or versatility
If multiple cameras are used to capture images from different viewpoints, then immersive experience with motion parallax is improved, but data processing complexity increases
Solution Approach 1:
The system performs preliminary action by capturing images from multiple viewpoints simultaneously during a camera capture period, storing them as first image data. This preliminary capture of all viewpoint data enables subsequent selective retrieval and processing based on detected pose, reducing real-time processing complexity while maintaining immersive experience
Data Source
AI summary
Techniques in connection with a first telepresence device including a display device and cameras arranged to capture images through the display device are disclosed, involving determining a pose of the device, determining a capture point of view for a second telepresence device based on at least the detected pose, obtaining first image data based on images captured by the cameras during a first camera capture period, selecting second image data from the first image data based on the determined capture point of view, and transmitting the selected second image data from the first telepresence device to the second telepresence device.