3D Terminal Display for Occluded Body Part Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies for virtual face-to-face communication lack realism, particularly in capturing and displaying users' hidden body parts during drawing or writing on a touch panel.
Innovation Solution
A terminal apparatus that captures images of users, generates 3D models, and supplements invisible body parts using supplementary images, enhancing the realistic feel of communication by displaying a more natural representation of the user.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a standard camera captures images of a user during virtual communication, then the communication function is achieved, but the hidden body parts (e.g., arms, legs) remain invisible, reducing realism
Solution Approach 1:
The patent introduces a supplementary camera as an intermediary device positioned behind the display to capture body parts that are occluded from the main camera's view. This mediator camera captures images of hidden body parts and transmits them to the terminal apparatus, which then synthesizes these images with the main camera footage to generate a complete 3D model, thereby resolving the information loss of hidden body parts.
Solution Approach 2:
The patent transitions from a single-viewpoint 2D image capture to a multi-viewpoint 3D reconstruction system. By positioning the supplementary camera behind the display and using depth information from both cameras, the system creates a three-dimensional model that can represent body parts from multiple spatial dimensions, making previously invisible areas observable.
2Illumination intensity
If a transparent display is used to show 3D models during drawing, then visibility of the display content is improved, but the user's body parts occluding the display become invisible
Solution Approach 1:
The supplementary camera positioned behind the transparent display acts as a mediator that captures the body parts occluding the display from the rear side. These occluded body parts are then processed and integrated with the main camera images to reconstruct complete 3D models, allowing both display visibility and body part visibility to coexist.
Solution Approach 2:
Instead of trying to see occluded body parts from the front (where they are blocked), the system inverts the approach by capturing images from the back side of the display through the supplementary camera. This reverse viewpoint allows observation of body parts that would otherwise be hidden behind the transparent display.
3Loss of information
If multiple cameras are used to capture all body parts, then completeness of 3D model is improved, but device complexity and cost increase
Solution Approach 1:
The patent applies local quality by using a high-resolution main camera for the primary field of view and a supplementary camera only for the specific local area where body parts are occluded. This selective deployment of imaging resources ensures complete body representation while avoiding the need for multiple high-resolution cameras throughout the entire system.
Solution Approach 2:
The system uses partial action by deploying only one supplementary camera positioned strategically behind the display to capture specifically the occluded body parts, rather than using multiple cameras covering all possible angles. This partial coverage approach achieves sufficient completeness for the virtual communication context without excessive device complexity.
Data Source
AI summary
A terminal apparatus includes a communication interface, a display, an imager configured to capture images of a user, and a controller configured to communicate using the communication interface. The controller is configured to receive information for displaying a 3D model representing another user, who uses another terminal apparatus, based on a captured image of the another user and a supplementary image that includes an invisible portion of the captured image, and control the display to display a 3D model in which the invisible portion in a 3D model based on the captured image is supplemented by a 3D model based on the supplementary image.