Interactive Shared Surface Using Projector and Depth Camera
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote collaborators face challenges in working together in full color on arbitrary surfaces in a natural way, as existing technologies require specialized and instrumented surfaces for sharing and interacting with documents or whiteboards.
Innovation Solution
The interactive and shared surface technique employs a hardware setup that includes a projector, RGB camera, depth camera, and optionally a microphone, allowing any surface to be projected upon, capturing video and depth information while preventing visual feedback, thus making any surface interactive and shared.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If specialized and instrumented surfaces are used for remote collaboration, then interaction capability is improved, but device complexity and cost increase
Solution Approach 1:
The system makes any surface interactive by projecting content onto arbitrary surfaces (whiteboards, walls, tables) and using standard cameras and projectors instead of requiring specialized instrumented surfaces. This universal approach allows collaboration on diverse surfaces without modifying them, resolving the contradiction between interaction capability and device complexity
Solution Approach 2:
The system captures video of the physical surface and objects on it, then projects this video feed back onto the surface or displays it remotely. This copying approach creates an interactive layer over any surface without requiring the surface itself to be instrumented, maintaining simplicity while enabling rich interaction
2Adaptability or versatility
If video feedback is transmitted between locations, then collaboration sharing is improved, but visual echo and latency increase
Solution Approach 1:
The system extracts and removes the projected content from the video feed before transmitting it to remote locations. By separating the projected content from the captured video, the system prevents visual echo where projected images would be captured and re-transmitted, creating harmful feedback loops
Solution Approach 2:
The system implements controlled feedback by projecting video content onto surfaces and capturing the interaction, then selectively transmitting only the necessary portions to remote locations. The feedback loop is managed through careful timing and content selection to minimize latency and avoid echo effects
Data Source
AI summary
The interactive and shared surface technique described herein employs hardware that can project on any surface, capture color video of that surface, and get depth information of and above the surface while preventing visual feedback (also known as video feedback, video echo, or visual echo). The technique provides N-way sharing of a surface using video compositing. It also provides for automatic calibration of hardware components, including calibration of any projector, RGB camera, depth camera and any microphones employed by the technique. The technique provides object manipulation with physical, visual, audio, and hover gestures and interaction between digital objects displayed on the surface and physical objects placed on or above the surface. It can capture and scan the surface in a manner that captures or scans exactly what the user sees, which includes both local and remote objects, drawings, annotations, hands, and so forth.