Depth Sensor Writing Surface Content Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote users in virtual communication sessions often face limitations in accessing, viewing, or interacting with content on physical writing surfaces like whiteboards or blackboards due to detection and accessibility issues.
Innovation Solution
A communication system that uses depth sensors to detect writing surfaces in a physical space and captures content on these surfaces during virtual communication sessions, processing images and depth data to identify and transmit the content to remote client devices, while also rendering occluding objects transparent to maintain visibility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth sensors and image processing are used to detect writing surfaces, then the ability to identify and capture writing surface content is improved, but the device complexity increases
Solution Approach 1:
The system segments the detection process into multiple independent modules: depth data acquisition from depth sensors, image data acquisition from cameras, surface detection module that processes both data types, criteria evaluation module that checks color/size/shape thresholds, and content capture module. This modular segmentation allows each component to be optimized independently while maintaining overall system accuracy for writing surface detection.
2Loss of information
If the system captures and transmits writing surface content to remote devices, then remote users' ability to view content is improved, but the loss of time for content transmission occurs
Solution Approach 1:
The system performs preliminary actions by continuously monitoring the physical space with depth sensors and cameras even before a communication session fully begins. Writing surfaces are pre-identified and their content is buffered locally, so when a communication session starts, the content is already prepared and can be transmitted immediately to remote devices, eliminating startup delays.
Solution Approach 2:
The system maintains continuous operation of depth sensors and cameras throughout the communication session, continuously tracking writing surfaces and their content changes. This continuous monitoring ensures that any modifications to writing content are captured and transmitted in real-time without interruption or delay, maintaining uninterrupted content accessibility for remote users.
3Loss of information
If objects obscuring the writing surface are detected and rendered transparent, then the visibility of writing surface content is improved, but the processing complexity increases
Solution Approach 1:
The system introduces an intermediary occlusion detection module that sits between the raw image data and the final displayed content. This module uses depth sensor data as an intermediary layer to identify objects in front of the writing surface, then selectively removes only those occluding elements while preserving the underlying writing content. This intermediary approach simplifies the processing by focusing only on occlusion removal rather than complete scene reconstruction.
Data Source
AI summary
A communication system captures writing surface content in a physical space for transmittal to remote client devices participating in a communication session. During a communication session with one or more remote client devices, a communication system captures image data and depth data describing objects in a physical space of the communication system. Based on the captured data, the communication system identifies a writing surface in the physical space and captures content on the writing surface. The communication system may also identify objects occluding content on the writing surface based on the captured data and may modify image data to make an object occluding the content at least partially transparent. The communication system transmits the content to at least one of the remote client devices participating in the communication session.