Whiteboard Text Detection for Focused Video Conference Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video systems fail to effectively display text on whiteboards during presentations, often mistaking other objects for whiteboards or showing entire whiteboards regardless of text presence, leading to illegible text on viewer displays.
Innovation Solution
A system that first detects all text-containing areas within the camera view and then determines whether each is a whiteboard area, using a combination of text detection and deep learning to accurately identify whiteboard text, and controls the video feed to focus on relevant whiteboard areas and active speakers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If the entire whiteboard is displayed in the video feed, then the complete whiteboard area is visible, but the text on the whiteboard becomes too small and illegible on viewer displays
Solution Approach 1:
The system extracts and displays only the text-containing regions from the whiteboard, removing unnecessary blank areas. This is achieved by detecting text locations and generating cropped video feeds that focus specifically on text-containing regions, thereby improving text readability without requiring the entire whiteboard to be visible.
Solution Approach 2:
The whiteboard display is segmented into text-containing regions and non-text regions. The system identifies and separates text-containing areas using text detection algorithms, then displays only these relevant segments in the video feed, allowing viewers to see text clearly while excluding irrelevant portions of the whiteboard.
2Measurement precision
If speaker tracking is used to focus on the speaker, then the speaker is clearly visible, but the whiteboard text is omitted from the view
Solution Approach 1:
The system merges speaker tracking functionality with text detection functionality. By combining these two detection mechanisms, the system can simultaneously identify both the speaker's location and text-containing regions on the whiteboard, then integrate this information to create a composite view that includes both the speaker and relevant whiteboard text in the video feed.
Solution Approach 2:
The system adds a new dimension to camera control by incorporating text location information alongside speaker position. Instead of controlling the view based solely on speaker position (one dimension), the system considers both speaker position and text location (two dimensions), enabling the camera to pan and zoom to views that satisfy both requirements simultaneously.
3Area of stationary object
If conventional whiteboard detection is used to identify the whiteboard location, then the whiteboard area can be displayed, but the detection is inaccurate and may identify non-whiteboard objects as whiteboards
Solution Approach 1:
The system introduces text detection as an intermediary step between whiteboard detection and video feed generation. Instead of directly detecting whiteboards and displaying them, the system first detects text locations, then uses these text locations to identify and display text-containing whiteboard regions. This intermediary approach improves reliability by focusing on the actual content (text) rather than just the whiteboard boundary.
Solution Approach 2:
The system inverts the conventional detection approach by detecting text first and then identifying the whiteboard, rather than detecting the whiteboard first and then searching for text. This inversion improves detection accuracy because text detection is more reliable and can serve as a anchor to identify the actual whiteboard region containing relevant content.
4Measurement precision
If manual whiteboard location definition is used, then accurate whiteboard positioning is achieved, but manual intervention is required which is inconvenient and impractical
Solution Approach 1:
The system performs automatic whiteboard and text detection without requiring manual intervention. Using text detection algorithms and deep learning models, the system automatically identifies text-containing regions and determines whiteboard locations, eliminating the need for manual positioning while maintaining high accuracy through automated content-based detection.
Data Source
Figure 1~2
Figure 3A~3D
Figure 4A~4C
AI summary
Systems and methods are provided for identifying and displaying whiteboard text and/or an active speaker in a video-based presentation, e.g., a video conference. Video images of an environment including a whiteboard may be captured by a video camera system. The video images may be analyzed to detect at least one text-containing area in the environment. Each text-containing area may be analyzed to determine whether it is an area of a whiteboard. When a text-containing area is identified as a whiteboard area, an area of view including the text-containing whiteboard area may be selected for display, e.g., a subset of the full frame captured by the video system. A video feed from the video camera system may be controlled to display the selected area of view at a client device, to provide a useful view of the whiteboard text and/or a speaking person located near the whiteboard text.