Screen Reflection Removal via Clean Frame Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing technologies fail to effectively remove visual content representing reflections of screen content from videos, which can be distracting and reveal unintended information, especially when captured by sensors such as those on eyeglasses lenses.
Innovation Solution
The implementation of a reflection removal function that applies a transform function to screen content to generate a masking frame, alters the display rate of screen content to capture clean video frames, detects and replaces reflection content with suitable replacement visual content, or removes tainted video frames, thereby eliminating the reflection from the target video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video recording is performed with screen content displayed, then the video captures the intended scene, but visual content representing screen reflections is inadvertently captured and included in the video
Solution Approach 1:
The system performs preliminary actions by capturing a reference video frame before the screen content is displayed, when the scene is in a known clean state without reflections. This reference frame is stored and later used to identify and remove reflection artifacts from the captured video frames, allowing the system to distinguish between actual scene changes and reflection artifacts.
Solution Approach 2:
The system extracts the screen reflection content from the captured video frames by comparing them against the reference video frame. The difference between the captured frame and reference frame identifies the reflection artifacts, which are then removed or masked from the final video output, separating the harmful reflection content from the intended scene content.
2Ease of operation
If screen content is displayed at normal rate, then the display function is maintained, but reflections of the screen content are captured by the sensor
Solution Approach 1:
The system implements periodic action by intermittently displaying screen content and capturing reference video frames at specific intervals. The screen content is displayed periodically, and reference frames are captured during these periodic intervals when the screen is not displaying content, allowing the system to establish a baseline for reflection removal while maintaining normal display functionality.
3Object-affected harmful factors
If reflection removal processing is applied to the video, then screen reflections are eliminated, but additional processing time and computational resources are required
Solution Approach 1:
The system performs preliminary action by capturing and storing the reference video frame in advance, before reflection removal processing is needed. This pre-captured reference frame is ready for immediate comparison with captured video frames, eliminating the need for complex real-time analysis and reducing the computational time required for reflection removal processing.
Solution Approach 2:
The system uses copying by creating a duplicate of the reference video frame and using this copy for comparison with captured frames. Instead of directly modifying the original reference frame or performing complex calculations, the system copies the reference data and uses it to identify reflection artifacts, simplifying the processing workflow and reducing computational overhead.
Data Source
AI summary
A screen is manipulated to display content whose reflection is not captured by a sensor. In an embodiment, the screen inserts a black frame between screen content frames, displaying the black frame during a particular time. A sensor captures a video frame during the particular time. The video frame does not include a screen reflection and is considered a clean frame. In another embodiment, the screen displays screen content with a particular polarization during a particular time. A sensor captures a video frame with another polarization during the same time. The polarizations are selected such that the sensor is unable to capture screen reflections. The video frame is considered a clean frame. The clean frame is used to generate a masking frame, which is applied to target video frames to remove screen reflection. A modified target video, including the reflection-removed target video frame, and/or the clean frame itself, is generated.