Screen Content Replacement in Camera Recordings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Media recordings often poorly represent visual content displayed on screens due to interference, mismatched refresh rates, dynamic range issues, and camera quality limitations, leading to suboptimal quality and increased file size.
Innovation Solution
A method and system that replace the camera-recorded visual content on screens with an original, higher-quality version, avoiding the 'digital-to-light-to-digital' conversion step by accessing and overlaying the original visual content using coordinates determined from the camera recording, thereby enhancing the media recording without increasing camera recording quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the quality of the camera recording is increased (e.g., by increasing recording resolution, framerate and/or video quality), then the quality of visual content on screen is improved, but the file size increases leading to larger bandwidth and storage requirements
Solution Approach 1:
The system segments the camera recording into different regions, identifying the screen area separately from the rest of the scene. This allows selective processing where only the screen region is replaced with original digital content, while the rest of the camera recording remains unchanged. This segmentation approach enables quality improvement of screen content without requiring the entire recording to be high-resolution.
Solution Approach 2:
The system extracts the visual content displayed on the screen from the camera recording by identifying screen coordinates and cropping the relevant region. This extracted screen content is then replaced with the original digital version, effectively removing the low-quality captured portion and substituting it with high-quality source material.
2Manufacturing precision
If the quality of the camera recording is increased, then the quality of visual content on screen is improved, but bandwidth and storage constraints are exceeded
Solution Approach 1:
The system divides the recording into screen region and non-screen region, applying different processing strategies. The screen region is replaced with original digital content obtained from metadata or external sources, while the non-screen region maintains the original camera recording quality. This selective approach ensures compatibility with standard bandwidth and storage constraints.
Solution Approach 2:
The system changes the quality parameters selectively - the screen content region is enhanced to match the original digital content quality, while the rest of the recording maintains its original parameters. This parameter differentiation allows the system to meet quality requirements for screen content without unnecessarily increasing overall file size and bandwidth requirements.
3Ease of operation
If standard camera recording methods are used, then the recording process is simple, but dynamic range problems occur (screens are over-exposed indoors or under-exposed outdoors)
Solution Approach 1:
The system extracts the screen content region from the camera recording and replaces it with the original digital version. This extraction and replacement process eliminates dynamic range issues that occur when cameras attempt to capture bright screens in various lighting conditions, as the original digital content retains its proper exposure and color information.
Solution Approach 2:
The system introduces an intermediary process that uses metadata (such as screen coordinates and content identification) to locate and retrieve the original digital content. This intermediary step acts as a bridge between the camera recording and the source material, allowing the system to substitute problematic captured screen content with properly exposed original content without complicating the overall recording workflow.
Data Source
Figure 1A~1B
Figure 2~3
Figure 4
AI summary
A system and method are provided for enhancing a media recording which comprises a camera recording of a scene, with the scene including a screen displaying visual content. In the camera recording, the visual content as displayed on the screen is typically of poor quality. By analysing the camera recording, accessing an original version of the visual content, and replacing, in the camera recording, the visual content displayed on the screen by the original version of the visual content, an enhanced media recording is obtained. Namely, in the enhanced media recording, a ‘digital-to-light-to-digital’ conversion of the visual content is avoided, being at least one reason for the visual content having a poor quality in the camera recording.