Mobile Screen Streaming via Video Frame Deskewing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices lack the capability to effectively stream screen content captured via video cameras, facing challenges such as off-axis recording, stability issues, and inefficient bandwidth usage, with no existing application supporting streaming of screen content via mobile video.
Innovation Solution
A computer-implemented method and system that captures video frames using a mobile device's camera, detects and processes the screen area, deskews the frames, and transmits the processed images to a remote server, allowing users to manually or automatically specify screen boundaries and correct for camera motion, enabling efficient streaming of screen content without requiring external software.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If unfiltered streaming is used from mobile devices, then bandwidth efficiency is poor and privacy concerns increase, but implementation is simple
Solution Approach 1:
The patent extracts and processes only the relevant screen content area from the full video frame, separating the useful information (screen content) from unnecessary data (background, off-axis areas). This extraction approach improves bandwidth efficiency by transmitting only essential content while maintaining manageable system complexity through automated detection algorithms.
2Ease of operation
If the recording device is handheld, then mobility and ease of use improve, but stability problems increase causing off-axis recording
Solution Approach 1:
The system performs self-correction by automatically detecting screen boundaries and deskewing the video content to compensate for handheld instability. The automated screen detection and geometric correction algorithms enable the system to self-adjust for off-axis recording issues, maintaining ease of handheld operation while improving recording stability.
3Adaptability or versatility
If screen content is streamed without external software hooks, then device versatility and ease of operation improve, but measurement precision of screen boundaries decreases
Solution Approach 1:
The patent replaces manual software hooks and mechanical integration methods with automated computer vision-based screen detection algorithms. The system uses image processing techniques to automatically identify screen boundaries, replacing the need for external software integration while maintaining sufficient detection precision through automated geometric analysis.
4Loss of information
If the entire video frame is transmitted, then no information is lost, but bandwidth usage increases inefficiently
Solution Approach 1:
The system extracts only the identified screen content portion from the full video frame for transmission, removing unnecessary background and off-axis areas. This selective extraction maintains the completeness of relevant screen information while significantly reducing bandwidth consumption by transmitting only the essential content area.
Data Source
AI summary
A system helps filter and correct video captured and streamed from a mobile device. In particular, the system detects and streams content shown on screens, allowing anyone to stream screen content immediately without needing to develop hooks into external software (i.e. without installing a screen recorder software in the computer). The system can use a variety of user-selectable techniques to detect the screen, and utilizes the mobile device's touchscreen to allow users to manually override detected corners. However, some of these approaches could potentially be applied to other types of content, such as identifying TV screens, appliance LCD screens, other mobile devices' screens, multifunction devices. (e.g. a remote technician could help troubleshoot a malfunctioning MFD by having the end-user point his cellphone to the LCD screen of the MFD).