Smartphone Camera Screen Mapping for Remote Content Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional remote controllers for large-screen devices like smart TVs are costly and complex, making it difficult for users to control diverse content and applications, especially with the increasing variety of web content displayed on these devices.
Innovation Solution
A method and apparatus using a smartphone with various input and output devices, such as buttons, multi-touch screens, and voice recognition, to capture and control remote screens by defining a remote screen area, performing screen mapping, and transmitting user input events to control objects on the remote screen.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a conventional remote controller with additional buttons is used to control diverse content and applications, then the functional versatility is improved, but the device complexity and ease of operation deteriorate
Solution Approach 1:
The smartphone serves as a universal remote control that can control diverse content and applications on the TV through a single device. The camera captures the TV screen, the display unit shows the captured image, and the communication unit transmits control signals, allowing one device to perform multiple control functions without requiring separate buttons for each function.
Solution Approach 2:
The camera-captured image of the TV screen acts as an intermediary between the user and the TV content. Instead of directly controlling the TV with physical buttons, the user interacts with the displayed image on the smartphone screen, which then translates user inputs into appropriate control signals for the TV.
2Ease of operation
If a multi-touch interface is applied to a large-screen device like a smart TV, then the ease of operation is improved, but the manufacturing cost increases
Solution Approach 1:
Instead of equipping the large-screen TV with an expensive multi-touch interface, the system creates a visual copy of the TV screen on the smartphone display. This copy allows the user to interact with TV content through the smartphone's multi-touch interface, providing the same ease of operation without modifying the TV hardware.
Solution Approach 2:
The control interface is moved from the two-dimensional TV screen to another dimension - the smartphone screen. By transferring the interaction interface to a different device with different capabilities, the system gains multi-touch functionality without the cost of implementing it on the large-screen device.
3Ease of operation
If a camera-based gesture recognition system is used to control remote content, then the ease of operation is improved, but the device complexity and cost increase
Solution Approach 1:
The camera on the smartphone serves a dual purpose: it captures images for display and simultaneously enables gesture recognition control. The same hardware component (camera) provides both the visual feedback to the user and the gesture detection capability, eliminating the need for separate sensors or additional complexity.
Data Source
AI summary
There is provided a mobile terminal with a camera, such as a smart phone, capable of controlling the functions of a remote screen and content displayed on the remote screen. The mobile terminal defines a remote screen area in an image obtained by capturing the remote screen, receives remote screen resource information, and performs screen mapping between the remote screen area and the remote screen using the remote screen resource information. If it receives a user input signal related to the remote screen area, the mobile terminal generates event information for controlling an object based on the results of the screen mapping, and transmits the event information to a remote apparatus with the remote screen. The remote apparatus processes the event information and controls the object.