Camera-Based Screen Interaction Engine for Large Display Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern electronic devices face limitations in allowing users to interact with and view displayed content due to small screen sizes, hindering simultaneous interaction and viewing capabilities.
Innovation Solution
A software engine that enables interaction with a separate display using a camera, allowing users to input commands and manipulate virtual environments through hand gestures or objects in real space, which are then reflected on a larger display.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users interact with the device using the small display, then user input capability is improved, but the ability to view displayed content is limited
Solution Approach 1:
The patent introduces a camera as an intermediary device to capture hand gestures in real space. The camera acts as a mediator between the user's physical movements and the digital interface, allowing users to interact with the device without directly touching the small display. Hand gestures captured by the camera are translated into input commands, enabling users to maintain input capability while viewing content on the full display area.
Solution Approach 2:
The patent extends the interaction space from the two-dimensional display surface to three-dimensional real space. By capturing hand gestures in 3D space through the camera, users can interact with the device beyond the constraints of the small display area. This dimensional expansion allows full display area to be used for viewing while maintaining input capability through spatial gestures.
2Area of stationary object
If the display area is increased to improve viewing, then the ability to interact directly on the display is reduced
Solution Approach 1:
The camera serves as an intermediary that bridges the gap between the increased display area and direct interaction capability. By capturing hand gestures in real space and translating them into input commands, the system maintains ease of operation despite the display being used primarily for viewing. The intermediary converts physical gestures into digital inputs without requiring direct display contact.
Solution Approach 2:
The patent replaces the mechanical touch interaction system with an optical recognition system. Instead of requiring physical contact with the display surface, the camera-based hand gesture recognition substitutes mechanical interaction with optical field-based gesture capture. This substitution allows the display to be used fully for viewing while maintaining interaction capability through gesture recognition.
3Adaptability or versatility
If hand gesture recognition is implemented to enable free-space interaction, then interaction freedom is improved, but system complexity increases
Solution Approach 1:
The patent leverages the existing camera component in mobile devices, giving it multiple functions: traditional photography and now hand gesture recognition. By making the camera universal and multi-functional, the system achieves improved interaction freedom without proportionally increasing device complexity. The same hardware component serves dual purposes, reducing the need for additional specialized sensors or devices.
Solution Approach 2:
The system utilizes the device's existing camera and processing capabilities to perform hand gesture recognition, allowing the device to serve itself for interaction purposes. Rather than requiring external specialized equipment, the mobile device uses its own built-in components for gesture recognition, reducing overall system complexity while maintaining interaction freedom.
Data Source
AI summary
The technology of this application relates to a software engine that enables a user to interact with a display (e.g., television) by using an image capture device (e.g., camera). In one non-limiting example, a user can operate a device (such as a mobile phone) to run an application that interfaces with the software engine, so that the device's camera can operate in unison with a separate display to enable unique types of user interactions. For example, the user may use such an application to tap a user interface button or draw a shape with their hand, directly on the surface of a larger display (e.g., television), in view of the mobile phone's camera; the application's visual state would update in response to the user's captured gestures and to the position and orientation of the mobile phone. In another non-limiting example, a user, while playing a game utilizing the software engine, swings a device (such as a mobile phone) in free space, to control the position and orientation of a paddle within the three dimensional scene of the game, presented on the larger display.