Mobile Device Gesture Control via Camera Intermediary
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices face limitations in user input options due to touchscreen size, which can impede the number and size of control elements, leading to frustrating user experiences and slow input operations.
Innovation Solution
A system and method that utilizes digital images to control mobile device functions by identifying user-selected content and user actions within these images, generating corresponding mobile device commands, and communicating them to the operating system, allowing for user-generated text and geometric shapes to trigger commands like executing image searches or controlling device operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If touchscreen size is increased to provide more control elements, then user input options increase, but device portability and compactness deteriorate
Solution Approach 1:
The patent extends the control interface from the two-dimensional touchscreen to the three-dimensional physical space surrounding the device. Users can draw gestures in the air using their finger, creating a spatial dimension for input that is not constrained by screen real estate. This allows complex multi-element controls to be represented through spatial trajectories rather than requiring proportional screen area.
Solution Approach 2:
The patent introduces an intermediary layer between the user and the device - a camera that captures visual images of gestures performed in physical space. This intermediary translates physical hand movements into digital commands, enabling rich control input without requiring corresponding increases in touchscreen area. The camera acts as a mediator that converts spatial gestures into actionable device commands.
2Adaptability or versatility
If more control elements are added to the touchscreen, then device functionality increases, but ease of operation deteriorates due to smaller element size
Solution Approach 1:
The patent moves control interactions from the constrained two-dimensional touchscreen plane to the expansive three-dimensional space around the device. By performing gestures in air space, users can execute complex commands with large, easy-to-hit target zones that are not limited by screen dimensions. The spatial dimension allows for larger effective control areas while maintaining compact device form factor.
Solution Approach 2:
The patent segments the control interface into distinct gesture components that can be performed sequentially or in combination. Complex functions are broken down into manageable gesture segments (e.g., draw a shape, then perform an action within it), making the overall interaction easier to learn and execute compared to navigating numerous small on-screen buttons and menus.
3Adaptability or versatility
If touchscreen control elements are reduced in size to fit more functions, then device versatility increases, but input speed deteriorates
Solution Approach 1:
The patent transitions control from the limited touchscreen surface to the unlimited spatial environment surrounding the device. Users can perform gestures in the air at natural speeds without the precision constraints of small touchscreen targets. This spatial approach enables faster input because users don't need to carefully position fingers on small screen elements - they can make larger, quicker movements in three-dimensional space that are still accurately captured by the camera.
4Device complexity
If traditional touchscreen controls are used, then device simplicity is maintained, but user experience deteriorates due to input limitations
Solution Approach 1:
The patent introduces a camera-based intermediary that captures visual images of user gestures and translates them into device commands. This intermediary layer adds minimal complexity to the device while dramatically improving ease of operation. The system maintains simplicity by using a familiar camera component and intuitive gesture-based interaction, avoiding the need for complex on-screen interfaces while enhancing user experience through natural, expressive input methods.
Data Source
AI summary
In one exemplary embodiment, a method includes receiving a digital image from a mobile device. A first element of the digital image and a second element of the digital image is identified. The first element of the digital image includes a user-selected content. The second element includes a depiction of a user action. An event trigger identified by the first element and the second element is determined. The event trigger includes a mobile device operating system command. A mobile device command is generated according to the event trigger. The mobile device command is communicated to a mobile operating system.