Gesture-Controlled Real-World Object Interface
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current devices for interacting with virtual objects in real-world environments, such as through eyewear or mobile devices, are limited in providing users with information and functionality, especially when users are away from substantial computing devices and rely on mobile devices with limited input and output capabilities.
Innovation Solution
A system and method using a camera, either head-mounted or part of a mobile device, to detect and analyze user gestures for controlling a user interface, allowing users to select and obtain information about real-world objects by projecting indicators or displaying data on a mobile device, enabling gestures to manipulate the UI and interact with real-world objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If users interact with virtual objects using traditional keyboard and mouse commands or touchscreen displays, then the interaction interface is well-established and reliable, but the system lacks real-world object recognition and gesture-based control capabilities
Solution Approach 1:
The patent introduces an intermediary system consisting of a camera, gesture recognition module, and object identification module that mediates between the user's physical gestures and the virtual UI elements. This intermediary layer enables real-world object interaction without requiring direct modification of existing input/output devices, thus resolving the contradiction between versatility and complexity.
Solution Approach 2:
The system makes the existing camera and processing units perform multiple functions: capturing images, recognizing gestures, identifying objects, and controlling UI interactions. By making existing components multi-functional, the system achieves real-world interaction capabilities without proportionally increasing device complexity.
2Weight of moving object
If users rely on mobile devices for computing functionality, then portability is improved, but input and output capabilities remain limited compared to substantial computing devices
Solution Approach 1:
The patent adds a spatial dimension to interaction by using gesture-based control in three-dimensional space. Instead of confined to a touchscreen or keyboard, users can interact with objects in the real-world environment through gestures, effectively multiplying the input capabilities without adding physical device size or weight.
Solution Approach 2:
The system replaces traditional mechanical input methods (keyboard, mouse, touchscreen) with gesture-based control detected by the camera. This substitution enables more intuitive and capable interaction while maintaining the portability of mobile devices, as no additional mechanical input hardware is required.
3Loss of information
If the system provides comprehensive information about real-world objects, then user decision-making is improved, but the amount of information processing and display requirements increase
Solution Approach 1:
The system creates a visual copy or representation of the real-world object within the UI, allowing users to interact with a simplified digital model that contains relevant information. This copying approach enables comprehensive information display without requiring complex processing of the entire real-world object, as the system works with a manageable digital representation.
4Speed
If the system detects and responds to user gestures in real-time, then interaction responsiveness is improved, but the computational requirements and processing time increase
Solution Approach 1:
The system performs preliminary action by pre-defining gesture patterns and object databases before actual interaction occurs. By having gesture templates and object information readily available, the system can quickly match observed gestures to predefined patterns and identify objects without performing complex analysis in real-time, thus reducing computational processing requirements while maintaining responsiveness.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables users to access significant information about products while shopping or in other contexts, enhancing the experience by allowing manipulation of object focus using gestures, providing a more active and informative interaction with the environment.
Implementation Method 1
receiving an image from a camera
Implementation Method 2
causing light to be projected on the physical object
Data Source
AI summary
Systems and methods described allow users to select and obtain significant information about objects in the real world, and further to employ gestures as a “real world” interface to manipulate information and to manipulate the selection of objects. In this way, users may be enabled to make better decisions when, e.g., traveling and shopping, and may further be enabled to obtain significant information about objects with which they are surrounded. The systems and methods may include a camera which monitors a user's hand movements or gestures to control a UI, particularly where a user is interacting with real-world objects. Gestures can move the focus of a UI from one real world object to another. The systems and methods may also include a projector to illuminate selected objects, or to display information about objects.