Free-Space Gesture Control via 3D Reconstruction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer systems rely on cumbersome input methods like mice, touch pads, and touch screens, which are inconvenient, inaccurate, and restrictive in terms of user movement and detection reliability.
Innovation Solution
The system interprets the positions, configurations, and motions of control objects in free space using an image-capture device, reconstructing them in three dimensions and tracking their motion over time to discern gestures, which are then mapped to virtual control constructs for input.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If touch screens or touch pads are used for input, then the interface is intuitive and direct, but the accuracy deteriorates because fingers are fat and can cover multiple links leading to erroneous selection
Solution Approach 1:
The patent introduces an intermediary device (mouse, touch pad, or touch screen) that mediates between the user's finger and the display elements. This intermediary allows precise control through cursor movement rather than direct finger contact with small elements, resolving the contradiction between intuitive touch operation and selection accuracy.
2Ease of operation
If mice are used to facilitate user input at distance, then the range of motion is improved, but the device complexity increases and requires flat surfaces or special mouse pads
Solution Approach 1:
The patent makes the display surface itself serve multiple functions: it acts as both the visual output medium and the input control surface. By enabling direct manipulation of display elements through touch, the system eliminates the need for separate mice or specialized surfaces, reducing device complexity while maintaining operational flexibility.
3Area of stationary object
If touch pads or touch screens are integrated into the keyboard area, then the device footprint is reduced, but the user must be in close physical proximity which restricts range of motion
Solution Approach 1:
The patent extends the interaction space by allowing users to reach across the display surface from a distance. The touch screen technology enables input operations without requiring the user to be in close proximity to a compact pad, effectively utilizing the vertical and depth dimensions of the display area to accommodate larger ranges of motion while maintaining a compact form factor.
4Device complexity
If traditional mice are positioned sub-optimally, then connectivity is simplified, but user comfort deteriorates due to discomfort or pain from prolonged use
Solution Approach 1:
The patent enables the display surface to serve its own input control function, eliminating the need for external mice that require optimization of positioning and connection. The touch-capable display autonomously provides both output and input capabilities, removing the source of comfort issues associated with prolonged mouse use while maintaining connection simplicity.
Data Source
AI summary
Embodiments of display control based on dynamic user interactions generally include capturing a plurality of temporally sequential images of the user, or a body part or other control object manipulated by the user, and computationally analyzing the images to recognize a gesture performed by the user. In some embodiments, a scale indicative of an actual gesture distance traversed in performance of the gesture is identified, and a movement or action is displayed on the device based, at least in part, on a ratio between the identified scale and the scale of the displayed movement. In some embodiments, a degree of completion of the recognized gesture is determined, and the display contents are modified in accordance therewith. In some embodiments, a dominant gesture is computationally determined from among a plurality of user gestures, and an action displayed on the device is based on the dominant gesture.