Virtual Controller Using Hand Gesture Background Area Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vision-based computer input methods are inefficient due to the need for unnatural hand gestures and complex detection paradigms, which are difficult to implement at common hand locations like the keyboard, and often require additional equipment, leading to decreased user efficiency and increased risk of injury from repetitive strain.
Innovation Solution
A virtual controller system that uses a camera or sensor to detect hand and finger gestures, specifically the 'thumb and forefinger interface' (TAFFI), to assign control parameters for manipulating on-screen images, allowing users to control visual displays without physical contact, enabling efficient navigation and interaction with multiple hands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If gesture detection and tracking paradigms are used for sensing hand gestures, then hand movement detection is enabled, but computational complexity increases and detection reliability decreases due to ambiguities in hand movements
Solution Approach 1:
The patent extracts the essential feature of hand gesture detection by focusing solely on counting independent background areas (holes) formed by fingers, rather than attempting to detect and track complex hand shapes and movements. This extraction simplifies the detection paradigm from sophisticated pattern recognition to basic connected component analysis, significantly reducing computational complexity while maintaining detection capability
Solution Approach 2:
Instead of detecting hand shapes and movements directly, the patent inverts the approach by detecting the background areas (holes) that are enclosed by fingers. This inversion transforms the detection problem from analyzing complex foreground hand gestures to analyzing simpler background regions, reducing computational requirements and improving reliability by avoiding ambiguities in hand movement detection
2Ease of operation
If pointed or outstretched finger gestures are used for input, then hand movement input is enabled, but detection difficulty increases due to similarity with natural hand positioning during typing
Solution Approach 1:
The patent converts the harmful similarity between typing hand positions and gesture positions into a benefit by using the presence or absence of enclosed background areas as the detection criterion. During typing, fingers are spread and do not enclose background areas, while during gesture input, fingers enclose specific background areas. This transformation allows reliable distinction between typing and gesturing without requiring complex gesture recognition
Solution Approach 2:
The patent replaces the mechanical system of detecting hand shape and position with an optical field-based approach that counts connected background components. This substitution eliminates the need for sophisticated pattern recognition algorithms and enables simple, reliable detection of finger encirclement gestures even during typing
3Ease of manufacture
If repetitive switching between keyboard and mouse is performed, then text entry and cursor manipulation are completed, but user efficiency decreases over time
Solution Approach 1:
The patent makes the hand gesture interface universal by enabling it to perform multiple input functions that previously required separate devices. The same hand gestures can be used for both text input (typing on virtual keyboard) and cursor manipulation (selecting, clicking, dragging), eliminating the need to switch between physical keyboard and mouse and thereby improving user efficiency
4Reliability
If additional equipment is used for vision-based input, then gesture detection capability is improved, but device complexity and risk of repetitive strain injury increase
Solution Approach 1:
The patent applies self-service by using the existing camera infrastructure already present in most computing devices for gesture detection, rather than requiring specialized sensors or equipment. The system serves itself by utilizing readily available hardware resources, thereby improving detection reliability without increasing device complexity or requiring additional equipment
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Virtual controllers for visual displays are described. In one implementation, a camera captures an image of hands against a background. The image is segmented into hand areas and background areas. Various hand and finger gestures isolate parts of the background into independent areas, which are then assigned control parameters for manipulating the visual display. Multiple control parameters can be associated with attributes of multiple independent areas formed by two hands, for advanced control including simultaneous functions of clicking, selecting, executing, horizontal movement, vertical movement, scrolling, dragging, rotational movement, zooming, maximizing, minimizing, executing file functions, and executing menu choices.