Bounding Box Gesture Recognition on Grid-Based Touch Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Grid-based touch sensors struggle to accurately distinguish multiple simultaneous contact points, leading to ambiguity and the need for expensive capacitive sensors in large interactive displays, which limits scalability and practicality.
Innovation Solution
The method identifies user gestures by analyzing the position, motion, shape, and deformation of a bounding box enclosing contact points, allowing for the use of inexpensive and reliable grid-based touch sensors, and executes commands such as pan, zoom, and rotate based on predetermined thresholds and characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If grid-based touch sensors are used to determine contact locations, then cost and reliability are improved, but measurement precision deteriorates due to ghosting effect
Solution Approach 1:
The patent combines multiple contact location measurements into a single bounding box representation. Instead of attempting to resolve individual ghosted contact points, the system merges all contact information within the ambiguous region into one unified bounding box, which then serves as the basis for gesture recognition.
Solution Approach 2:
The patent transitions from analyzing individual contact point coordinates (2D precise points) to analyzing bounding box characteristics (4D parameters including position, size, shape, and orientation). This dimensional transformation allows gesture recognition to proceed despite loss of precise contact location information.
2Measurement precision
If capacitive sensors are used to directly determine contact positions, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces expensive, complex capacitive sensors with inexpensive grid-based sensors. While the grid-based sensors provide less precise information (bounding boxes instead of exact points), this approach is sufficient for gesture recognition and dramatically reduces system cost and complexity.
Solution Approach 2:
The patent changes the parameter representation from precise contact coordinates to bounding box characteristics. This parameter transformation allows the system to work with less precise sensor data while maintaining adequate gesture recognition capability, thereby enabling the use of cheaper sensors.
3Ease of manufacture
If grid-based sensors report bounding box instead of precise contact points, then ease of manufacture is improved, but loss of information increases
Solution Approach 1:
The patent performs preliminary transformation of contact information into bounding box characteristics before gesture recognition. By pre-processing the data into position, size, shape, and orientation parameters, the system prepares the information in a form that is sufficient for gesture identification, compensating for the information loss through timely data transformation.
Solution Approach 2:
The system uses feedback from multiple bounding box measurements over time to improve gesture recognition accuracy. By analyzing changes in bounding box characteristics across successive measurements, the system can distinguish between different gesture types even though individual measurements lack precise contact point information.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables intuitive user interaction with large displays using cost-effective grid-based sensors by accurately interpreting gestures through bounding box analysis, enhancing scalability and reliability while avoiding the limitations of capacitive sensors.
Implementation Method 1
a point of contact C1 interrupts the beam of light passing from an emitter E1x to a sensor S1x and the beam of light passing from an emitter E1y to a sensor S1y
Data Source
AI summary
The invention provides a method and apparatus for identifying gestures performed by a user to control an interactive display. The gestures are identified based on a bounding box enclosing the points at which a user contacts a touch sensor corresponding with the display surface. The invention thus permits the use of inexpensive and highly reliable grid-based touch sensors that provide a bounding box to describe contact information. In identifying the gestures, the position, motion, shape, and deformation of the bounding box may all be considered. In particular, the center, width, height, aspect ratio, length of the diagonal, and orientation of the diagonal of the bounding box may be determined. A stretch factor, defined as the maximum of the ratio of the height of the bounding box to the width of the bounding box and the ratio of the width of the bounding box to the height of the bounding box, may also be computed. Finally, gestures may be identified based on the changes in time of these characteristics and quantities.