Multidirectional Gesturing for On-Display Item Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional web browsing and computer operating systems require high-precision selection techniques for extracting web content, which are cumbersome and time-consuming, especially on mobile devices with small screens and inaccurate finger-based interactions, leading to cognitive and physical overload when trying to collect and prioritize information for research and exploration tasks.
Innovation Solution
A method of controlling a computing system via visual display gestures, where the system monitors user input to recognize item-action gestures, such as wiggling or reciprocating movements, to identify and manipulate on-display items without the need for explicit user actions like button presses, allowing for efficient selection, copying, and rating of content.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional high-precision selection techniques are used, then selection accuracy is improved, but user effort and time consumption increase significantly
Solution Approach 1:
The system automatically detects and interprets user intent from casual gesturing patterns without requiring explicit selection actions. The computing system serves itself by monitoring gestures and autonomously determining selection boundaries, eliminating the need for users to manually adjust selection handles with high precision.
Solution Approach 2:
Instead of requiring users to precisely define selection boundaries through cursor placement, the system inverts the approach by having users gesture casually within a region and letting the system automatically determine the selection. The burden shifts from user precision to system interpretation.
2Measurement precision
If conventional selection techniques are used, then selection precision is maintained, but time consumption increases
Solution Approach 1:
The system performs preliminary gesture monitoring and interpretation in real-time as users naturally gesture, automatically determining selection boundaries without requiring subsequent manual adjustments. This preliminary automatic interpretation eliminates the time-consuming back-and-forth of manual selection handle positioning.
Solution Approach 2:
The gesture monitoring operates continuously without interruption, allowing users to maintain their natural reading or browsing flow while the system continuously interprets gestures for selection purposes, eliminating the discontinuous interruptions of conventional selection methods.
3Ease of operation
If finger-based touch interactions are used on mobile devices, then direct manipulation is achieved, but selection accuracy deteriorates
Solution Approach 1:
The system introduces gesture pattern recognition as an intermediary layer between finger touch and selection outcome. Instead of directly mapping finger position to selection boundary, the system interprets the pattern and context of the gesture to determine the intended selection, compensating for the inherent imprecision of finger-based input.
4Measurement precision
If multiple micro-adjustments of selection handles are made, then selection accuracy is improved, but cognitive load increases
Solution Approach 1:
The computing system performs the cognitively demanding task of determining selection boundaries autonomously by monitoring and interpreting gesture patterns. Users simply need to gesture casually within the general region of interest, and the system handles the complex boundary determination without user intervention.
Data Source
AI summary
Methods of controlling a computing system based on wiggling and/or other types of continuous multidirectional gesturing. In some embodiments, the methods monitor user gesturing for the occurrence of a recognizable item-action gesture that a user has made without having provided any other input to the computing system and then take one or more actions in response to recognizing the item-action gesture. In some embodiments, the actions include identifying one or more on-display item underlying the item-action gesture, duplicating the identified item(s) to one or more target locations, and adding one or more visual indicia to the identified on-display item(s), among other things In some embodiments, a user can append an item-action gesture with one or more action extensions that each cause the computing system to take one or more additional actions concerning the identified on-display item(s). Software for performing one or more of the disclosed methodologies is also disclosed.