Context-Aware Gesture System for Multi-Screen Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in intuitively interacting with multiple-screen devices, as existing technologies lack well-defined methods for easily opening applications, sharing data, and attaching files between multiple open applications, leading to cumbersome user inputs and increased CPU and memory usage.
Innovation Solution
A context-aware gesture system is introduced, allowing users to perform actions like opening applications, sharing data, and attaching files with reduced inputs by defining custom actions responsive to specific user motions, which are executed automatically based on the application context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If multiple inputs (taps, clicks, touches) are required to move application icons between screens and perform actions, then application interaction capability is provided, but user operation complexity increases and processing resources are consumed
Solution Approach 1:
The system segments the user interaction into two distinct parts: gesture recognition (detecting the throw motion) and context interpretation (determining the target application and action). This segmentation allows the system to process complex interactions in manageable steps, reducing the perceived complexity for the user while maintaining sophisticated functionality.
Solution Approach 2:
The system enables self-service by automatically interpreting the user's throw gesture and executing the appropriate action without requiring additional inputs. Once the user performs the throw gesture, the system autonomously determines the target application, screen, and action type, eliminating the need for the user to manually navigate through multiple menus or buttons.
2Adaptability or versatility
If multiple inputs are required to interact with applications on multi-screen devices, then application functionality is accessible, but CPU cycles and memory usage increase
Solution Approach 1:
The system performs preliminary action by pre-defining gesture patterns (such as throw gestures) and their corresponding actions. When a user performs a recognized gesture pattern, the system can directly execute the pre-determined action without requiring additional processing steps, thereby reducing CPU cycles and memory usage while maintaining versatile application interaction capability.
3Reliability
If traditional multi-step input methods are used, then precise application control is achieved, but user time and operational efficiency are reduced
Solution Approach 1:
The system merges multiple traditional input steps into a single unified gesture action. By combining the selection, navigation, and execution steps into one throw gesture, the system maintains precise application control while dramatically reducing the time required to complete interactions.
Data Source
AI summary
A gesture definition is provided for intuitively interacting with applications on a multi-screen device. The gesture is defined by one or more user motions signaling a user intent to perform an action on an object based on a context within the multi-screen environment. Unlike traditional gestures, the defined gesture is “context aware.” That is, the gesture may initiate different actions in different contexts. Actions responsive to the gesture include opening an application, sharing data between applications (e.g., cut/paste), and/or attaching files (e.g., to a mail message). App developers can define custom actions responsive to the gesture for particular applications. Avoiding cumbersome, multi-step user inputs and interactions, the actions defined for the gesture may be performed automatically. In this way, the well-defined gesture allows users to intuitively interact with applications between and among the various screens of a multi-screen device, including seamlessly opening applications, sharing data, and attaching files.