Touch and Non-Touch Input Integration Across Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current air gesture technologies for computing devices lack the ability to seamlessly integrate touch and non-touch inputs across multiple displays, limiting the versatility and engagement of user interactions, particularly in manipulating virtual objects across different screens and devices.
Innovation Solution
The integration of image detectors and ultrasonic sensors with gesture recognition modules allows for the detection and processing of air movements and touch inputs, enabling the creation of virtual bindings and mappings that allow virtual objects to be moved and manipulated across multiple displays, including those on separate devices, using a combination of touch screens, 3D cameras, and ultrasonic sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If air gesture technology uses sensors or 3D camera technology to approximate body motion, then non-touch input capability is provided, but the ability to seamlessly integrate touch and non-touch inputs across multiple displays is limited
Solution Approach 1:
The patent combines touch screen sensors and ultrasonic sensors into a unified input system that operates across multiple displays. The gesture recognition module integrates signals from both sensor types to create seamless touch and non-touch input capability, allowing virtual objects to be manipulated consistently across different screens without requiring separate interaction modes for each display.
Solution Approach 2:
The system creates a universal input interface where the same virtual object can be interacted with using either touch gestures on a screen or air gestures detected by ultrasonic sensors. The gesture recognition module serves multiple functions by interpreting both touch and air gestures as equivalent control commands, enabling one interface to serve multiple input modalities across various displays.
2Adaptability or versatility
If virtual objects are moved across multiple displays using gesture recognition, then user interaction versatility is improved, but the precision and consistency of object manipulation across different screens is challenging
Solution Approach 1:
The system implements visual feedback by rendering virtual objects on multiple displays with consistent positioning and appearance. When a user performs a gesture to manipulate an object, the system provides real-time visual confirmation across all relevant displays, ensuring that the object's position and state are accurately reflected and consistent across different screens, thereby improving measurement precision through visual verification.
3Difficulty of detecting and measuring
If multiple sensors and gesture recognition modules are integrated, then detection capability is enhanced, but the processing complexity and computational resources required increase
Solution Approach 1:
The patent divides the detection system into separate functional modules: touch screen sensors for direct contact input, ultrasonic sensors for air gesture detection, and a gesture recognition module that processes signals from both sources. This segmentation allows each component to specialize in specific detection tasks, improving overall detection capability while managing processing complexity through modular architecture where each module handles specific computational responsibilities.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution enhances user experience by providing more engaging and versatile interactions, allowing for the manipulation of virtual objects across multiple screens and devices, offering real-time visual feedback and dynamic behavior, thus improving the overall user interface experience.
Implementation Method 1
an ultrasonic sensor to detect motion within 10 centimeters or less of the display
Data Source
AI summary
Various embodiments are generally directed a method and apparatus having a touch screen module to receive first input data from a touch screen sensor based on one or more detected touch inputs at a first location of a virtual object displayed on a display. In addition, an ultrasonic module may receive second input data from an ultrasonic sensor based on detected non-touch motion associated with the virtual object. The detected non-touch motion may be tracked from the first location to a second location in a direction away from the first location based on the second input data and used to determine the second location for the virtual object based on the tracking.