Depth Camera and Resistive Touch Screen Hybrid Sensing for Drag Gesture Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current touch screen devices struggle with accurately detecting multiple touches, distinguishing between single and multiple fingers, and supporting a hover state, leading to limitations in user interface richness and performance, especially with resistive touch screens that break contact during drag gestures.
Innovation Solution
A system combining a physical contact sensing component, such as a resistive touch screen, with a depth camera to capture object location and distance information, allowing for enhanced accuracy in touch event detection and enabling multi-touch capabilities and hover states by correlating data from both sources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a resistive touch screen is used to register physical contact, then single touch can be reliably detected, but touches over longer distances (dragging) break contact with the screen surface
Solution Approach 1:
The patent combines a depth camera with a resistive touch screen to create a hybrid sensing system. The depth camera continuously tracks the position of the finger or object during drag gestures, while the resistive touch screen provides accurate contact detection when the object is touching the surface. This merging of two different sensing modalities allows the system to maintain reliable detection throughout the entire drag operation, not just when contact is made.
Solution Approach 2:
The depth camera acts as an intermediary that bridges the gap between discrete touch contacts. By providing continuous spatial tracking data, it mediates the information between the resistive touch screen (which only detects contact) and the user interface system (which needs continuous gesture information). This intermediary fills in the information gaps during drag operations where the resistive screen alone would lose tracking.
2Measurement precision
If a traditional touch screen is used, then the screen can sense physical touches, but it cannot sense the presence of a finger or object held close to, but not physically touching the screen
Solution Approach 1:
The patent adds a third dimension (depth/distance from screen) to the traditional two-dimensional touch screen sensing. The depth camera provides Z-axis information about the position of fingers or objects relative to the screen surface, enabling the system to detect hover states at different distances. This dimensional extension transforms the sensing capability from binary (touch/not touch) to continuous (distance-based presence detection).
Solution Approach 2:
The patent replaces the mechanical contact-based sensing of traditional touch screens with an optical sensing system (depth camera) that can detect presence without physical contact. Instead of requiring mechanical pressure to register a touch, the depth camera uses optical fields to detect the position and presence of fingers or objects in three-dimensional space, enabling hover state detection.
3Measurement precision
If many touch screens devices are used, then single touch detection is supported, but they do not support multiple simultaneous touches
Solution Approach 1:
The patent segments the touch detection function into two complementary components: the resistive touch screen provides precise location data for each contact point, while the depth camera provides spatial segmentation of multiple fingers or objects in three-dimensional space. By processing data from both sources, the system can distinguish and track multiple simultaneous touches independently, with each touch maintaining its own accurate location information.
Solution Approach 2:
The combined sensing system achieves multi-functionality by simultaneously supporting single touch detection, multi-touch detection, and hover state detection. The depth camera provides a universal spatial awareness capability that works across all interaction modes, while the resistive touch screen provides precise contact information when needed. This universal system adapts to handle any number of simultaneous touches or hover states without requiring different hardware configurations.
Data Source
AI summary
The present invention provides a method (500) executed on a processor for providing object information about an object in proximity to a display screen. The method includes the step of determining whether an at least first object is located within a proximal distance of the first surface of the display screen based on information provided by a depth camera (step 510). The method also includes the step of determining whether the at least first object has physically contacted a first surface of the display screen based on information provided by a physical contact sensing component (step 520). The information about (1) the proximal distance of the at least first object from the screen and (2) whether the at least first object has physically contacted the display surface of the screen to provide proximal event information and based on the information provided, it is determining whether the user interface should be modified (step 560).