3D Fingertip Model for Vision-Based Touch Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current touch detection systems in mixed reality environments struggle to accurately detect touch events on real objects without traditional touch sensors, especially when the touching object is occluded or not directly sensed by the device's sensors.
Innovation Solution
A method involving two electronic devices capturing images of a finger or hand from different perspectives to generate a 3D model of the touching object, allowing for vision-based touch detection and gesture recognition, even when the touching object is occluded or not directly sensed by the device's sensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional touch sensors are used to detect touch events, then touch detection accuracy is improved, but device complexity and cost increase
Solution Approach 1:
The patent replaces traditional mechanical/electrical touch sensors with a vision-based system using cameras and 3D modeling. The system captures images from multiple perspectives, generates a 3D model of the touching object, and detects touch events through visual analysis rather than physical sensor contact, thereby eliminating the need for complex touch sensor hardware.
Solution Approach 2:
The patent creates a 3D digital copy or model of the touching object based on images captured from multiple perspectives. This virtual 3D model serves as a representation that can be analyzed for touch detection without requiring physical sensors on the target surface, enabling touch detection through visual copying and modeling rather than direct physical sensing.
2Manufacturing precision
If multiple cameras are used to capture images from different perspectives, then 3D model accuracy is improved, but device complexity increases
Solution Approach 1:
The patent divides the imaging task across multiple cameras positioned at different perspectives. Each camera captures a specific view of the touching object, and these segmented visual data sets are then integrated to construct a complete 3D model. This segmentation approach improves model accuracy by capturing geometric details from multiple angles while distributing the computational complexity across separate imaging components.
3Ease of operation
If vision-based touch detection is implemented without touch sensors, then ease of operation is improved, but measurement precision deteriorates
Solution Approach 1:
The patent transitions from 2D image capture to 3D model generation by incorporating depth information through multiple perspective imaging. This dimensional enhancement allows the vision system to accurately detect touch events, contact points, and object geometry in three-dimensional space, thereby improving measurement precision while maintaining the ease of operation associated with sensorless detection.
Data Source
AI summary
Generating a 3D model may include determining, based on sensor data from a touch sensor on a first device, a touch event, wherein the touch event comprises a touch on the first device by a touching object. Generating a 3D model may also include, in response to a touch event, obtaining a first image of the touching object by a first camera of the first device, and obtaining, from a second device, a second image of the touching object, wherein the first image of the touching object captures a first view of the touching object, and wherein the second image of the touching object captures a second view of the touching object. A model of the touching object is generated based on the first image and the second image.