Semitransparent Tactile Surface Sensor for Robotic Grasping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing tactile sensors are opaque, obstructing visual views of objects during robotic interactions, and lack precise feedback on object pose changes, leading to imprecise manipulation and potential object toppling or translation.
Innovation Solution
A semitransparent tactile surface (STS) sensor with an image sensor and semitransparent membrane that captures both visual and tactile streams, processed through a multimodal deep neural network to provide unobstructed visual information and precise tactile feedback, enabling prediction of object configuration and improved robotic control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an opaque tactile sensor is used, then tactile sensing capability is provided, but visual view of objects during interaction is obstructed
Solution Approach 1:
The patent applies a semitransparent membrane that can change its optical properties. The membrane appears opaque when no contact is detected (maintaining tactile sensing) and becomes transparent when contact is detected (allowing visual view), thus resolving the contradiction between tactile sensing capability and visual information access
Solution Approach 2:
The membrane transitions dynamically between opaque and transparent states based on contact detection. This dynamic property allows the sensor to adapt its transparency in real-time according to operational conditions, enabling both tactile sensing and visual viewing at different times
2Ease of operation
If a conventional grasping sensor is used, then basic tactile feedback is provided, but precise prediction of object configuration changes is not achieved
Solution Approach 1:
The system implements a feedback mechanism where the semitransparent membrane detects contact and transmits this information to a control system. The feedback includes contact location, force magnitude, and visual information, enabling precise prediction of object configuration changes and improved grasping control
Solution Approach 2:
The patent combines tactile sensing (through the membrane's contact detection) and visual sensing (through the image sensor capturing light transmission changes) into a unified sensing system. This merging of modalities enables comprehensive object configuration prediction that neither sensor could achieve alone
3Loss of information
If the semitransparent membrane is made more transparent, then visual view is improved, but tactile sensing precision may be reduced
Solution Approach 1:
The membrane utilizes light transmission properties where the degree of transparency correlates with contact detection. When transparent, it allows visual view; when opaque, it indicates contact. This optical property change enables the membrane to provide both visual information and tactile sensing precision without compromising either function
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The STS sensor allows for precise robotic manipulation by providing high-resolution tactile and visual information, enabling accurate prediction of object configuration and enhancing control during interactions, reducing the likelihood of object toppling or translation.
Implementation Method 1
a visual light source configured to emit light to an object provided above the semitransparent layer, the semitransparent layer being configured to transmit light emitted by the visual light source
Implementation Method 2
a tactile light source provided below the semitransparent layer, the semitransparent layer being configured to reflect light emitted by the tactile light source
Implementation Method 3
an image sensor provided below the transparent gel layer. The image sensor may be configured to capture images of the semitransparent layer during periods of light emission by the tactile light, and the image sensor is configured to capture images through the semitransparent layer during periods of light emission by the visual light source
Data Source
AI summary
A method of sensing an interaction with an object using a semitransparent tactile surface (STS) sensor having an image sensor and a semitransparent membrane includes capturing, by the image sensor, a stream of images; separating the stream of images into a visual stream including light traveling through the semitransparent membrane and a tactile stream including light reflecting off of the semitransparent membrane; and processing the visual stream and tactile stream through a multimodal deep neural network.