Visuotactile Sensor With Transmissive Membrane for Dynamic Object Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current robot perception systems, relying on vision and touch, fail to accurately capture the behavior of target objects during dynamic and uncontrolled motion, leading to uncertainties in dexterous and dynamic manipulation tasks.
Innovation Solution
A novel multimodal visuotactile sensor integrating an RGB camera, air pressure sensor, and infrared time-of-flight camera through a selectively transmissive soft membrane to provide simultaneous visuotactile and proximity depth data, enabling precise localization and deformation tracking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a robot perception system uses only vision and touch sensors, then the system structure remains simple, but it fails to accurately capture object behavior during dynamic transitions
Solution Approach 1:
The patent combines multiple sensing modalities (vision, touch, and proximity sensing) into a single integrated sensor system. The visuotactile sensor merges an RGB camera, infrared ToF camera, and air pressure sensor within one device, enabling simultaneous capture of visual, tactile, and depth information to accurately track object behavior during dynamic transitions.
Solution Approach 2:
The integrated sensor system performs multiple functions simultaneously: visual imaging, tactile sensing through membrane deformation, and proximity measurement via infrared ToF. This multi-functional approach allows the single sensor to replace multiple separate sensors while capturing comprehensive object behavior data during dynamic transitions.
2Measurement precision
If the membrane is made opaque to block external light, then tactile imaging quality improves, but proximity depth sensing capability is lost
Solution Approach 1:
The membrane is designed with different optical properties for different wavelengths: it is opaque to visible light to block external illumination and improve tactile image contrast, but transparent to infrared light to allow the ToF camera to capture proximity depth information. This wavelength-selective quality enables both tactile imaging and proximity sensing to function simultaneously without interference.
Solution Approach 2:
The membrane's optical transmission properties are optimized by changing parameters such as material composition and thickness to achieve selective transparency. The membrane is engineered to transmit infrared wavelengths while blocking visible wavelengths, allowing the system to maintain both high-quality tactile imaging and accurate proximity depth sensing.
3Loss of information
If the membrane is made transparent to allow proximity sensing, then depth data is captured, but tactile imaging quality deteriorates due to external light interference
Solution Approach 1:
The membrane exhibits different optical properties at different wavelengths: transparent to infrared light for proximity sensing and opaque to visible light for tactile imaging. This localized optical quality differentiation allows the system to capture proximity depth data while maintaining high tactile imaging quality by blocking external visible light interference.
4Adaptability or versatility
If multiple separate sensors are used for vision, touch, and proximity sensing, then each sensor can be optimized independently, but the overall system complexity increases
Solution Approach 1:
The patent integrates multiple sensing modalities into a single visuotactile sensor device. The RGB camera, infrared ToF camera, and air pressure sensor are combined within one integrated structure, reducing the number of separate sensors while maintaining the ability to optimize each modality's performance through dedicated design elements.
Solution Approach 2:
The integrated sensor system performs vision, touch, and proximity sensing functions within a single device, reducing system complexity while maintaining optimization flexibility. Each sensing modality is designed with its own optimized components (e.g., specific membrane properties for tactile imaging, ToF camera for depth sensing) within the unified sensor structure.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The sensor effectively captures key information about target object transitions, enhancing sensor-based control in dynamic manipulation tasks by providing synchronized, high-resolution tactile and proximity data without compromising spatial resolution.
Implementation Method 1
the deformable membrane allowing at least partial transmission of light therethrough and onto the imaging train
Implementation Method 2
infrared time-of-flight (ToF) camera to sense proximity
Data Source
AI summary
A visuotactile sensor, comprising: a deformable membrane; and an imaging train, the deformable membrane allowing at least partial transmission of light therethrough and onto the imaging train such that the imaging train collects light reflected through the membrane by an object on the opposite side of the membrane or contacting the membrane, and the imaging train further configured to collect light indicative of a deformation of the membrane by the object. A method, comprising: with an imaging train, collecting (a) a first light reflected through a deformable membrane by an object proximate to or contacting the deformable membrane and (b) a second light indicative of a deformation of the deformable membrane; and relating the at least one of the first light and the second light to an estimated position of the object, an estimated motion of the object, and an estimated deformation experienced by the membrane.