Robotic Manipulator Tactile Sensing with External Event Camera
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic grippers face challenges with slippage detection, contact force estimation, and grasp control due to the limitations of conventional tactile sensors, particularly in unstructured environments and high-precision tasks, and require multiple sensory systems for accurate machining, leading to increased cost and complexity.
Innovation Solution
Development of a neuromorphic event-based camera integrated with a robotic finger, utilizing an optic mirror system to provide proprioceptive and exteroceptive information, enabling robust grasping and precise manipulation through a single sensor system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a camera is placed within the gripper's fingers to capture tactile activity, then vision-based tactile sensing capability is improved, but the camera wiring and structure restrict the movement of the gripper
Solution Approach 1:
The camera is extracted from the interior of the gripper fingers and repositioned externally. The optical system captures tactile information by viewing the fingertips from the outside, eliminating the constraint of internal mounting that restricted gripper movement while preserving tactile sensing capability.
Solution Approach 2:
An optical system consisting of a camera and mirror arrangement serves as an intermediary to capture tactile information indirectly. Instead of placing the camera inside the gripper, the system uses optical reflections from the fingertip surfaces to obtain tactile data, thereby avoiding direct interference with gripper mechanics.
2Measurement precision
If the camera is placed within the gripper, then vision-based tactile sensing is achieved, but the gripper operation affects camera performance due to vibration
Solution Approach 1:
The camera is extracted from the vibrating gripper structure and mounted externally on a stable platform. This separation removes the camera from the source of vibration, ensuring reliable operation while maintaining its function of capturing fingertip tactile information through the optical system.
3Measurement precision
If the camera is placed within the gripper, then tactile sensing is enabled, but the camera can be damaged by the gripper's movement
Solution Approach 1:
The camera is extracted from the hazardous environment inside the gripper and positioned externally where it is protected from mechanical damage. The optical system allows the camera to observe fingertip contact without being exposed to the forces and movements that could damage it.
4Measurement precision
If multiple sensory systems are used for accurate machining, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The system merges vision-based tactile sensing with proprioceptive information from the robotic arm into a single integrated sensing framework. The event-based camera captures both external tactile events at the fingertip and internal proprioceptive data, combining multiple sensing functions into one unified system that reduces overall complexity.
Solution Approach 2:
The event-based camera serves multiple functions simultaneously: it provides vision-based tactile sensing at the fingertip, captures proprioceptive information from the robotic arm, and enables real-time event detection. This multi-functionality eliminates the need for separate sensory systems while maintaining high measurement precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The neuromorphic tactile sensor enhances sensitivity and range, reducing cost and complexity by providing real-time tactile feedback, improving grasp stability and precision in handling diverse objects and facilitating high-precision machining.
Implementation Method 1
Event cameras detect transient changes in dynamic scenes in terms of brightness intensity
Implementation Method 2
utilizing an optic mirror system to provide proprioceptive and exteroceptive information
Data Source
AI summary
A sensor system (e.g., for a robotic manipulator) may include a tactile sensor configured to engage with an object. The tactile sensor can include one or more transparent rigid layers, which may be configured to engage with the object and allow light to travel through the one or more layers.