Soft Gripper Optical Flow Segmentation for Finger State Sensing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Soft grippers lack proprioceptive sensing due to the absence of built-in joint-state encoders, leading to impractical precise control and limited grasping capabilities, especially in industrial automation tasks involving diverse objects.
Innovation Solution
A robotic system uses a single camera with a wide-angle lens mounted on the robot's palm or wrist to achieve millimeter-precise control through active optical flow-based self-segmentation, enabling real-time finger state estimation and manipulation without requiring camera calibration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If soft grippers use compliant materials without built-in encoders, then compliance-matching and stress distribution are improved, but proprioceptive sensing and measurement precision deteriorate
Solution Approach 1:
The patent introduces an external camera system as an intermediary to capture images of the soft gripper fingers. The camera serves as a mediator between the compliant soft materials and the measurement system, allowing optical flow analysis to estimate finger states without requiring encoders embedded in the soft materials themselves.
Solution Approach 2:
The patent replaces the mechanical encoder system with an optical measurement system. Instead of using mechanical joint-state encoders that would compromise the compliant nature of soft materials, the system uses image sensors and optical flow algorithms to non-contactively measure finger positions and states.
2Measurement precision
If image sensors are integrated within fingers or mounted on separate benches, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent makes the camera system multi-functional by mounting it on the robot's palm or wrist, allowing it to serve both as part of the robotic structure and as the measurement device. This universal placement eliminates the need for separate bench-mounted sensors or integrated finger sensors, reducing overall system complexity.
Solution Approach 2:
The patent merges the camera system with the robotic arm structure by mounting it on the palm or wrist. This consolidation combines the imaging function with the existing robotic structure, eliminating separate sensor assemblies and reducing the number of independent components.
3Adaptability or versatility
If modular soft grippers are used for diverse objects, then adaptability is improved, but control precision deteriorates due to absence of encoders
Solution Approach 1:
The patent implements a feedback control system where the camera continuously captures images of the soft gripper fingers, optical flow analysis estimates the current finger states, and this information feeds back to the controller. This closed-loop feedback enables precise control of modular soft grippers for handling diverse objects without requiring encoders.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables millimeter-level precision in grasping and manipulation of various objects with continuous finger state measurement, reducing costs and enhancing automation capabilities in industrial applications.
Implementation Method 1
an image sensor fixedly mounted on the arm to move with the multi-fingered soft gripper, and operable to capture image data of the multi-fingered soft gripper
Implementation Method 2
deriving an optical flow based on the image data and consecutive arm kinematic and velocity frame states
Data Source
AI summary
A robotic system, including: a robot having an arm with a multi-fingered soft gripper that is formed of a compliant material; an image sensor fixedly mounted on the arm to move with the multi-fingered soft gripper, and operable to capture image data of the multi-fingered soft gripper composed of a compliant material; and processor circuitry operable to generate a model of a state of the multi-fingered soft gripper by: controlling the arm to move in a predefined motion pattern while the image sensor captures the image data; deriving an optical flow based on the image data and consecutive arm kinematic and velocity frame states; and segmenting between a multi-fingered soft gripper portion and a background portion of the image data based on the optical flow, wherein static regions of the optical flow represent the multi-fingered soft gripper portion, dynamic regions of the optical flow represent the background portion, and non-coherent regions of the optical flow represent contour between the multi-fingered soft gripper portion and the background portion of the image data.