Robot Grasp Selection From Handwritten Image Input
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing remote control systems for robots lack intuitive operation methods, relying on inputting predetermined instruction figures by handwriting, which is not user-friendly.
Innovation Solution
A remote control system that includes an imaging unit to capture the environment, a recognition unit to identify graspable objects, and an operation terminal for receiving handwritten input information, allowing the system to estimate which object to grasp and how to perform the grasping motion based on recognized objects and input information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If predetermined instruction figures are input by handwriting, then the robot can be remotely controlled, but the operation is not intuitive and user-friendly
Solution Approach 1:
The patent replaces the mechanical system of selecting from predetermined instruction figures with an image processing system that recognizes handwritten gestures. The recognition unit processes handwritten input images to directly identify grasping operations, substituting the figure-matching mechanism with automated image recognition technology, thereby improving intuitiveness while managing system complexity through software-based solutions.
Solution Approach 2:
The system enables self-service by allowing the user's natural handwriting to directly convey operational intent without requiring selection from predefined options. The recognition unit automatically interprets the handwritten input as specific grasping operations, making the system adapt to the user's natural expression rather than requiring the user to adapt to system constraints.
2Ease of operation
If natural handwriting input is accepted, then operational intuitiveness is improved, but the system complexity increases due to image recognition requirements
Solution Approach 1:
The patent extracts and isolates the image recognition functionality into a dedicated recognition unit that processes only handwritten input images. This modular approach separates the complex recognition task from the overall control system, allowing the natural handwriting input to be processed by a specialized component while the rest of the system remains relatively simple and focused on executing recognized commands.
Solution Approach 2:
The system uses image processing to create a digital copy of the user's handwriting and analyzes this copy to determine the intended operation. By working with a digital representation rather than requiring direct interpretation of physical handwriting variations, the system manages complexity while maintaining natural input capabilities.
Data Source
AI summary
A remote control system includes: an imaging unit that shoots an environment in which a device to be operated including an end effector is located; a recognition unit that recognizes objects that can be grasped by the end effector based on a shot image of the environment shot by the imaging unit; an operation terminal that displays the shot image and receive handwritten input information input to the displayed shot image; and an estimation unit that, based on the objects that can be grasped and the handwritten input information input to the shot image, estimates an object to be grasped which has been requested to be grasped by the end effector from among the objects that can be grasped and estimates a way of performing a grasping motion by the end effector, the grasping motion having been requested to be performed with regard to the object to be grasped.