Robot Manipulator Learning Data Generation Using Virtual Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for generating learning data for object holding in robot systems are labor-intensive and time-consuming, especially when the object needs to be imaged from various orientations and positions to improve recognition and holding success rates.
Innovation Solution
An information processing apparatus that obtains holding position and orientation data, along with success or failure information, to generate learning data by associating this information with images of the object, using a convolutional neural network for machine learning and object recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If learning data is generated by imaging the target object from various positions and orientations to improve recognition capability, then the recognition accuracy is improved, but the time and labor required for data generation increases significantly
Solution Approach 1:
The patent uses computer graphics (CG) to create virtual captured images as copies of real objects. Instead of physically capturing images from multiple positions and orientations, the system generates synthetic images through computer rendering, dramatically reducing the time and labor for data generation while maintaining the variety needed for accurate recognition
Solution Approach 2:
The system performs automatic generation of training samples through computer graphics without requiring manual intervention for each image capture. The automated process generates diverse learning data independently, eliminating the need for human operators to physically position and image objects from multiple angles
2Measurement precision
If learning data is generated by performing actual holding operations with a robot manipulator to improve holding success rate, then the holding accuracy is improved, but the time required for learning increases due to only one piece of data per holding operation
Solution Approach 1:
The patent generates multiple virtual holding operation images through computer graphics instead of performing multiple physical holding operations. Each virtual image represents a different holding position or orientation, providing diverse training data without the time penalty of actual robot operations
Solution Approach 2:
The system pre-generates diverse holding operation images through computer graphics before actual robot learning begins. This preliminary data preparation allows the robot to learn from a comprehensive set of virtual examples, improving learning efficiency and reducing the time needed for actual holding operations
3Reliability
If multiple captured images are used for learning to improve recognition and holding performance, then the learning quality is improved, but the complexity of data collection and processing increases
Solution Approach 1:
The patent replaces complex physical data collection with computer graphics generation. Virtual images are created through software rendering rather than physical camera positioning and object manipulation, simplifying the data collection process while maintaining the diversity needed for high-quality learning
Solution Approach 2:
The patent substitutes mechanical image capture systems with computer graphics rendering. Instead of using physical cameras, tripods, and manual positioning, the system uses software to generate images, eliminating the mechanical complexity of data collection apparatus while maintaining learning quality
Data Source
AI summary
An information processing apparatus includes a first obtaining unit configured to obtain a holding position and orientation of a manipulator when holding of a target object is performed and holding success or failure information of the target object in the holding position and orientation, a second obtaining unit configured to obtain an image in which the target object is imaged when the holding of the target object is performed, and a generation unit configured to generate learning data when the holding of the target object by the manipulator is learnt on a basis of the holding position and orientation and the holding success or failure information obtained by the first obtaining unit and the image obtained by the second obtaining unit.