A robotic arm grasping result detection method based on visual haptics and attention mechanism

By employing a feature extraction and two-stage fusion method based on visual-tactile and attention mechanisms, the problem of neglecting the relative importance of visual and tactile information between modalities is solved, resulting in more accurate evaluation of grasping results and improved stability.

CN118397371BActive Publication Date: 2026-07-24SICHUAN UNIV
0 Cites 0 Cited by

Patent Information

Application Number
CN202410620601.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-17
Publication Date
2026-07-24
Estimated Expiration
2044-05-17

AI Technical Summary

Technical Problem

Existing technologies struggle to accurately acquire the pose information of the object being grasped, neglect the relative importance of visual and tactile information between modalities, and lack sufficient spatiotemporal feature fusion methods, leading to instability in the grasping process.

Method used

A robotic arm grasping result detection method based on vision-touch and attention mechanism is adopted. The feature extraction module obtains the spatiotemporal features of vision and touch, and the two-stage fusion module performs cross-modal and cross-spatiotemporal feature fusion. The grasping result is predicted by combining a fully connected neural network.

Benefits of technology

It improves the accuracy and stability of grasping result evaluation, enhances the adaptability of the model, effectively extracts detailed and pose information from the visual-tactile sequence, and improves the coherence of the grasping process.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The application discloses a kind of based on visual tactile sensation and attention mechanism's mechanical arm grabbing result detection method, and grabbing result discriminant model according to visual-tactile sequence real-time data set, output article grabbing result information, the generation process of model includes: S1, the visual-tactile sequence initial data set of the article that mechanical arm grabs is acquired, data set is preprocessed and data is enhanced, and then it is divided into training set and test set;S2, training set is input into feature extraction module, obtain the space-time feature based on visual tactile sensation and input into two-stage fusion module, obtain fusion feature and input into grabbing result prediction module to train, obtain grabbing result discriminant initial model;S3, repeat step S2, obtain multiple grabbing result discriminant initial models and input test set, output the grabbing result discriminant intermediate model of highest accuracy as grabbing result discriminant model.Compared with prior art, the adaptability of grabbing model and the accuracy of grabbing result are effectively improved.
Need to check novelty before this filing date? Find Prior Art