Grabbing task-oriented visual relationship understanding method and system

The proposed method addresses the inefficiencies in robotic grasping by using a deep learning-based, Transformer-based model to understand scene semantics and determine optimal grasping sequences, improving grasping efficiency and naturalness while reducing computational complexity.

CN119658684AActive Publication Date: 2025-03-21HUAZHONG UNIV OF SCI & TECH
5 Cites 0 Cited by

Patent Information

Application Number
CN202411798826.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-09
Publication Date
2025-03-21
Estimated Expiration
2044-12-09

AI Technical Summary

Technical Problem

Existing methods for robotic grasping tasks focus primarily on identifying grasping locations without adequately understanding scene semantics and fail to prioritize the order of object manipulation when the target object cannot be directly grasped, leading to inefficiencies and complexities in real-time operations.

Method used

A method involving deep learning techniques, including a Transformer-based target detection model with an encoder-decoder architecture, to extract scene features, filter irrelevant object pairs, and generate a relationship matrix for determining optimal grasping sequences, which simplifies the grasping process by focusing on semantic understanding and prioritizing relevant object interactions.

Benefits of technology

This approach enhances grasping efficiency and naturalness by eliminating the need for complex spatial relationship descriptions and reduces computational load in complex scenes, improving the overall system's efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to a grabbing task-oriented visual relationship understanding method and a grabbing task-oriented visual relationship understanding system. The grabbing task-oriented visual relationship understanding method comprises the following steps of: acquiring a scene image containing a plurality of objects; performing feature extraction on the scene image based on a deep convolutional network to obtain a basic feature layer of the scene image; inputting the basic feature layer into a target detection network, and obtaining classification information and position information of each object; the classification information and the position information are mapped to a basic feature layer to obtain a region of interest of each object, irrelevant object feature pairs are removed through a relation filter, and relevant object feature pairs are obtained; inputting the object feature pairs into a visual relation detection network to obtain a relation matrix; and the object grabbing sequence is determined by combining the grabbing task and the relation matrix, and the target object is grabbed according to the object grabbing sequence. According to the method, the reasonable grabbing sequence is obtained by understanding the visual relation of the multiple target objects in the complex scene, and the safety and reliability of the robot in the grabbing operation process are guaranteed.
Need to check novelty before this filing date? Find Prior Art