Method and system for detecting grabbable points of target object based on freehand sketch

Abstract

The invention relates to a target object grabbable point detection method and system based on a freehand sketch, and the method comprises the steps: obtaining a natural scene picture set marked with a grabbing frame and a freehand sketch sample set corresponding to an object in a scene, and carrying out the preprocessing of a sample; matching the sketch with the grabbable frame of the target object, and constructing a training set; constructing a grabable box detection network based on the sketch content as guidance; training a grabbable box detection network based on sketch content guidance; and using the trained grabbable box detection network to position a grabbable box corresponding to an object conforming to the sketch content in the natural scene picture. Compared with the prior art, the method has the advantages that the robot is guided to complete the target object grabbing task based on the sketch, the network convergence is fast, and the generalization ability is high.

Description

technical field [0001] The invention relates to the technical field of robot visual grasping, in particular to a method and system for detecting graspable points of objects based on hand-drawn sketches. Background technique [0002] Compared with the description form of natural language, hand sketching is a more intuitive way for humans to describe the objects they see. Sketch is also an abstract form of expression based on the understanding of semantic information of objects. At present, most of the existing technologies have enabled robots to understand natural language and interact appropriately with humans, but they still cannot be applied to some scenes in real life that cannot be expressed in language. For example, in the case of aphasic patients who cannot form language, who know what they want but cannot verbally describe the object; Sudden forgetting the name of an object, but still able to know the appearance of the object, etc. If the robot can understand hand-...

AI Technical Summary

Problems solved by technology

However, this retrieval method can only filter the most similar pictures from the image library (the pictures usually only contain prominent main objects), and the actual robot grasping application often needs to grasp the target objects in the scene. area, and the scene usually contains multiple randomly placed objects

In terms of grasping point detection, unlike the existing grasping point detection methods, most of the existing methods directly detect the graspable points of all objects in the scene. The detected graspable points are relatively messy, and often encounter Situations that the robot does not recognize

Images