Hand-eye coordination grabbing method based on human eye gaze point

Abstract

The invention belongs to the technical field of hand-eye coordination, and particularly discloses a hand-eye coordination grabbing method based on a human eye gaze point. The method comprises the steps of S1, determining the human eye gaze point by using an eye movement instrument, and determining the position of an object of interest of a user based on the human eye gaze point; S2, performing discretization traversal to obtain a feasible grabbing gesture set of a mechanical arm at each position in space and storing the feasible grabbing gesture set in a server; S3, accessing a server according to the position of the object of interest of the user so as to query and obtain the feasible grabbing gesture set of the position; S4, obtaining multiple groups of solutions of each joint angle of the mechanical arm based on inverse solutions of each grabbing posture in the feasible grabbing posture set, and determining an optimal solution from the multiple groups of solutions of the joint angles, namely obtaining the optimal joint angle corresponding to each joint of the mechanical arm; and S5, making each joint of the mechanical arm move to the corresponding optimal joint angle, and grabbing the object of interest of the user in the optimal grabbing posture. The method has the advantages of accurate grabbing and convenient operation.

Description

technical field [0001] The invention belongs to the technical field of hand-eye coordination, and more specifically relates to a hand-eye coordination grasping method based on the gaze point of human eyes. Background technique [0002] On the one hand, the gaze point of the human eye can be issued by the person effortlessly, and can convey certain information to the outside world. For example, by observing the eyes of others, it is possible to know the general gaze direction and position of the person, combined with the scene information, Objects of interest to the person can also be learned. On the other hand, for handicapped disabled people, they have the need to grab things, but cannot use traditional human-computer interaction methods. At this time, eye trackers can be used to extract gaze points and analyze their intentions to achieve simple and intuitive human-computer interaction. machine interaction. In summary, it is an urgent problem to design a system that is ea...

AI Technical Summary

Problems solved by technology

Patent US8888287B2 proposes a human-computer interaction system based on a three-dimensional gaze point tracker, which uses a depth camera and an RGB camera to extract gaze direction and gaze point for human-computer interaction. It is used for human-computer interaction, but it is not used to control the robotic arm to grab the object of interest to the user in the actual scene

Images