Method for controlling robot by body sense, robot and control device thereof

Abstract

The embodiment of the invention relates to the technical field of visual identification, and discloses a method for controlling a robot by body sense, a robot and a control device thereof. The methodfor controlling the robot by the body sense includes the steps that a color image and a depth image containing an operator are obtained; the color image is input into a convolutional neural network toobtain two-dimensional coordinates of articulation points of the operator on the color image and the connected relation between the articulation points; the depth image and the color image are registered to obtain three-dimensional coordinates of the two-dimensional coordinates of the articulation points of the operator mapping into the depth image; the angles of joints of the operator are calculated according to the three-dimensional coordinates and the connected relation between the articulation points; and according to the three-dimensional coordinates and the angles of the joints, the robot is controlled to act following the actions of the operator. According to the method for controlling the robot by the body sense, the robot and the control device thereof, the robot can identify thepostures of the operator timely and accurately when multiple people are involved, and can simulate the postures of the operator timely and accurately.

Description

technical field [0001] Embodiments of the present invention relate to the field of visual recognition technology, and in particular to a method for controlling a robot by somatosensory control, a robot and a control device. Background technique [0002] At present, the application of visual monitoring and recognition is becoming more and more popular. The extraction of human skeleton in the fields of security, transportation and entertainment is the basis of many behavior recognition and somatosensory interactions. After the successful application of the deep network, this technology has been greatly improved on Microsoft's COCO dataset. After the large-scale application of deep learning, common methods for identifying human bodies are as follows: [0003] Existing technology 1: Use a pedestrian detection frame to select a single pedestrian in the image, and then judge the pose of the single person in the frame. However, in prior art 1, when there are too many people in th...

AI Technical Summary

Problems solved by technology

Since prior art 2 calculates globally based on all human body joints by solving the integer linear programming problem, and the integer linear programming problem is an NP-Hard problem, the average processing time for solving this type of problem is about several minutes to several hours, therefore, real-time performance cannot be guaranteed

[0005] To sum up, when using the existing human body detection method to control the robot to follow the human body's movement, it cannot timely and accurately recognize the gesture of the operator when there are many people, so it cannot be timely and accurately controlled. The robot mimics the pose of the operator

Images