Uncalibrated human-computer interaction control system and method based on Kinect

Abstract

The invention discloses an uncalibrated human-computer interaction control system and method based on Kinect. The system comprises human skeleton information acquisition, a computer software system and a mechanical arm, wherein human depth image information acquisition is mainly accomplished by a Kinect sensor; the computer software system processes acquired field depth data by utilizing a skeleton tracking technology and establishes a 3D (three-dimensional) coordinate of 20 human skeleton points; and a control system of the mechanical arm acquires a control command sent by conversion of the computer system, and controls an end actuator to follow hand motion to achieve real-time guide of a hand to the end actuator of the mechanical arm. According to the system and the method, master-slave control is performed by constructing a motion mapping relation of the hand relative to a hipbone center and the end actuator relative to a base of the mechanical arm, so that real-time interaction of the hand and the mechanical arm is achieved. An experimental result indicates that the system can well accomplish hand tracking and master-slave control tasks, and has higher timeliness and interactivity.

Description

technical field [0001] The invention belongs to the technical field of information barrier-free engineering, and in particular relates to a Kinect-based non-calibration human-computer interaction control system and method. Background technique [0002] The master-slave teleoperated robotic arm control system is widely used in high-risk operations such as nuclear reactor maintenance, manned spaceflight, and medical surgery simulation training. The hand is the most commonly used part of the human body. Therefore, realizing the robot arm to follow the movement of the hand can greatly improve the human's perceptual ability and behavioral ability. It is of great theoretical and practical significance to study the various movements of the hand and the interaction between the hand and the robotic arm. [0003] At present, there are mainly two types of existing hand tracking and positioning technologies, which are based on data glove technology and gesture recognition based on imag...

AI Technical Summary

Problems solved by technology

However, color images are easily affected by lighting conditions, and traditional vision-based tracking of moving objects requires calibration of camera internal and external parameters, which cannot be accurately calibrated in actual situations

Images