Gesture depth image continuous detection method using multi-head mask balance fusion unit

Abstract

The invention discloses a gesture depth image continuous detection method using a multi-head mask balance fusion unit. The method includes: capturing a depth image sequence dynamically changed by a human hand through a depth camera; gridding each frame of projection three-dimensional space point cloud as an input grid of the frame to form a grid sequence; traversing each frame, and inputting the frames into a volume convolutional neural network single-frame detection model for processing to obtain hand joint pseudo-Gaussian spatial distribution of each frame; selecting hand joint pseudo-Gaussian spatial distribution of a previous frame, inputting the hand joint pseudo-Gaussian spatial distribution into a multi-head mask equalization fusion unit to obtain a mask of a current frame, and thenperforming mask superposition on the mask of the current frame and an input grid; inputting the multi-frame detection model of the volume convolutional neural network to obtain hand joint pseudo-Gaussian spatial distribution of the current frame; repeating the steps, and converting the hand joint pseudo-Gaussian spatial distribution of each frame into three-dimensional coordinates of the hand bone joint points. According to the method, the hand joint pseudo-Gaussian space distribution is introduced, so that the gesture detection stability is greatly improved.

Description

technical field [0001] The invention belongs to a gesture depth image recognition and detection method in the technical field of image automatic recognition, and more specifically relates to a gesture depth image continuous detection method based on a multi-head mask equalization fusion unit. Background technique [0002] Gesture detection is the cornerstone technology in many gesture human-computer interaction fields, such as gesture servo of robotic arm, virtual reality and augmented reality interaction. The single-frame gesture detection solution based on the monocular depth camera to capture the depth image of the human hand and the supervised learning of the deep convolutional neural network has become the mainstream method for the coordinate detection of each joint of the gesture. However, the existing single-frame gesture depth image detection and recognition methods, due to the high-dimensional complexity of gesture space expression and the problems of self-occlusion...

AI Technical Summary

Problems solved by technology

[0003] The purpose of the present invention is to provide a gesture depth image sequence that relies on the quasi-Gaussian spatial distribution of hand joints to apply a multi-head mask mechanism to the spatial position relationship for the existing single-frame gesture detection method. Detection method, the method greatly improves the stability of gesture detection by introducing the quasi-Gaussian spatial distribution of hand joints

Images