Method for tracking deformable hand gesture based on video streaming

Abstract

The invention discloses a method for tracking deformable hand gesture based on video streaming, comprising the steps of: obtaining a frame image, and extracting a sub-image containing a human hand from the obtained frame image; selecting feature tracking points from the sub-image containing the human hand, and initializing a continuously self-adaptive mean shift tracker by the sub-image containing the human hand; performing optical flow calculation on the selected feature tracking points to serve as a local tracking result, and synchronously overall tracking the human hand by the continuouslyself-adaptive mean shift tracker to obtain a global tracking result; updating the feature tracking points; and adopting the result of the optical flow tracking as the final output result of the deformable hand gesture. The method for tracking the deformable hand gesture based on video streaming can be used for tracking the human hand with randomly deformable hand gesture and enabling human-computer gesture interaction to operate in a more comfortable manner. According to the invention, the tracking can be performed aiming at the randomly deformable hand gesture, the interference from change of a background and a large area of complexion is eliminated, and the real-time robust hand gesture tracking is achieved.

Description

technical field [0001] The invention relates to technical fields such as image processing, pattern recognition, and human-computer interaction, and in particular to a method for tracking deformation gestures based on video streams. Background technique [0002] The human-computer gesture interaction based on computer vision, its core is to use computer vision, image processing, pattern recognition and other technologies to detect, track and recognize the frame images in the video sequence collected by the camera, get various gestures of people, and Comprehend and respond to the behavior indicated by the gesture. In recent years, with the continuous development of computer vision and pattern recognition technology, gesture interaction technology has been applied to TV remote control, video games, video conferencing and other fields, bringing great convenience to people's life and entertainment. [0003] Gesture interaction based on computer vision is a key technology in the ...

AI Technical Summary

Problems solved by technology

First of all, how to deal with changing environments, such as light, background, viewing angle, etc., is also a common problem in the field of computer vision; secondly, how to deal with various deformations of the human hand, which is one of the most flexible organs and has a high degree of freedom Whether the human hand can recognize various gestures is a key issue. In the process of using gestures to interact, if the shape and movement of the human hand are restricted too much, it will definitely cause problems for the user. It brings discomfort and does not satisfy the original intention of nature, comfort and harmony

Images