A First View Fingertip Detection Method Based on Convolutional Neural Network and Heat Map

Abstract

The invention discloses a first view fingertip detection method based on a convolutional neural network and a heat map, which includes the following steps: collecting a gesture image, marking the position of the bounding rectangle of the gesture and the coordinates of the fingertip, and cutting the original gesture image and updating the fingertip position through the bounding rectangle to generate a fingertip heat map; designing a gesture detection convolutional neural network, extracting gesture features, and training the network using the image before cutting and the bounding rectangle to make the network converge; designing a fingertip heat map regression convolutional neural network, extracting fingertip features, and training the network using the image after cutting and the heat mapto make the network converge; and segmenting an input first view video into frames, using the trained gesture detection convolutional neural network to get the bounding rectangle of the gesture, cutting out the gesture part and inputting the gesture part to the fingertip heat map regression convolutional neural network to predict the heat map of the fingertip, and getting the coordinates of the fingertip according to the heat map. The position of the fingertip can be accurately detected in complex backgrounds and under different light conditions.

Description

technical field [0001] The present invention relates to the technical fields of computer vision and machine learning, in particular to a first-view fingertip detection method based on convolutional neural networks and heat maps. Background technique [0002] In recent years, the increasingly mature theory and technology of human-computer interaction, expanding application fields, and huge commercial value and development potential have made the field of human-computer interaction attract more and more researchers' attention and the injection of fresh blood. . And with the rapid development of computer technology. The focus in the field of human-computer interaction has gradually shifted to making human-computer interaction more intelligent, improving the interaction ability between computers and users, improving the interaction methods, allowing users to "communicate" with computers more conveniently, thereby improving the efficiency and experience of human-computer interac...

AI Technical Summary

Problems solved by technology

The traditional method is easily disturbed by factors such as the quality of gesture data, background color, and lighting conditions, and the performance of the algorithm has encountered a bottleneck period. In addition, gesture recovery has high requirements for real-time performance. Therefore, the practicability of traditional finger tracking algorithms has been reduced. increasingly unsatisfactory users

Images