First view fingertip detection method based on convolutional neural network and heat map

Abstract

The invention discloses a first view fingertip detection method based on a convolutional neural network and a heat map, which includes the following steps: collecting a gesture image, marking the position of the bounding rectangle of the gesture and the coordinates of the fingertip, and cutting the original gesture image and updating the fingertip position through the bounding rectangle to generate a fingertip heat map; designing a gesture detection convolutional neural network, extracting gesture features, and training the network using the image before cutting and the bounding rectangle to make the network converge; designing a fingertip heat map regression convolutional neural network, extracting fingertip features, and training the network using the image after cutting and the heat mapto make the network converge; and segmenting an input first view video into frames, using the trained gesture detection convolutional neural network to get the bounding rectangle of the gesture, cutting out the gesture part and inputting the gesture part to the fingertip heat map regression convolutional neural network to predict the heat map of the fingertip, and getting the coordinates of the fingertip according to the heat map. The position of the fingertip can be accurately detected in complex backgrounds and under different light conditions.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine

Login to view more

AI Technical Summary

Benefits of technology

This patented technology allows users to annotate their positions onto images they are editing or painting them without being affected from complicated backgrounds like sunlight or shade. It uses artificial intelligence techniques called convolutional Neural Networks to quickly process this data into an accurate representation of these annotations. Additionally, there exist methods such as thermal mapping used during rendering processes to improve upon the precision of touch recognition. Overall, this technology helps create high-quality visual experiences through animation tools and digital cameras.

Problems solved by technology

This patented technical problem addressed in this patents relates to how humans interact better through their sense of touch or hand movements compared to other interactions like typing characters over words alone. It suggests that there are many ways to improve these processes, including detecting fingernails from images captured during image processing techniques (such as depth cameras). However current technologies require significant amounts of time and resources due to complicated environments caused by ambient lights and shadows. Additionally, existing approaches may also cause issues when used without proper training.

Images