Multi-model fusion video hand division method based on Kinect

Abstract

The invention provides a multi-model fusion video hand division method based on Kinect, which comprises the following steps of (1) capturing video information, (2) dividing images in a video respectively to obtain division results, namely a depth model, a skin color model and a background model, (3) calculating an overlapping rate of every two division results as a characteristic of judging division effects of the results and inputting the three overlapping rates into a neural network, (4) allowing the neural network to output three coefficients (namely confidence coefficients) showing respective reliability of the three models, and weighting the three division results with the confidence coefficients, (5) conducting linear superposition on the weighted division results of the three models, (6) outputting a final binary image of a superposed result through a threshold function and finally dividing an obtained video hand region, and (7) updating the background model, wherein the division results are expressed as binary images. The method has the advantages of low cost, good flexibility and the like.

Description

technical field [0001] The invention relates to a computer image processing and pattern recognition technology, in particular to a Kinect-based multi-model fusion video hand segmentation method. Background technique [0002] Video hand segmentation is an important field of computer vision and artificial intelligence. As a basic step in a series of human-computer interaction applications such as gesture remote control, virtual writing and painting, sign language recognition, etc., it has an important impact on subsequent steps. Traditional video hand segmentation methods can be divided into two main areas: segmentation methods based on 2D vision and segmentation methods based on 3D hand modeling. Previous researchers have their own limitations in the attempts of 2D vision methods. sex. This includes the widely used skin tone model, but it is susceptible to lighting changes and cannot resolve the overlap between human hands and skin-like regions. On the other hand, the frame...

AI Technical Summary

Problems solved by technology

However, the depth information provided by Kinect has defects such as noise and inaccurate edges.

Therefore, simply using depth information often does not get good segmentation results. Some researchers combine skin color model and depth model to build a more robust segmentation model. However, due to Kinect's own imaging principle, these two information are out of sync. Therefore, there is a certain delay between the color image and the depth image, and how to register and make up for the non-overlapping parts has become a new problem

Images