The invention discloses a natural interactive method based on three-dimensional gestures, the method utilizes a computer vision technology to obtain local features of a hand by foreground segmentation and fingertip detection, and the local features include fingertip position, palm contour, palm center position and the like. By adopting the stereoscopic vision technology, the hand features such as the fingertip position, the palm center position and the like are reconstructed in the three-dimensional space. The finger tip position, the palm center position and the like in the three-dimensional space are parameterized, and a three-dimensional interactive model based on points, lines and planes is defined, thus realizing various three-dimensional gestures in the three-dimensional space, such as fingertip clicking, fingertip squeezing, palm overturning, fingertip directing and the like. The method needs only two ordinary network cameras to meet the demands of real-time man-machine interaction.