The invention relates to a gesture recognition method and a gesture recognition device. The gesture recognition method comprises a training step and a recognition step, wherein the training step comprise the sub-steps of: S1, synchronously acquiring a to-be-trained image with depth information and color information; S2, based on the depth information, determining a primary hand outline; S3, based on color information, determining a precise hand outline; S4, calling the depth information and the color information of the precise hand outline, determining a weighted average by self-adaption weighting to establish a three-dimensional gesture model, and training the three-dimensional gesture models of a plurality of to-be-trained images by using a classifier method to obtain an optimized three-dimensional gesture model. The recognition step is the same as the extraction mode of hand outline, and the optimized three-dimensional gesture model can be obtained according to S4, and the corresponding three-dimensional gesture is recognized. By using the depth information and the color information for recognition, the method has the advantages of high accuracy and high precision.