The invention discloses a gesture classification method based on transfer learning. The gesture classification method is suitable for gesture image classification. The method comprises the steps of 1,converting a gesture video V into a gesture frame data set G0; 2, performing noise removal, binarization and background segmentation processing on the G0 through a Gaussian filtering method, an OTSUalgorithm and image AND operation to obtain a gesture frame data set G1, and setting a label for the G1 to obtain a frame label data set L; 3, carrying out transfer learning by using a MobileNet convolutional neural network architecture and the weight file, and creating and training a model M1; 4, extracting features of the frame data set G1 through the model M1 to obtain a frame feature vector set F0; and 5, classifying the test set by taking the XGBoost as a classification model to obtain a final classification result. According to the method, the weight of the trained MobileNet convolutional neural network is migrated to a gesture image data set for feature extraction, and XGBoost is adopted as a classification model, so that the model calculation amount is reduced while the classification accuracy is improved.