The invention relates to an RGB-D image classification method and system. The method comprises the steps of: S1, utilizing a convolution neural network (CNN) to process a source RGB image and a Depth image respectively, and extracting low level characteristics; S2, utilizing a recursion neural network (RNN) to carry out feedback learning on the image low level characteristics, and extracting image middle level characteristics; S3, adopting a block interior constraint dictionary learning method, carrying out characteristic set sparse expression on the image middle level characteristics, and obtaining high level characteristics of the RGB-D images; and S4, inputting the high level characteristics of the RGB-D images into a linear SVM to complete the classified identification of the RGB-D images. According to the invention, automatic characteristic extraction of the images is realized, learning RGB-D image characteristic expressions can effectively distinguish classification of noise data from high similarity images, and the classification precision of the RGB-D images is improved; in addition, the linear SVM is utilized, and the image classification speed is improved.