The invention discloses a human face set matching method and system based on similarity learning. Training set samples and testing set samples are selected from original data samples, training samples are selected, the true similarity is calculated and is compared with the worked out calculation similarity, and then a final classifier is selected; the geometric mean of each type of training samples and all of testing samples in the testing set samples are substituted into the final classifier, classifying results are obtained, and then the categories of the testing samples are obtained. According to the scheme, part of samples are selected as the training samples to carry out training, classifiers are selected, and therefore it is avoided that all samples serve as the training samples to be trained, the training process is simplified, the complex process is avoided, and the training speed is improved. In addition, according to the scheme, the geometric mean of each type of samples in the training set samples is selected to construct a plurality of different classifiers, and the effect of the accurate result through the simple operating process is achieved.