The invention relates to a data classification system and method based on KL divergence optimizationmethod for classifying data based on KL divergence optimization. The method comprises the steps thatdata preprocessing is conducted on original images, t, texts and other data, and objects are modeled into multi-dimensional distribution; S; selecting a certain amount of triple from the tagged training data to carry out model training; T; the selected triple serves as training data, a linear mapping A is applied to all the mean vectors, t, the optimal linear mapping is learned through iterativeoptimization, and the learning process is based on the basic assumption of metric learning, t, that is, t, the distance between samples of the same kind becomes smaller, and the distance between samples of different kinds becomes larger; A; an intrinsic gradient descent algorithm is adopted for optimization, and after the gradient of an objective function is projected to the tangent space of the same manifold, Riemannian gradient descent is executed on the manifold of an SPD matrix with given affine invariant Riemannian metric; A; and calculating the KL divergence between the test set and thetraining set, and classifying the samples by adopting a K-nearest neighbor (KNN) classifier. The method can effectively improve the classification precision of the system, and has more stable performance.