The present invention discloses an image recognition-based facial skin analysis system. The image recognition-based facial analysis system is capable of conducting a series of evaluation on the condition of the skin to obtain an overall evaluation. The system mainly comprises a face detection and segmentation module, a whitening degree detection module, a roughness detection module, a stain amount detection module and a comprehensive judgment module. Firstly, a detection image is input to be subjected to front face detection, area location and segmentation by a classifier. Secondly, the segmented image is subjected to skin color brightness and ruddy condition analysis, wherein the roughness of a feature index quantification detection area is obtained based on a gray level co-occurrence matrix. Thirdly, the stain amount is analyzed based on the similar watershed method through calculating the sum of sectional areas. Finally, the comprehensive judgment is conducted according to all detection results. Based on the technical means, the application effect of a product is quantified, so that the quality of the product is controlled. The system has great economic value for cosmetic manufacturers. Meanwhile, the system brings more economic benefits.