The invention discloses a fabric surface defect detection machine set, and a detection method based on the fabric surface defect detection machine set. The fabric surface defect detection machine set comprises a machine frame, a feeding roller assembly, a foreign matter removing device, a flattening device, a deviation correcting device, a drawing roller assembly, a wrinkle removing device, an industrial camera, a marking machine, and a coiling machine. According to the detection method based on the fabric surface defect detection machine set, before detection, impurity removing, flattening, deviation correcting, and wrinkle removing are adopted so as to conveying flat fabric into a detection zone steadily; the fabric surface defect detection machine set can be used for on-line detection of fabric surface, increasing fabric surface detection efficiency and fabric detection result accuracy.