The embodiment of the invention discloses a real-time cloth defect detection method and system based on deep learning, and the method comprises the steps: 1, collecting different types of cloth defect images, and constructing a defect data set; 2, performing data expansion firstly, and then performing data expansion by means of a generative adversarial network; 3, carrying out labeling processing on the expanded defect data set; 4, constructing a deep learning target detection network to perform cloth defect detection; 5, training a cloth defect detection network; and 6, capturing images of the cloth in real time by using a camera, inputting the captured images into the trained cloth defect detection network, judging whether defects exist in the images, determining the types of the defects, positioning the defects, and finally storing a result into an output file. According to the method, manual design of features can be omitted, the robustness of a defect detection system is improved, the detection performance is greatly improved, manpower can be liberated, and the intelligent degree of the textile industry is further improved.