The invention relates to a method and device for rapidly detecting and classifying defects of a glass image, wherein the method comprises the steps of: 1, scanning windows of an input glass image, measuring according to the balance of gray level distribution in the windows to obtain candidate defect windows; 2, merging the adjacent candidate defect windows according to the position relationship of the candidate defect windows to obtain a candidate defect region; 3, obtaining background information of the candidate defect region, and extracting a defect domain according to the gray level distribution mode of the candidate defect region; and 4, normalizing the defect domain according to the scale, extracting characteristic vectors, and classifying the defects according to the characteristic vectors to obtain a defect classifying result. By adopting the method, the defects in a glass image frame containing the noise can be accurately detected, the class of the defects can be effectively distinguished and the undefined defects can be judged.