The invention discloses a metal surface defect image recognition non-destructive testing device and method, and relates to the field of automatic metal surface defect testing. The device comprises a mobile unit, a light sensation unit, a light source unit, an optical amplification unit, an image acquisition unit, an image analysis unit, a data storage unit and an alarm unit, wherein the light sensation unit and the light source unit ensure suitable brightness of an acquired image in a tested area, the optical amplification unit is used for optically amplifying the acquired image, the image acquisition unit acquires the image, the image analysis unit analyzes the acquired image in the tested area and judges whether defects exist or not, the data storage unit stores data needed by running ofthe device, the acquired image and analysis results, and the alarm unit gives an alarm for a tested problem area. Metal surfaces can be full-automatically tested, the tested area is large, the testing method is simple and rapid, and testing time and labor cost are reduced.