The invention relates to the technical field of
deep learning and defect detection, and discloses a multi-
modal fusion defect
perception and identification method based on
deep learning, and the method comprises the steps: 1, obtaining multi-
modal data: obtaining the multi-
modal data of a target object through a collection device, and obtaining the multi-
modal data of the target object; comprising visual data and non-visual data, and the
data format covers two-dimensional images, three-dimensional point clouds,
time series data and the like. According to the multi-modal fusion defect
perception and identification method based on
deep learning, the advantages of visual data and non-visual data can be integrated through multi-
modal data fusion, the characteristics of a target object are described more comprehensively,
information loss caused by single-
modal data is reduced, the accuracy of defect detection is improved, and in industrial part detection, the detection efficiency is improved. By fusing the image and the
point cloud data, defects such as cracks and holes on the surface of the part can be identified more accurately, and the detection precision is obviously improved compared with that of a single-mode method.