The application discloses an intestinal
polypus identification method based on images and multi-
modal data, comprising the following steps: acquiring a plurality of
microscope pathological images, objective
magnification information corresponding to the
pathological images, and text information corresponding to the
colonoscopy; performing data preprocessing on each frame of acquired
pathological images and text information to obtain a training
data set; constructing a multi-scale multi-
modal feature classification network, training the network using the training
data set, and obtaining a trained multi-scale multi-
modal feature fusion classification network; collecting
microscope pathological images of a patient, acquiring corresponding
colonoscopy text information, inputting the data after data preprocessing into the multi-scale multi-modal
feature fusion classification network, and obtaining a
class prediction of the
disease by the model. The application fully fuses pathological images and
colonoscopy text information, finally realizes accurate classification and prediction of the
disease type, improves the effect and efficiency of multi-modal diagnosis of intestinal diseases, and thus improves the accuracy of diagnosis of the patient.