Pathological classification method and system based on multi-modal deep learning

Abstract

The invention provides a pathological classification method and system based on multi-modal deep learning. The method comprises: extracting pre-selected attributes from the electronic medical recordsto serve as feature representation vectors of structural data, randomly discarding the feature representation vectors according to a preset proportion after being averagely amplified, and replacing the discarded parts with numbers 0 to serve as medical record feature vectors of the structural data in the electronic medical records; obtaining a histopathology image corresponding to the electronic medical record, performing global average pooling on the feature map of each convolutional layer of the convolutional neural network, and splicing the feature maps into a one-dimensional vector to serve as a rich image feature vector of the histopathology image; and splicing the image feature vector and the medical record feature vector together to obtain a multi-mode fusion vector, and inputting the multi-mode fusion vector into a full connection layer to obtain a binarized pathological classification result. The technical problem that the accuracy of pathological benign and malignant classification through single-mode feature representation is not high is solved.

Description

technical field [0001] The present invention relates to the field of computer vision and image processing in computer technology, in particular to a pathological classification method and system based on multimodal deep learning. Background technique [0002] Cancer is an important worldwide public health problem. Among all cancer types, breast cancer is the second most common cancer in women. Additionally, breast cancer has a very high mortality rate compared to other types of cancer. Despite the rapid development of medical science, pathological image analysis is still the most widely used method in breast cancer diagnosis. However, the dramatic increase in complexity and workload of histopathology images makes this task time-consuming, and its results are susceptible to pathologist subjectivity. In the face of this problem, the development of accurate breast cancer automatic diagnosis method is a very urgent need in this field. [0003] In recent years, deep learning ...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the deficiencies of the above-mentioned prior art, the present invention proposes a breast cancer classification method based on multimodal deep learning, which solves the problem of inaccurate classification of benign and malignant breast cancer based on single-modal feature representation in the prior art. high technical issues

Images