Multi-modal breast cancer classification training method and system based on graph attention network

Abstract

The invention discloses a multi-modal breast cancer classification training method and system based on a graph attention network, and the method comprises the following steps: firstly, carrying out the pathological feature extraction and text processing of an electronic medical record, forming a medical record text, and obtaining a text feature through a pre-training model; meanwhile, performing high-order feature extraction on the patient pathology image set by using a graph attention network; thirdly, fusing the obtained image, text and pathological features through a multi-modal adaptive gating unit to obtain multi-modal fusion features of the patient; and finally, inputting the fused multi-modal features into a multi-layer perceptron for classification prediction, and defining a cross entropy loss function training model. According to the method provided by the invention, the features of the image, the text and the pathology are fused to classify the breast cancer, the performance of the proposed network structure is obviously superior to that of a single-mode method, and the purpose of improving the breast cancer classification accuracy is achieved.

Description

technical field [0001] The invention belongs to the fields of deep learning and disease classification, and in particular relates to a multimodal breast cancer classification training method and system based on a graph attention network. Background technique [0002] Breast cancer is one of the most serious diseases threatening human life and health, and it is a medical and health problem of common concern all over the world. According to the data released by the International Center for Research on Cancer (IARC) under the World Health Organization (WHO) in 2020, the number of new cases of breast cancer reached 2.26 million, exceeding the 2.2 million cases of lung cancer. Breast cancer replaced lung cancer and became the first in the world. big cancer. Breast cancer can occur in both men and women, and more than 98% of breast cancer patients are women. The incidence of breast cancer ranks first in the world, and its incidence rate is increasing year by year and the trend o...

AI Technical Summary

Problems solved by technology

However, these methods still have some shortcomings: (1) A patient may contain multiple pathological images of various parts of breast cancer, and there are some interactions between these images

(2) Most of the existing research uses pathological images as the input of convolutional neural network, but it is difficult to meet the requirements of clinical diagnosis for the classification of benign and malignant breast cancer by considering only single-modal image data

(3) There are correlations between the data of different modalities, and the simple fusion method will not give full play to the complementarity between the modalities

Images