The invention discloses an improved multi-
modal three-dimensional medical image classification method based on fusion assistance. The method comprises the following steps: 1, inputting multi-
modal medical image data into ResNet to extract
modal features and fuse the modal features; 2, constructing a multi-
branch network, inputting a fusion feature into a main classification
branch, and extracting a cross-plane global context and a fine-grained feature in combination with cross-plane key
slice selection and Transform; 3, introducing
discriminant prior knowledge generated by the main classification
branch into a fusion auxiliary branch, and extracting enhanced features; 4, fusing the
main branch fine features and the auxiliary branch enhanced features to obtain semantic
level fusion features; 5, inputting the fused features into a classifier to output category probabilities, and taking a category corresponding to the maximum value as a diagnosis
label; and 6, joint loss is constructed based on prediction and real labels, and the multi-branch multi-
task network is trained and optimized. According to the method, complementary information of the multi-modal medical image is fully mined, focus
perception is enhanced by combining judgment prior guidance and multi-task collaborative optimization, and the diagnosis accuracy and stability are improved.