The invention discloses a multi-
modal brain disease diagnosis method based on double contrast learning, which comprises the following steps of: acquiring multi-
modal brain image data containing
functional magnetic resonance imaging (fMRI) and
diffusion tensor imaging (DTI) and corresponding diagnosis text description, and constructing a multi-
modal brain disease data set; establishing a multi-modal
brain disease diagnosis
model architecture based on double contrast learning, wherein the architecture comprises a multi-modal contrast learning stage and a cross-modal
recovery learning stage; a two-stage training strategy is adopted, firstly, a complete sample is used for multi-modal contrast learning training, then, an incomplete sample is used for cross-modal
recovery learning training, and the training process is optimized through joint optimization of contrast loss, reconstruction loss and classification loss; and finally, performing modal
recovery and
feature fusion on the to-be-diagnosed sample with the missing modal through the training model, and outputting a brain
disease diagnosis result. According to the method, the accuracy and robustness of brain
disease diagnosis are remarkably improved, the adaptability to multi-
modal data under different missing rate conditions is enhanced, and the technical problems that in the prior art, the diagnosis performance is reduced due to modal missing,
semantic consistency of recovery features is lacked, and the model generalization ability is insufficient under the high missing rate condition are solved.