The invention discloses a three-dimensional medical image visual
language model pre-training method for reconstructing a
hybrid strategy, which belongs to the technical field of medical image calculation and comprises the steps of constructing a medical image text pair
data set, a language text
mask reconstruction strategy, a visual image
mask reconstruction strategy, a semantic
perception fusion strategy and multi-task joint learning. According to the method, the large
language model is finely adjusted, the diagnosis and attribute information in the
medical report is extracted by using the finely-adjusted large
language model, the efficient prompt is generated, and the large language model has relatively high generalization ability, so that the cost of
manual annotation is greatly saved; according to the semantic
perception fusion strategy, the text feature obtained by the text
encoder and the image feature obtained by the image
encoder are combined to obtain the new text feature, so that the text perceives the diagnosis and attribute information of the image in advance, the alignment of the image and the text in the embedding space is further optimized, and the pre-training efficiency is improved.