The invention discloses a medical
image analysis method and
system based on a visual
language model, and belongs to the technical field of medical image intelligent diagnosis, and the
system comprises an image preprocessing unit which carries out the down-sampling of an original
retina OCT image to 256 * 256 and carries out the normalization of the original
retina OCT image; the feature encoding unit comprises an image
encoder based on RET Found in combination with LoRA optimization and a text
encoder based on BioClinicalBERT; the class balance comparison
learning unit is used for adjusting loss through class balance coefficients so as to relieve the
class imbalance problem; the
uncertainty estimation unit is used for calculating confidence quality and uncertainty scores based on
Dirichlet distribution, and determining a threshold value in combination with an improved Youden index; and the model training unit adopts a total
loss function of class balance loss and uncertainty loss, outputs a diagnosis result and an uncertainty
score through transfer learning, and further comprises an image input module, a result display module and a data storage module.
Rare disease classification performance and reliability are improved, training efficiency is improved through LoRA optimization, and an accurate and reliable scheme is provided for detection of the rare
retina diseases.