The invention provides a multi-
modal large
language model training method,
system and device and a storage medium, and belongs to the technical field of biological
medicine. The invention provides a method for training a multi-
modal large
language model in the field of biological
medicine. The method comprises an entity recognition module, a
hypertext embedding module, a lexical element division module and a model training module. According to the training method provided by the invention, names or anaphora of genes, proteins and molecules are automatically identified from natural text descriptions of literatures and patents by adopting a general large
language model, then embedding is carried out, identification is carried out on the basis of real natural texts in the literatures instead of a mode of artificially constructing sentences from data, and thus the identification efficiency is greatly improved. And more natural
modal fusion can be realized. According to the method, the matched multi-modal sequence data is embedded into the text according to the marking rule, so that the original text
semantics are not damaged while the text information is enriched, and a better fitting effect on the data is realized.