The invention relates to the technical field of
artificial intelligence, can be applied to the field of
medical health, and discloses a supervised
fine tuning method, device and equipment based on a
large model and a medium. The method comprises the steps that multi-
modal training data composed of texts, images and audios are acquired and subjected to standardized preprocessing; constructing
a domain knowledge graph containing entity nodes and relation paths; carrying out
feature fusion on the multi-
modal data by adopting an attention mechanism; logic constraint conditions are generated based on entity relations in the
knowledge graph, and the logic constraint conditions and the fusion features are jointly input into the
large model to jointly optimize parameters; and finally, dynamically adjusting
model parameters through an adversarial training framework of the generator and the
discriminator, and improving the understanding ability of the model to a complex logic relationship in combination with a
reinforcement learning strategy. According to the method, the problems of lack of knowledge logic constraints in the target domain and poor multi-
modal semantic
collaboration are effectively solved, and the
joint analysis capability of the model on cross-
modal data and the reasoning accuracy of the model on complex tasks are remarkably improved.