The invention discloses a
psychological health support dialogue generation method based on multi-
modal input. The method comprises the steps that 1, text data are acquired and preprocessed; 2, calculating context similarity to obtain a high-quality example set; 3, extracting key visual clues from the image by utilizing a visual
language model, wherein the key visual clues comprise facial expressions, actions and emotions; 4, integrating an external common
knowledge base to understand the emotion and intention of the user; 5, generating a consultation strategy by using a large
language model; and 6, dialogue generation in combination with the context. According to the method, the visual information, the dialogue context and the consultation strategy are comprehensively utilized, the performance of a large
language model in the
psychological health dialogue is enhanced, more detailed and similar supportive replies can be generated, the
emotion recognition and response quality are remarkably improved, the emotion state of a user can be comprehensively understood and responded, and the user experience is improved. Therefore, the
mental health support effect is effectively improved.