The present application relates to the technical field of
artificial intelligence and psychology, and particularly relates to a digital sand table multi-
modal data analysis method based on an intelligent chat
robot, which comprises the following steps: constructing a knowledge-enhanced large
language model architecture; using
synonym replacement, scene generalization and dialogue
rewriting strategy data enhancement on three types of
training data sets; executing four-stage model training; synchronously collecting sand tool operation,
facial expression, behavior and voice response data based on the large
language model, synchronously integrating the multi-
modal data sets according to timestamps, training a psychological analysis model; based on the explainability analysis of the screening results of the psychological analysis model, extracting the model attention weight and feature importance
score, and performing attribution analysis on the sand table image, audio and video data to generate a text analysis report. The method can deeply mine subjective psychological motivation through dynamic interaction, realize deep fusion analysis of multi-
modal data, and provide an explainability analysis report, and is suitable for depression
risk screening of a large number of people.