The invention relates to the technical field of
artificial intelligence and man-
machine interaction, and discloses a picture book co-reading method,
system and device based on multi-
modal emotion recognition and a medium, and the method comprises the following steps: obtaining multi-
modal time sequence data of a user in picture book co-reading; establishing and updating a longitudinal emotion file for recording historical interaction data for the user; based on the data and the archive, personalized multi-dimensional internal state evaluation is carried out, and the evaluation comprises calculation of
cognitive load indexes representing difficult understanding, recognition of instant emotional states and analysis of contextual emotional deviations of the emotional states and current plot expectations; and finally, based on the multi-dimensional
evaluation result, determining and executing dynamic interaction strategies such as simplification, guidance or excitation. According to the method, deep
personalization is achieved by constructing the multi-dimensional
state model and combining the longitudinal archives, the
user state can be more accurately judged, prospective and self-adaptive interaction is achieved, and the interaction
fineness and effectiveness are improved.