An end-side-cloud three-in-one active chat
robot system for high-emotional merchants comprises an end side, a local
edge server and a cloud end, and the end side is used for collecting multi-
modal data of a user and performing local lightweight real-
time processing and response execution; the local
edge server is used for receiving and fusing the multi-
modal features and the context information from the end side, and carrying out sentiment calculation,
dialogue management and active trigger
decision making with medium complexity; the cloud end is used for operating a super-large-
scale model and providing global
knowledge management, long-term user portrait storage and model training optimization; and the end side, the local
edge server and the cloud end carry out cooperative communication through an encrypted channel to form a distributed intelligent
processing architecture. According to the method,
global optimization is realized by integrating end-side lightweight sensing, edge multi-
modal fusion and cloud long-
term memory. A composite
finite state machine (FSM) active questioning mechanism is combined with sentiment calculation, and is different from traditional rule type triggering. Off-line and on-line fusion scheduling and multi-agent role playing are combined to be applied to a high-emotional-quotient interaction scene, and the
simulation is improved. Multi-modal
emotion perception circulation is introduced, and the problem of single text emotion misjudgment is solved.