The application relates to the technical field of computer application, and provides a question and answer
interaction method and device combined with facial expressions of characters. The purpose is to solve the problem that direct
transcoding and merging processes in the existing scheme can significantly affect performance. The main scheme comprises the following steps: collecting facial expressions of users when the users answer questionnaire problems, obtaining
facial expression images of a plurality of users,
cutting the horizontal side and the vertical side of each
facial expression image, then performing aggregation to obtain a sub-image, performing
information extraction on all user
facial expression sub-images to obtain an expression hidden representation vector; using a nonlinear function to perform mapping to obtain three types of emotions, obtaining
branch fixed texts corresponding to all emotion categories, using a BERT model to respectively
encode the fixed texts, user labels and original texts of the questionnaire problems to obtain hidden vectors, using a weight modification coefficient to map the obtained hidden vectors to a question space, and the next most suitable question can be obtained.