Multi-sensor-based assessment system and method for students' classroom mastery

Abstract

The invention discloses a multi-sensor-based system and method for assessing students' classroom mastery. The implementation includes: student data collection; student facial image sequence and voice sequence preprocessing respectively; facial expression and voice feature extraction; facial expression, voice, Classification and processing of test scores; fusion of classification results using Gaussian mixture model; analysis of fusion results to give classroom evaluation and suggestions. In the voice emotion processing of the present invention, the convolutional neural network in deep learning is used to avoid complex manual feature extraction; the Gaussian mixture model is used to make the classification confidence of each classifier depend on the sample distribution, and the self-adaptive fusion is performed. Based on the idea of ​​combining students' facial expressions, students' voice and students' test scores, the invention designs a new scheme for evaluating students' classroom mastery degree based on multi-sensors. More objectively and accurately evaluate the mastery status of students in the classroom, make judgments on the mastery status of students, and give teaching evaluation results and corresponding suggestions.

Description

technical field [0001] The invention belongs to the technical field of image and voice processing, and further relates to pattern recognition technology, in particular to a multi-sensor-based evaluation system and method for students' classroom mastery. It is used in teaching, psychology, human-computer interaction and other fields. Background technique [0002] Most of the existing teaching evaluation systems are based on people's subjective judgments, and the evaluation results vary from person to person, which has an impact. Therefore, it has become an educational task to conduct emotional analysis on students' facial expressions and voices while listening to the class, so as to make the evaluation results as fair and accurate as possible. pursuit. Teaching managers can also keep abreast of teachers' teaching effects and students' learning conditions, and adjust the goals, methods and strategies of teaching management and decision-making. [0003] The patent application...

AI Technical Summary

Problems solved by technology

Using global statistical sound quality features to achieve Chinese speech emotion representation independent of the speaker and text content; using fuzzy support vector machine to obtain better recognition accuracy under mixed speech emotion conditions, but the method still has the following shortcomings: SVM The final decision function of is only determined by a small number of support vectors. Since SVM uses quadratic programming to solve the support vectors, and solving the quadratic programming will involve the calculation of the matrix. When the number is large, the storage and calculation of the matrix will consume a lot of The machine memory and computing time; the support vector machine algorithm only gives the algorithm of two-class classification, but in practical applications, it is generally necessary to solve multi-class classification problems, which is not conducive to real-time processing

[0005] However, the shortcomings of this teaching evaluation system are: it does not consider the role of students in the teaching process, making the teaching quality evaluation data sources appear single, and the processing is single

In the existing technology, there is still a link of manual feature selection in speech emotion processing, and there is a deviation in manual feature selection, which will directly affect the final classification result; in multi-source data fusion processing, the product rule and mean value rule are often used etc., there is the problem of inaccurate multimodal information fusion

Images