System and method for evaluating students' mastery degree in class based on multiple sensors

Abstract

The invention discloses a system and a method for evaluating students' mastery degree in class based on multiple sensors. The method comprises the following steps: collecting student data; preprocessing a student face image sequence and a speech sequence; extracting facial expressions and speech features; classifying facial expressions, speeches and examination results; using a Gaussian mixture model to integrate the classification results; and analyzing the integration result and giving class evaluation and suggestions. The convolution neural network in deep learning is adopted in speech emotion processing, and complicated artificial feature extraction is avoided. Through use of the Gaussian mixture model, the degree of classification confidence of classifiers is determined according to sample distribution and fused adaptively. A novel student in-class mastery degree evaluation scheme based on multiple sensors is designed by combining the facial expressions, speeches and examination results of students. The mastery degree of students in class can be evaluated more objectively and accurately, the mastery degree of students can be judged, and a teaching evaluation result and corresponding suggestions can be given.

Description

technical field [0001] The invention belongs to the technical field of image and voice processing, and further relates to pattern recognition technology, in particular to a multi-sensor-based evaluation system and method for students' classroom mastery. It is used in teaching, psychology, human-computer interaction and other fields. Background technique [0002] Most of the existing teaching evaluation systems are based on people's subjective judgments, and the evaluation results vary from person to person, which has an impact. Therefore, it has become an educational task to conduct emotional analysis on students' facial expressions and voices while listening to the class, so as to make the evaluation results as fair and accurate as possible. pursuit. Teaching managers can also keep abreast of teachers' teaching effects and students' learning conditions, and adjust the goals, methods and strategies of teaching management and decision-making. [0003] The patent application...

AI Technical Summary

Problems solved by technology

Using global statistical sound quality features to achieve Chinese speech emotion representation independent of the speaker and text content; using fuzzy support vector machine to obtain better recognition accuracy under mixed speech emotion conditions, but the method still has the following shortcomings: SVM The final decision function of is only determined by a small number of support vectors. Since SVM uses quadratic programming to solve the support vectors, and solving the quadratic programming will involve the calculation of the matrix. When the number is large, the storage and calculation of the matrix will consume a lot of The machine memory and computing time; the support vector machine algorithm only gives the algorithm of two-class classification, but in practical applications, it is generally necessary to solve multi-class classification problems, which is not conducive to real-time processing

[0005] However, the shortcomings of this teaching evaluation system are: it does not consider the role of students in the teaching process, making the teaching quality evaluation data sources appear single, and the processing is single

In the existing technology, there is still a link of manual feature selection in speech emotion processing, and there is a deviation in manual feature selection, which will directly affect the final classification result; in multi-source data fusion processing, the product rule and mean value rule are often used etc., there is the problem of inaccurate multimodal information fusion

Images