Teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection

Abstract

The invention discloses a teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection. The teaching quality evaluation and optimization platform comprises a classroom behavior recognition module, a teaching quality evaluation module and a teaching analysis and optimization module. The classroom behavior recognition module acquires and preprocesses video information generated in a classroom video, then detects skeleton key points in the classroom video by using an OpenPose skeleton key point detection model to recognize teacher and student classroom behaviors, and then performs training based on a support vector machine classification algorithm to generate a classroom behavior classification model to classify the teacher and student classroom behaviors; the teaching quality evaluation module is used for calculating the class positive degree, the class attending seriousness of the students and the teaching seriousness of the teachers and evaluating the teaching quality; the teaching analysis and optimization module carries out teaching analysis and optimization based on teacher and student classroom behaviors and teaching quality evaluation. According to the invention, classroom behaviors of teachers and students can be identified and classified in real time, and comprehensive, objective and reliable teaching quality evaluation is provided.

Description

Technical Field

[0001] The present invention belongs to the field of visual recognition technology, and specifically relates to a teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection. Background Art

[0002] With the development of educational technology, the field of education increasingly needs an objective and accurate way to evaluate teaching quality. Through technical means, the behavior of teachers and students in the classroom can be quantified, so as to more scientifically evaluate the teaching quality and provide an accurate basis for teaching improvement. In traditional teaching quality evaluation, subjective observations and questionnaires are often relied on. For example, the teaching quality is judged by students' evaluation questionnaires on teachers' teaching, but this method has problems such as the influence of students' subjective factors and the incomplete evaluation results. Teachers' judgment on students' seriousness in class is also mostly based on personal experience and local observations, and it is difficult to accurately count the concentration of each student throughout the class. Summary of the invention

[0003] The purpose of the present invention is to provide a teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection. The present invention can identify and classify the classroom behaviors of teachers and students in real time, and provide a comprehensive, objective and reliable teaching quality evaluation.

[0004] The technical solution of the present invention is: a teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, including a classroom behavior recognition module, a teaching quality evaluation module and a teaching analysis and optimization module;

[0005] The classroom behavior recognition module obtains and preprocesses the video information generated in the classroom video, then uses the OpenPose skeleton key point detection model to detect the skeleton key points in the classroom video to identify the classroom behavior of teachers and students, and then generates a classroom behavior classification model based on the support vector machine classification algorithm to classify the classroom behavior of teachers and students;

[0006] The teaching quality evaluation module calculates the classroom enthusiasm, student listening attentively and teacher teaching attentively based on the classified classroom behaviors of teachers and students, and evaluates the teaching quality based on the calculated student classroom enthusiasm, student listening attentively and teacher teaching attentively;

[0007] The teaching analysis and optimization module performs teaching analysis and optimization based on teacher and student classroom behaviors and teaching quality evaluation.

[0008] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the process of identifying the classroom behaviors of teachers and students by skeleton key points adopts the skeleton key point direct coordinate method and the skeleton key point relationship feature extraction method;

[0009] In the classroom behaviors of teachers and students, for those with obvious geometric relationships between the coordinates of skeletal key points, the skeletal key point direct coordinate method is used for identification; for those with similar movements, the skeletal key point relationship feature extraction method is used for identification.

[0010] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the skeleton key point direct coordinate method is used to identify the classroom behaviors of teachers and students, including students raising their hands, students stretching, teachers teaching, and teachers writing on the blackboard.

[0011] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeletal key point detection, the skeletal key point relationship feature extraction method is used to identify the classroom behaviors of teachers and students, including students writing, students sleeping, students playing with mobile phones, and teachers lying down, and teachers playing with mobile phones; the classroom behaviors of teachers and students are identified through the extraction of arm skeletal key point features and head skeletal key point features.

[0012] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the classification is to divide the classroom behaviors of teachers and students into positive behaviors and negative behaviors; the training generation process of the classroom behavior classification model is to collect three actions of writing, sleeping and playing with mobile phones, and extract arm feature vectors from them. and the head feature vector The training is carried out through the support vector machine classification algorithm, and finally a classroom behavior classification model is generated.

[0013] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the calculation formula for the classroom enthusiasm is:

[0014] Classroom activity = (number of times writing + number of times raising hands) / total class time.

[0015] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the calculation method of the student's listening attentively is to collect image data once every 10 seconds, and count the number of negative behaviors of students within 60 seconds. If a student exhibits any negative behavior three times within 60 seconds, it is determined that he has not listened to the class attentively during this time period. The number of students who listened attentively in the whole class is counted and divided by the total number of students in the class to obtain the overall student listening attentively in the class.

[0016] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the calculation formula for the teacher's teaching seriousness is:

[0017] Professor's seriousness = (number of lectures + number of blackboard writing) × weight of positive behavior - (number of times lying down + number of times playing with mobile phone) × weight of negative behavior.

[0018] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, the analysis process includes the analysis of the distribution of teacher and student behaviors and the changing trend of teaching quality, and optimization suggestions are put forward based on the results of the analysis.

[0019] In the aforementioned teaching quality evaluation and optimization platform based on OpenPose skeletal key point detection, the teaching quality evaluation and optimization platform based on OpenPose skeletal key point detection also includes a basic information management module and a quality education module; the basic information management module is used for user information management, course information management, and system management and maintenance, and provides data support for other modules; the quality teaching reference module collects and organizes high-quality classroom videos based on the analysis results of the teaching analysis and optimization module.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The present invention can collect classroom video data in real time, use the OpenPose model to detect the key points of the skeletons of teachers and students, rely on the direct coordinate method of the skeleton key points and the feature extraction method of the skeleton key points to identify and judge the classroom behaviors of teachers and students, and combine the support vector machine algorithm to accurately identify and classify classroom behaviors, thereby providing comprehensive, objective and reliable teaching quality evaluation and optimization suggestions. The present invention improves the teaching quality evaluation system and provides more comprehensive and accurate evaluation and optimization guidance for students, teachers and schools. In addition, the construction of this platform helps to explore the laws of the impact of teachers' classroom behaviors on students' classroom performance and provide support for educational decision-making. The present invention realizes the intelligent detection and evaluation of classroom teaching activities, which will promote the process of school education informatization and inject new impetus into the sustainable development of education. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a structural diagram of the present invention;

[0023] Figure 2 This is a flow chart of the teacher-student classroom behavior recognition module of the present invention;

[0024] Figure 3 The skeleton key points of the present invention are shown in FIG. Figure 1 ;

[0025] Figure 4 The skeleton key points of the present invention are shown in FIG. Figure 2 ;

[0026] Figure 5 A schematic diagram of high-dimensional hyperplane classification of the support vector machine classification algorithm of the present invention;

[0027] Figure 6 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0028] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but they are not intended to limit the present invention.

[0029] Embodiment: A teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, such as Figure 1 As shown, it includes basic information management module, classroom behavior recognition module, teaching quality evaluation module, teaching analysis and optimization module and high-quality teaching reference module.

[0030] The basic information management module mainly realizes the background functions of this system and provides necessary data support for subsequent modules. The basic information management module includes user information management, course information management and system management and maintenance.

[0031] The user objects of this platform are teachers, teaching administrators and system administrators. As the implementers of teaching, teachers can view the classroom behavior recognition results of students in the class they teach, and analyze the indicators of enthusiasm and seriousness, so as to adjust the teaching strategy and optimize the teaching effect. Relying on this platform, teaching administrators can view the classroom behavior recognition results and status indicator analysis of all class teachers and students, and are responsible for supervising the teaching quality and ensuring that the teaching process is standardized and efficient; system administrators have the ability to add, delete and modify user information, and can also assign corresponding operation permissions according to the type of user object to ensure the security and stable operation of the system.

[0032] Course information management sets and manages relevant information such as subjects, classes, teachers and course time to ensure that course information is timely and accurate.

[0033] The system administrator is responsible for managing the operating status of the entire system, including database maintenance, system log management, permission management, etc., to ensure the normal operation and efficient maintenance of the system.

[0034] like Figure 2 As shown in the figure, the classroom behavior recognition module obtains the video information generated in the classroom video and preprocesses it, including video denoising and image segmentation. Then, the OpenPose skeleton key point detection model is used to detect the skeleton key points in the classroom video and identify the classroom behavior of teachers and students.

[0035] The OpenPose skeleton key point detection model is an open source project model developed with Caffe as the framework, based on convolutional neural networks and supervised learning. Through a bottom-up human skeleton key point detection method, it can quickly identify and estimate multiple important feature points of the human body, and has good robustness and real-time performance.

[0036] The process of identifying the classroom behaviors of teachers and students by skeleton key points adopts the skeleton key point direct coordinate method and the skeleton key point relationship feature extraction method.

[0037] The classroom behaviors of teachers and students include students' classroom behaviors and teachers' classroom behaviors. Students' classroom behaviors include writing, raising hands, sleeping, playing with mobile phones and stretching. Teachers' classroom behaviors include lecturing, writing on the blackboard, lying down and playing with mobile phones.

[0038] For classroom behaviors of teachers and students with obvious geometric relationship of skeleton key point coordinates, such as students raising hands and stretching and teachers teaching and writing on the blackboard, Figure 3 As shown in the figure, the direct coordinate method of skeleton key points is used for recognition. This recognition method has high accuracy and fast calculation speed. Taking the student's hand-raising gesture as an example, when the key point of the human hand is higher than the key point of the nose, it can be determined that the action at this time is raising the hand.

[0039] Taking raising the left hand as an example, the specific calculation method is: taking the upper left corner as the coordinate origin, when the vertical coordinate of the key point H of the left hand is less than the vertical coordinate of point N, and the vertical coordinate of the key point of the right hand is greater than N, the classroom behavior at this time is raising the left hand.

[0040] For classroom behaviors of teachers and students that are similar in action, such as students writing, sleeping, playing with mobile phones and teachers lying down, playing with mobile phones, Figure 4 As shown in the figure, the skeleton key point relationship feature extraction method is used for recognition. Since the key point features of the above actions are different mainly in the arms and head, the classroom behaviors of teachers and students are identified by extracting the skeleton key point features of the arms and head.

[0041] Taking the calculation of head feature vector as an example, we first extract the head and neck feature vector between the neck key point C1 and the head key point N1. and the shoulder feature vector between the key points S1 and S2 of the left and right shoulders Calculate the head and neck eigenvector The eigenvector at the shoulder The projection length and shoulder eigenvector in the direction The ratio of the projection lengths Y1:

[0042]

[0043] In the formula, Y1 is the characteristic vector of the left arm of the human body. Due to the directionality of the vector, when N1 is above C1, Y1 is positive, and when N1 is below C1, Y1 is negative. In this way, the action features of raising and lowering the head can be distinguished.

[0044] Based on the support vector machine classification algorithm, a classroom behavior classification model is trained and generated. The classroom behaviors of teachers and students are classified through the classroom behavior classification model, and the classroom behaviors of teachers and students are divided into positive behaviors and negative behaviors, as shown in Table 1. The training and generation process of the classroom behavior classification model is to collect three actions of writing, sleeping and playing with mobile phones, and extract arm feature vectors from them. and the head feature vector Trained by support vector machine classification algorithm, such as Figure 5 As shown, a classroom behavior classification model is finally generated.

[0045] Table 1 Classification of teacher and student classroom behaviors

[0046]

[0047]

[0048] This module uses support vector machine as the classification algorithm for teacher and student classroom behaviors. Support vector machine is a commonly used classification algorithm in machine learning, with good generalization ability and robustness, and is suitable for different classification problems.

[0049] like Figure 6 As shown in the figure, the front-end will adopt the modern Web development technology stack, use the front-end framework Vue.js to improve development efficiency and component management, use flex layout to achieve responsive layout, use Echarts to draw charts to achieve visualization of some data on the web page, etc.; the back-end will use the high-performance server-side programming language Java, combined with the deep learning framework TensorFlow / PyTorch, to handle the deployment and reasoning of the OpenPose model, and use the database technology MySQL to store and manage the classroom behavior data of teachers and students. At the same time, the back-end will also be responsible for the training and deployment of SVM classifiers, real-time processing and analysis of video data from classroom cameras, etc. The entire system adopts a development model with front-end and back-end separation to ensure the scalability and maintainability of the system.

[0050] The teaching quality evaluation module calculates classroom enthusiasm, student listening attentively and teacher teaching attentively based on the classified classroom behaviors of teachers and students, and conducts teaching quality evaluation based on the calculated student classroom enthusiasm, student listening attentively and teacher teaching attentively.

[0051] For students, classroom enthusiasm and seriousness are important indicators for evaluating the quality of classroom teaching.

[0052] Classroom activity reflects the real-time record of students writing and raising their hands in class. The calculation formula is:

[0053] Classroom activity = (number of times writing + number of times raising hands) / total class time.

[0054] Add up the enthusiasm of each student and divide it by the total number of students in the class to get the overall student enthusiasm of the class.

[0055] The calculation method of students' listening attentively is to collect image data every 10 seconds and count the number of negative behaviors of students within 60 seconds. If a student exhibits any negative behavior three times within 60 seconds, it is judged that he / she did not listen to the class attentively during this time period. The number of students who listened attentively in the whole class is counted and divided by the total number of students in the class to obtain the overall student listening attentively in the class.

[0056] For teachers, attention should be paid to their teaching seriousness to evaluate their teaching quality.

[0057] The teacher's teaching seriousness reflects the number of positive and negative behaviors of the teacher in the classroom. The more positive behaviors and the fewer negative behaviors, the higher the teacher's teaching seriousness. The calculation formula is:

[0058] Professor's seriousness = (number of lectures + number of blackboard writing) × weight of positive behavior - (number of times lying down + number of times playing with mobile phone) × weight of negative behavior.

[0059] Classroom teaching quality assessment uses students' classroom enthusiasm and students' listening attentively as key indicators to accurately assess classroom teaching quality. Through in-depth mining of student behavior, it helps teachers understand students' learning status and needs more clearly, so as to adjust teaching strategies and improve teaching effectiveness. In addition, this part also supports real-time viewing. Teachers can view videos and corresponding indicators of a specified time period at any time, so as to adjust teaching progress and methods in a timely manner.

[0060] While focusing on students' learning status, the teacher teaching quality assessment will also assess the teacher's teaching behavior in the classroom. Through the teacher's teaching seriousness index, the quality of their classroom teaching can be objectively and comprehensively evaluated. This function provides a more comprehensive and integrated evaluation method for teaching administrators and schools, helping them to clarify the causes of performance in different classes and provide praise and warnings for teachers; it also helps to promote the growth of the teaching team and improve the overall teaching quality of the school.

[0061] The teaching analysis and optimization module analyzes and optimizes based on the classroom behavior of teachers and students and the teaching quality evaluation. The analysis process includes the analysis of the distribution of teacher and student behaviors and the changing trends of teaching quality. Based on the results of the analysis, optimization suggestions are put forward to provide accurate data support for teaching optimization.

[0062] The high-quality teaching reference module collects and organizes high-quality classroom videos based on the analysis results of the teaching analysis and optimization module, and analyzes the teaching methods and teaching effects of excellent teaching cases to provide a reference for teaching experience.

[0063] In summary, the present invention can identify and classify the classroom behaviors of teachers and students in real time, and provide a comprehensive, objective and reliable teaching quality evaluation.

Claims

1. A teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection, characterized by: It includes classroom behavior recognition module, teaching quality assessment module and teaching analysis and optimization module; The classroom behavior recognition module obtains and preprocesses the video information generated in the classroom video, then uses the OpenPose skeleton key point detection model to detect the skeleton key points in the classroom video to identify the classroom behavior of teachers and students, and then generates a classroom behavior classification model based on the support vector machine classification algorithm to classify the classroom behavior of teachers and students; The teaching quality evaluation module calculates the classroom enthusiasm, student listening attentively and teacher teaching attentively based on the classified classroom behaviors of teachers and students, and evaluates the teaching quality based on the calculated student classroom enthusiasm, student listening attentively and teacher teaching attentively; The teaching analysis and optimization module performs teaching analysis and optimization based on teacher and student classroom behaviors and teaching quality evaluation.

2. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 1 is characterized by: The process of identifying the classroom behaviors of teachers and students by skeleton key points adopts the skeleton key point direct coordinate method and the skeleton key point relationship feature extraction method; In the classroom behaviors of teachers and students, for those with obvious geometric relationships between the coordinates of skeletal key points, the skeletal key point direct coordinate method is used for identification; for those with similar movements, the skeletal key point relationship feature extraction method is used for identification.

3. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 2 is characterized in that: The direct coordinate method of skeleton key points is used to identify classroom behaviors of teachers and students, including students raising their hands, students stretching, teachers teaching, and teachers writing on the blackboard.

4. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 2 is characterized in that: The skeleton key point relationship feature extraction method is used to identify the classroom behaviors of teachers and students, including students writing, students sleeping, students playing with mobile phones, and teachers lying down, and teachers playing with mobile phones; the classroom behaviors of teachers and students are identified by extracting the key points of the arm skeleton and the key points of the head skeleton.

5. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 4 is characterized in that: The classification is to divide the classroom behaviors of teachers and students into positive behaviors and negative behaviors; the training generation process of the classroom behavior classification model is to collect three actions of writing, sleeping and playing with mobile phones, and extract arm feature vectors from them and the head feature vector The training is carried out through the support vector machine classification algorithm, and finally a classroom behavior classification model is generated.

6. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 4 is characterized in that: The calculation formula for the classroom enthusiasm is: classroom enthusiasm = (number of writing times + number of hand raising times) / total class time.

7. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 5 is characterized by: The calculation method of the student's listening attentively is to collect image data once every 10 seconds, and count the number of negative behaviors of students within 60 seconds. If a student exhibits any negative behavior three times within 60 seconds, it is determined that he or she has not listened to the class attentively during this time period. The number of students who listened attentively in the whole class is counted and divided by the total number of students in the class to obtain the overall student listening attentively in the class.

8. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 5 is characterized by: The calculation formula for the teacher's teaching seriousness is: Professor's seriousness = (number of lectures + number of blackboard writing) × weight of positive behavior - (number of times lying down + number of times playing with mobile phone) × weight of negative behavior.

9. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 1, characterized in that: The analysis process includes analyzing the distribution of teacher and student behaviors and the changing trend of teaching quality, and making optimization suggestions based on the results of the analysis.

10. The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection according to claim 1, characterized in that: The teaching quality evaluation and optimization platform based on OpenPose skeleton key point detection also includes a basic information management module and a high-quality education module; the basic information management module is used for user information management, course information management, and system management and maintenance, and provides data support for other modules; the high-quality teaching reference module collects and organizes high-quality classroom videos based on the analysis results of the teaching analysis and optimization module.

Images