Teaching quality diagnosis and evaluation method based on behavior recognition

Abstract

The invention discloses a teaching quality diagnosis and evaluation method based on behavior recognition, and relates to the technical field of behavior recognition, and the teaching quality diagnosis and evaluation method based on behavior recognition comprises the steps: 1, collecting a real classroom teaching video, and intercepting a key frame from the video content; 2, a target in the teaching video image is detected, and behavior recognition is achieved based on target detection and a human body posture evaluation algorithm; step 3, using the trained target detection model to detect a target teaching video image data set, and dividing different class types based on detected teacher behaviors; 4, determining student behaviors influencing the teaching quality according to the class type, and calculating a teaching quality scoring index; 5, evaluating the teaching quality according to the teaching quality scoring index, and diagnosing problems affecting the teaching quality; through classroom data acquisition from various sources, the classroom teaching quality is accurately analyzed and evaluated, and accurate teaching of teachers and personalized learning of students are assisted.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior recognition, and in particular relates to a teaching quality diagnosis and evaluation method based on behavior recognition. Background Art

[0002] With the application of information network technology in the field of intelligent education, a large amount of big data resources in the field of education have been accumulated, including text, images, videos and other educational data. These huge data contain rich and valuable information. The advent of the era of educational big data has brought new ideas and opportunities for the development of educational informatization. As people pay more and more attention to education, traditional classroom teaching can no longer meet people's needs. Traditional classroom teaching is a teacher's interaction with students through classroom observation and questioning. Undoubtedly, due to the limited energy of teachers, there are certain limitations in the information transmission and feedback between teachers and students, and teachers cannot grasp the changes of students' attention in a timely and accurate manner. Therefore, using classroom teaching video data, combined with computer vision and video intelligent processing technology, to identify student behavior in classroom teaching videos, and analyze the status of students, to build an intelligent intelligent teaching status mining system, so that teachers can understand the changes of students' classroom status and attention in a timely manner, it has important research value and application prospects. Summary of the invention

[0003] The purpose of the present invention is to provide a teaching quality diagnosis and evaluation method based on behavior recognition, which is used to solve the technical problem that the prior art cannot accurately grasp the changes in teaching quality.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] Step 1. Collect real classroom teaching videos and capture key frames of the video content; collect real classroom teaching videos based on the high-definition camera equipment installed in the classroom, ensure that the real classroom teaching videos are panoramic videos of the classroom, collect videos from multiple angles in a classroom, and ensure that the collected classroom teaching videos have stable pictures, clear sounds, and include pictures of everyone in the classroom, determine the key frame capture interval of m frames, that is, capture a key frame every m frames, each key frame corresponds to a video image, classify them by classroom and class, and construct a teaching video image dataset, where one dataset corresponds to a class in a classroom, one class is t minutes, and one minute contains n key frames.

[0006] Step 2: Detect targets in teaching video images and realize behavior recognition based on target detection and human posture assessment algorithms; annotate classroom teaching behaviors for each video image, and define classroom teaching behaviors that need to be annotated based on the content of teaching behaviors and behaviors that affect the quality of classroom teaching. Teaching behaviors include teacher behaviors and student behaviors. Student behaviors include raising the head, lowering the head, tilting the head, talking, and bad behaviors that are harmful to teaching quality. Speaking behaviors are annotated based on video images combined with video and sound information at the corresponding time. Bad behaviors include sleeping, playing with mobile phones, and eating. Teacher behaviors include blackboard writing, multimedia presentations, stillness, patrolling, and interaction. Interactive behaviors are annotated based on video images combined with video and sound information at the corresponding time. Classify the annotated image data sets into training data sets and comparison data sets at a ratio of 6 to 1.

[0007] Use the images in the training data set to train the target detection model, perform scaling, cropping, and normalization preprocessing operations on the images, use YOLOv9 as the target detection model, use the OpenPose algorithm to estimate the human posture, obtain the coordinates of the key points of the human body, and predict the position coordinates of each key point of the human body to determine the spatial position relationship between the key points, thereby obtaining the predicted human skeleton. The model is trained using the self-supervised learning method. The specific contents of the self-supervised learning method are as follows:

[0008] Image segmentation is performed with people as the center. Each key frame image is divided into M grids. A key frame set is constructed and the spatial features of each key frame image are extracted. The spatial features include: contour features, motion features and temporal coding. The parameters of the parameterized human body template are regressed by the regressor. The strategy of synthetic analysis is adopted. The forward rendering is compared with the network estimation results to generate gradients to adjust the shape and motion parameters. The predicted results are constrained by learning the bone distribution to make the effect smoother and more natural. Differentiable rendering is used for self-supervised learning. The parameters of the parameterized human body template are parameterized by self-supervised learning methods, and the three-dimensional joint points based on differentiable rendering are I 3D 、Loss function I LOSS and the two-dimensional reprojection value I 2D To perform matching, the supervised model looks like this:

[0009] I=I 3D +I 2D +I LOSS ;

[0010] in:

[0011] I LOSS =f1L1+f2L2+f3L3;

[0012] L1 represents the target confidence loss, L2 represents the category confidence loss, and L3 represents the coordinate regression loss; and f1, f2, and f3 are corresponding loss parameters, which are set according to the actual situation. For example, in this embodiment, f1=0.125, f2=0.375, and f3=0.5 are set;

[0013] By using the parameterized model of the 3D human body shape as the mesh framework, the human body model parameters are regressed through the regressor to obtain the 3D mesh vertex coordinates. The linear relationship between the 3D mesh vertices and the 3D body joints can be used to obtain the 3D joint points X. The 3D joint points I based on differentiable rendering 3D The specific calculation formula is as follows:

[0014]

[0015] X i represents the three-dimensional joint point coordinates of the i-th vertex, Represents the 3D joint coordinates of the i-th vertex after differentiable rendering.

[0016] The forward synthesis model is used to synthesize the measurement values ​​of the corresponding frame, and a differentiable renderer is used to render the object contour. The surface coordinates corresponding to each pixel in the i frame are taken, and their coordinates in the next frame are calculated. The projection difference is taken to obtain the projection matrix P. The corresponding joint point reprojection is obtained by combining the projection matrix with the three-dimensional joint point coordinates, and the two-dimensional reprojection value I 2D The specific calculation formula is as follows:

[0017]

[0018] where x k represents the two-dimensional point coordinates obtained by reprojecting the joint points, Represents the two-dimensional point coordinates corresponding to the kth joint point coordinates.

[0019] Step 3: Use the trained target detection model to detect the target teaching video image dataset, and divide the classroom into different types based on the detected teacher behavior. Use the trained target detection model to detect the target teaching video image dataset, and classify and judge the teaching quality according to the detection behaviors divided by different detection targets. Different detection targets refer to students and teachers.

[0020] For the same teaching video image dataset, there are t*n key frames in total, and each key frame contains at most 2 behaviors. Blackboard writing behavior refers to the teacher's behavior of modifying the blackboard content, multimedia demonstration behavior refers to the teacher's behavior of using multimedia tools, patrol behavior refers to the teacher's behavior of moving under the podium, and static behavior refers to the teacher's behavior without movement and position change. Each key frame contains at most one of the four behaviors of blackboard writing, multimedia demonstration, static, and patrol. Interactive behavior is determined by sound information. The time period length is set to L seconds for each key frame. The corresponding time of the key frame is the midpoint of the time period. When the sound information of the teacher communicating with the students appears during this period, it is determined that the key frame contains interactive behavior. According to the proportion of different behaviors of teachers, the classroom types of the teaching video image dataset are divided. The specific division method is as follows:

[0021] Class types are divided into self-study classes, discussion classes and regular classes. One class is set to correspond to one class hour. Within one class hour, there are a1 marked blackboard writing behaviors of the teacher, a2 multimedia demonstration behaviors, a3 static behaviors, a4 patrol behaviors, and a5 interactive behaviors, and a1+a2+a3+a4=t*n. Set interactive behavior thresholds b1 and b2, and b2>b1. When a5≤b1, a1+a2<a3+a4, the class type is judged to be a self-study class, when a1+a2≥a3+a4, the class type is judged to be a regular class, when a5≥b2, a1+a2<a3+a4, the class type is judged to be a discussion class, when a1+a2≥a3+a4, the class type is judged to be a regular class, and when b1<a5<b2, the class type is judged to be a regular class.

[0022] Step 4. According to the class type, determine the student behaviors that affect the teaching quality and calculate the teaching quality scoring indicators; student behaviors include positive behaviors, negative behaviors, meaningless behaviors and bad behaviors. Bad behaviors among student behaviors will inevitably lead to a decline in teaching quality. According to the class type, determine the specific behaviors included in positive and negative behaviors. In addition to bad behaviors, for other student behaviors, such as: raising the head, lowering the head, tilting the head, talking, in different class types, set different thresholds for each behavior to determine which ones are positive, negative or meaningless.

[0023] For example: in a self-study class, within one class hour, the thresholds for the frequencies of raising the head, lowering the head, tilting the head, and speaking are set to x1, x2, x3, and x4 respectively. When the frequency of raising the head is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of lowering the head is greater than or equal to its corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of tilting the head is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of speaking is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior.

[0024] In a discussion class, the thresholds for the frequency of raising the head, lowering the head, tilting the head, and speaking are set as follows: It is set that when the frequency of raising the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of lowering the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of tilting the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of speaking is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior.

[0025] In a regular class, the thresholds for the frequency of raising the head, lowering the head, tilting the head, and speaking are set as It is set that when the frequency of raising the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior; when the frequency of lowering the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior; when the frequency of tilting the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior; when the frequency of speaking is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior.

[0026] The teaching quality scoring index function S is as follows:

[0027]

[0028] Where a represents the number of students in the class, A represents the positive behavior adjustment value, and ensure that the S function result is greater than 0, N PB represents the number of positive behaviors in a class, Z NB It is expressed as the impact value of negative behavior, and k1 is expressed as Z NB The weight of Z SB It is expressed as the impact value of bad behavior, and k2 is expressed as Z SB The weight size is k1+k2=1, and k2>k1.

[0029] The impact value calculation formula is as follows:

[0030]

[0031] where λ max Indicates the maximum value of the corresponding behavior in a class period, represents the average value of the corresponding behavior in one class, λ i Indicates the number of behaviors in the corresponding key frame i.

[0032] Step 5: Evaluate the teaching quality according to the teaching quality scoring indicators, and diagnose the problems that affect the teaching quality; set the teaching quality standards as A, B, C, and D, and divide the corresponding standard intervals according to the teaching quality scoring indicators. The standard interval corresponding to the A standard is [Ab, A), the standard interval corresponding to the B standard is [A-2b, Ab), and the standard interval corresponding to the C standard is [A-3b, A-2b). Those not in the above standard intervals are divided into the D standard. b represents the indicator adjustment parameter, which is a virtual setting. The size of parameter b can be adjusted according to actual conditions. The larger the value of the teaching quality scoring indicator function S, the better the teaching quality. When the teaching quality rating is A, it means that most students in the corresponding teaching class listen carefully. When the teaching quality rating is B, it means that most students in the corresponding teaching class listen carefully. When the teaching quality rating is C, it means that some students in the corresponding teaching class listen carefully. When the teaching quality rating is D, it means that most students in the corresponding teaching class do not listen carefully. The teaching quality is evaluated through the rating and division of teaching quality.

[0033] Adjustments will be made to further improve the teaching quality in the classroom. For the classroom type with the most A-level standards, students and teachers will be encouraged. For the classroom type with the most B-level and C-level standards, students and teachers will be reminded, including warning students with bad behavior to reduce the frequency of bad behavior in the classroom, reminding all students to reduce the number of negative and meaningless behaviors, reminding teachers to make appropriate adjustments to their teaching methods, and for the classroom type with the most D-level standards, the frequency of this class type will be reduced, or discipline control over this class type will be strengthened to achieve diagnosis of teaching quality.

[0034] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0035] 1. Using differentiable rendering to perform self-supervised learning on the target detection model reduces the reliance of the target detection model on labeled data, making it possible to effectively train and optimize when data is scarce. By matching the three-dimensional joint points based on differentiable rendering with the two-dimensional reprojection values, the shape and posture of the human body can be estimated more accurately, thereby improving the estimation accuracy, the generalization ability of the model, and the reliability in practical applications.

[0036] 2. Based on the comprehensive analysis of the teacher behavior identification results, conduct a comprehensive evaluation of the classroom types, classify different classroom types, and on this basis judge the impact of student behavior in different classroom types, and evaluate the impact of student behavior on teaching quality from different angles;

[0037] 3. Based on the comprehensive analysis of the results of student behavior identification, the teaching situation is comprehensively evaluated taking into account the student behavior and class type, and the teaching quality is quantified to provide data basis for teaching quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0039] Figure 1 A flow chart of a teaching quality diagnosis and evaluation method based on behavior recognition is shown;

[0040] Figure 2 A flow chart showing a method for calculating teaching quality rating indicators;

[0041] Figure 3 A flow chart of the method for self-supervised learning of target detection models is shown. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Example 1

[0044] Step 1. Collect real classroom teaching videos and capture key frames of the video content; collect real classroom teaching videos based on the high-definition camera equipment installed in the classroom, ensure that the real classroom teaching videos are panoramic videos of the classroom, collect videos from multiple angles in a classroom, and ensure that the collected classroom teaching videos have stable pictures, clear sounds, and include pictures of everyone in the classroom, determine the key frame capture interval of m frames, that is, capture a key frame every m frames, each key frame corresponds to a video image, classify them by classroom and class, and construct a teaching video image dataset, where one dataset corresponds to a class in a classroom, one class is t minutes, and one minute contains n key frames.

[0045] Step 2: Detect targets in teaching video images and realize behavior recognition based on target detection and human posture assessment algorithms; annotate classroom teaching behaviors for each video image, and define classroom teaching behaviors that need to be annotated based on the content of teaching behaviors and behaviors that affect the quality of classroom teaching. Teaching behaviors include teacher behaviors and student behaviors. Student behaviors include raising the head, lowering the head, tilting the head, talking, and bad behaviors that are harmful to teaching quality. Speaking behaviors are annotated based on video images combined with video and sound information at the corresponding time. Bad behaviors include sleeping, playing with mobile phones, and eating. Teacher behaviors include blackboard writing, multimedia presentations, stillness, patrolling, and interaction. Interactive behaviors are annotated based on video images combined with video and sound information at the corresponding time. Classify the annotated image data sets into training data sets and comparison data sets at a ratio of 6 to 1.

[0046] Use the images in the training data set to train the target detection model, perform scaling, cropping, and normalization preprocessing operations on the images, use YOLOv9 as the target detection model, use the OpenPose algorithm to estimate the human posture, obtain the coordinates of the key points of the human body, and predict the position coordinates of each key point of the human body to determine the spatial position relationship between the key points, thereby obtaining the predicted human skeleton. The model is trained using the self-supervised learning method. The specific contents of the self-supervised learning method are as follows:

[0047] Image segmentation is performed with people as the center. Each key frame image is divided into M grids. A key frame set is constructed and the spatial features of each key frame image are extracted. The spatial features include: contour features, motion features and temporal coding. The parameters of the parameterized human body template are regressed by the regressor. The strategy of synthetic analysis is adopted. The forward rendering is compared with the network estimation results to generate gradients to adjust the shape and motion parameters. The predicted results are constrained by learning the bone distribution to make the effect smoother and more natural. Differentiable rendering is used for self-supervised learning. The parameters of the parameterized human body template are parameterized by self-supervised learning methods, and the three-dimensional joint points based on differentiable rendering are I 3D 、Loss function I LOSS and the two-dimensional reprojection value I 2D To perform matching, the supervised model looks like this:

[0048] I=I 3D +I 2D +I LOSS ;

[0049] in:

[0050] I LOSS =f1L1+f2L2+f3L3;

[0051] L1 represents the target confidence loss, L2 represents the category confidence loss, and L3 represents the coordinate regression loss; and f1, f2, and f3 are corresponding loss parameters, which are set according to actual conditions. For example, in this embodiment, f1=0.125, f2=0.375, and f3=0.5 are set.

[0052] By using the parameterized model of the 3D human body shape as the mesh framework, the human body model parameters are regressed through the regressor to obtain the 3D mesh vertex coordinates. The linear relationship between the 3D mesh vertices and the 3D body joints can be used to obtain the 3D joint points X. The 3D joint points I based on differentiable rendering 3D The specific calculation formula is as follows:

[0053]

[0054] X i represents the three-dimensional joint point coordinates of the i-th vertex, Represents the 3D joint coordinates of the i-th vertex after differentiable rendering.

[0055] The forward synthesis model is used to synthesize the measurement values ​​of the corresponding frame, and a differentiable renderer is used to render the object contour. The surface coordinates corresponding to each pixel in the i frame are taken, and their coordinates in the next frame are calculated. The projection difference is taken to obtain the projection matrix P. The corresponding joint point reprojection is obtained by combining the projection matrix with the three-dimensional joint point coordinates, and the two-dimensional reprojection value I 2D The specific calculation formula is as follows:

[0056]

[0057] where x k represents the two-dimensional point coordinates obtained by reprojecting the joint points, Represents the two-dimensional point coordinates corresponding to the kth joint point coordinates.

[0058] Step 3: Use the trained target detection model to detect the target teaching video image dataset, and divide the classroom into different types based on the detected teacher behavior. Use the trained target detection model to detect the target teaching video image dataset, and classify and judge the teaching quality according to the detection behaviors divided by different detection targets. Different detection targets refer to students and teachers.

[0059] For the same teaching video image dataset, there are t*n key frames in total, and each key frame contains at most 2 behaviors. Blackboard writing behavior refers to the teacher's behavior of modifying the blackboard content, multimedia demonstration behavior refers to the teacher's behavior of using multimedia tools, patrol behavior refers to the teacher's behavior of moving under the podium, and static behavior refers to the teacher's behavior without movement and position change. Each key frame contains at most one of the four behaviors of blackboard writing, multimedia demonstration, static, and patrol. Interactive behavior is determined by sound information. The time period length is set to L seconds for each key frame. The corresponding time of the key frame is the midpoint of the time period. When the sound information of the teacher communicating with the students appears during this period, it is determined that the key frame contains interactive behavior. According to the proportion of different behaviors of teachers, the classroom types of the teaching video image dataset are divided. The specific division method is as follows:

[0060] Class types are divided into self-study classes, discussion classes and regular classes. One class is set to correspond to one class hour. Within one class hour, there are a1 marked blackboard writing behaviors of the teacher, a2 multimedia demonstration behaviors, a3 static behaviors, a4 patrol behaviors, and a5 interactive behaviors, and a1+a2+a3+a4=t*n. Set interactive behavior thresholds b1 and b2, and b2>b1. When a5≤b1, a1+a2<a3+a4, the class type is judged to be a self-study class, when a1+a2≥a3+a4, the class type is judged to be a regular class, when a5≥b2, a1+a2<a3+a4, the class type is judged to be a discussion class, when a1+a2≥a3+a4, the class type is judged to be a regular class, and when b1<a5<b2, the class type is judged to be a regular class.

[0061] Step 4. According to the class type, determine the student behaviors that affect the teaching quality and calculate the teaching quality scoring indicators; student behaviors include positive behaviors, negative behaviors, meaningless behaviors and bad behaviors. Bad behaviors among student behaviors will inevitably lead to a decline in teaching quality. According to the class type, determine the specific behaviors included in positive and negative behaviors. In addition to bad behaviors, for other student behaviors, such as: raising the head, lowering the head, tilting the head, talking, in different class types, set different thresholds for each behavior to determine which ones are positive, negative or meaningless.

[0062] For example: in a self-study class, within one class hour, the thresholds for the frequencies of raising the head, lowering the head, tilting the head, and speaking are set to x1, x2, x3, and x4 respectively. When the frequency of raising the head is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of lowering the head is greater than or equal to its corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of tilting the head is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of speaking is greater than or equal to its corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior.

[0063] In a discussion class, the thresholds for the frequency of raising the head, lowering the head, tilting the head, and speaking are set as follows: It is set that when the frequency of raising the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of lowering the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior. When the frequency of tilting the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior. When the frequency of speaking is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior.

[0064] In a regular class, the thresholds for the frequency of raising the head, lowering the head, tilting the head, and speaking are set as It is set that when the frequency of raising the head is greater than or equal to the corresponding threshold, it is judged as a positive behavior, otherwise it is a meaningless behavior; when the frequency of lowering the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior; when the frequency of tilting the head is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior; when the frequency of speaking is greater than or equal to the corresponding threshold, it is judged as a negative behavior, otherwise it is a meaningless behavior.

[0065] The teaching quality scoring index function S is as follows:

[0066]

[0067] Where a represents the number of students in the class, A represents the positive behavior adjustment value, and ensure that the S function result is greater than 0, N PB represents the number of positive behaviors in a class, Z NB It is expressed as the impact value of negative behavior, and k1 is expressed as Z NB The weight of Z SB It is expressed as the impact value of bad behavior, and k2 is expressed as Z SB The weight size is k1+k2=1, and k2>k1.

[0068] The impact value calculation formula is as follows:

[0069]

[0070] where λ max Indicates the maximum value of the corresponding behavior in a class period, represents the average value of the corresponding behavior in one class, λ i Indicates the number of behaviors in the corresponding key frame i.

[0071] Step 5: Evaluate the teaching quality according to the teaching quality scoring indicators, and diagnose the problems that affect the teaching quality; set the teaching quality standards as A, B, C, and D, and divide the corresponding standard intervals according to the teaching quality scoring indicators. The standard interval corresponding to the A standard is [Ab, A), the standard interval corresponding to the B standard is [A-2b, Ab), and the standard interval corresponding to the C standard is [A-3b, A-2b). Those not in the above standard intervals are divided into the D standard. b represents the indicator adjustment parameter, which is a virtual setting. The size of parameter b can be adjusted according to actual conditions. The larger the value of the teaching quality scoring indicator function S, the better the teaching quality. When the teaching quality rating is A, it means that most students in the corresponding teaching class listen carefully. When the teaching quality rating is B, it means that most students in the corresponding teaching class listen carefully. When the teaching quality rating is C, it means that some students in the corresponding teaching class listen carefully. When the teaching quality rating is D, it means that most students in the corresponding teaching class do not listen carefully. The teaching quality is evaluated through the rating and division of teaching quality.

[0072] Adjustments will be made to further improve the teaching quality in the classroom. For the classroom type with the most A-level standards, students and teachers will be encouraged. For the classroom type with the most B-level and C-level standards, students and teachers will be reminded, including warning students with bad behavior to reduce the frequency of bad behavior in the classroom, reminding all students to reduce the number of negative and meaningless behaviors, reminding teachers to make appropriate adjustments to their teaching methods, and for the classroom type with the most D-level standards, the frequency of this class type will be reduced, or discipline control over this class type will be strengthened to achieve diagnosis of teaching quality.

[0073] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0074] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. The teaching quality diagnosis and evaluation method based on behavior recognition is characterized by: The following steps are involved: Step 1: Collect real classroom teaching videos and capture key frames of the video content; Step 2: Detect the targets in the teaching video images and realize behavior recognition based on target detection and human posture assessment algorithms; Step 3: Use the trained target detection model to detect the target teaching video image dataset and divide different classroom types based on the detected teacher behaviors; Step 4: According to the class type, determine the student behaviors that affect the teaching quality and calculate the teaching quality scoring indicators; Step five: Evaluate the teaching quality based on the teaching quality scoring indicators and diagnose problems that affect the teaching quality.

2. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 1 is characterized in that: include: The specific contents of step one include determining the interval of m frames for capturing key frames, that is, capturing a key frame every m frames, each key frame corresponding to a video image, classifying by classroom and class time, and constructing a teaching video image dataset, where one dataset corresponds to a class time in a classroom, one class time is t minutes, and one minute contains n key frames.

3. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 1 is characterized in that: include: The specific contents of step two include labeling classroom teaching behaviors for each video image, defining the classroom teaching behaviors that need to be labeled based on the behaviors that affect the quality of classroom teaching, including student behaviors and teacher behaviors, classifying the labeled image data sets, scaling, cropping, and normalizing the images for preprocessing, using YOLOv9 as the target detection model, using the OpenPose algorithm for human posture estimation, and training the model through self-supervised learning methods.

4. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 3 is characterized in that: include: The specific contents of training the model through self-supervised learning method are as follows: Image segmentation is performed with people as the center. Each key frame image is divided into M grids. A key frame set is constructed and the spatial features of each key frame image are extracted. The spatial features include: contour features, motion features and temporal coding. The parameters of the parameterized human body template are regressed by a regressor. The strategy of synthetic analysis is adopted. The forward rendering is compared with the network estimation results to generate gradients to adjust the shape and parameters. The predicted results are constrained by the bone distribution. The parameters of the parameterized human body template are parameterized, and a self-supervised learning method is used. The three-dimensional joint points I based on differentiable rendering are 3D 、Loss function I LOSS and the two-dimensional reprojection value I 2D For matching, the self-supervised model is as follows: I=I 3D +I 2D +I LOSS ; in: I LOSS =f1L1+f2L2+f3L3; L1 represents the target confidence loss, L2 represents the category confidence loss, L3 represents the coordinate regression loss; and f1, f2, and f3 are the corresponding loss parameters.

5. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 3 is characterized in that: include: 3D Articulation Based on Differentiable Rendering I 3D The specific calculation method is as follows: By using the parameterized model of the 3D human body shape as the mesh framework, the human body model parameters are regressed through the regressor to obtain the 3D mesh vertex coordinates. The linear relationship between the 3D mesh vertices and the 3D body joints can be used to obtain the 3D joint points X. The 3D joint points I based on differentiable rendering 3D The specific calculation formula is as follows: X i represents the three-dimensional joint point coordinates of the i-th vertex, Represents the 3D joint coordinates of the i-th vertex after differentiable rendering.

6. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 3 is characterized in that: include: 2D reprojection value I 2D The specific calculation method is as follows: The forward synthesis model is used to synthesize the measurement values ​​of the corresponding frame, and a differentiable renderer is used to render the object contour. The surface coordinates corresponding to each pixel in the i frame are taken, and their coordinates in the next frame are calculated. The projection difference is taken to obtain the projection matrix P. The corresponding joint point reprojection is obtained by combining the projection matrix with the three-dimensional joint point coordinates, and the two-dimensional reprojection value I 2D The specific calculation formula is as follows: where x k represents the two-dimensional point coordinates obtained by reprojecting the joint points, Represents the two-dimensional point coordinates corresponding to the kth joint point coordinates.

7. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 1 is characterized in that: include: The specific contents of step three are as follows: According to the proportion of different teacher behaviors, the classroom types of the teaching video image dataset are divided. The specific division method is as follows: Class types are classified, and one class corresponds to one class period. In one class period, the number of marked blackboard writing behaviors of the teacher is a1, the number of multimedia demonstration behaviors is a2, the number of static behaviors is a3, the number of patrol behaviors is a4, the number of interactive behaviors is a5, and a1+a2+a3+a4=t*n; Set the interactive behavior thresholds b1 and b2, and b2>b1. When a5≤b1, a1+a2<a3+a4, the class type is judged as a self-study class. When a1+a2≥a3+a4, the class type is judged as a regular class. When a5≥b2, a1+a2<a3+a4, the class type is judged as a discussion class; when a1+a2≥a3+a4, the class type is judged as a regular class; When b1<a5<b2, the class type is judged to be a regular class.

8. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 1 is characterized in that: include: Step 4: According to the class type, determine the student behaviors that affect the teaching quality, including dividing the student behaviors into positive behaviors, negative behaviors, meaningless behaviors and bad behaviors. Except for bad behaviors, the specific behaviors of other student behaviors in different class types are different. On the basis of determining the class type, set different thresholds for each specific behavior to determine which specific behaviors belong to positive behaviors, negative behaviors or meaningless behaviors in the corresponding class type. Calculate the teaching quality scoring index and set the teaching quality scoring index function S as follows: Where a represents the number of students in the class, A represents the positive coefficient, A is used to ensure that the S function result is greater than 0, A is a virtual setting, N PB represents the number of positive behaviors in a class, Z NB is the impact value of negative behavior, k1 is Z NB The weight, Z SB It is expressed as the impact value of bad behavior, and k2 is expressed as Z SB The weight of , k1+k2=1, and k2>k1; The impact value calculation formula is as follows: where λ max Indicates the maximum value of the corresponding behavior in a class period, represents the average value of the corresponding behavior in one class, λ i Indicates the number of behaviors in the corresponding key frame i.

9. The teaching quality diagnosis and evaluation method based on behavior recognition according to claim 1 is characterized in that: include: The specific contents of step 5 are as follows: The teaching quality standards are set as A, B, C, and D respectively. The corresponding standard intervals are divided according to the teaching quality scoring indicators. The A standard corresponds to the standard interval [Ab, A), the B standard corresponds to the standard interval [A-2b, Ab), and the C standard corresponds to the standard interval [A-3b, A-2b). Those not within the above standard intervals are classified as D standards. b represents the indicator adjustment parameter. The larger the value of the teaching quality scoring indicator function S, the better the teaching quality.

Images