A live teaching analysis method based on portrait point pattern matching

Abstract

The application discloses a live teaching analysis method based on portrait point mode matching and belongs to the technical field of live teaching analysis, which comprises the following steps: obtaining the portraits of students when live teaching starts; extracting the triplet portrait graph G of each student according to the characteristics of each student portrait; constructing a point set to be matched for each student's triplet portrait graph G; constructing a Laplace matrix for the point set to be matched respectively; and performing singular decomposition on the student portrait graph to obtain the eigenvalue and eigenvector of the student portrait graph. The application evaluates the performance of students during live teaching by displaying the change of the head posture of students during live teaching, and provides a basis for scoring for teachers when scoring the usual scores of the final teaching by setting the distraction index, and solves the problem that the existing Internet live platform cannot analyze the concentration degree of students in real time in combination with the student portraits during live teaching.

Description

TECHNICAL FIELD

[0001] The application relates to a live teaching analysis method, in particular to a live teaching analysis method based on portrait point pattern matching, and belongs to the technical field of live teaching analysis. BACKGROUND

[0002] In the research and application of a network course platform based on modern technology, an Internet live learning platform based on SaaS technology development is mentioned, but there is a problem in the live teaching on the Internet live teaching platform, that is, the teacher focuses on the explanation of the courseware when teaching, ignores the listening situation of the students, and the distraction index of the students when attending classes cannot be recorded and analyzed, thereby affecting the teaching effect of the teacher and the learning effect of the students, but there is no method for analyzing the student's portrait during live teaching and timely reminding the teacher when the student's distraction index is insufficient, and the single learning effect test cannot reflect the state of the students when attending classes in real time, and in view of the above problems, how to research a live teaching analysis method based on portrait point pattern matching is a current problem to be solved, and the method can use the video portrait on the student listening window for point pattern matching, and remind the teacher of the concentration degree of the students when attending classes through the analysis and calculation value of the distraction index. SUMMARY

[0003] The main purpose of the application is to solve the problem that the existing Internet live learning platform cannot analyze the concentration degree of students in real time in combination with the student portrait during live teaching, and provide a live teaching analysis method based on portrait point pattern matching.

[0004] The purpose of the application can be achieved by adopting the following technical scheme:

[0005] A live teaching analysis method based on portrait point pattern matching, the method comprises:

[0006] Step one: obtaining the portrait of the students when the live teaching starts;

[0007] Step two: extracting the triple portrait graph G of each student according to the characteristics of each student portrait, constructing a to-be-matched point set for the triple portrait graph G of each student, respectively constructing a Laplace matrix for the to-be-matched point set, singularly decomposing the student portrait graph to obtain the eigenvalue and eigenvector of the student portrait graph, constructing an initial matching relationship matrix through the decomposed eigenvalue and eigenvector, combining the Hungarian algorithm to minimize the matching cost between the point sets, obtaining a new matching relationship matrix, and realizing the matching of the student portrait feature points;

[0008] Step three: setting the portrait matching task every ten minutes, setting the matching time as 2 minutes each time, and setting the matching image times as 4 times.

[0009] Step four: when matching the portrait of each student, the matching student distraction index formula is set as:

[0010]

[0011] In the formula: Z c denoted as the image matching student distraction index, A c denoted as the sum of image feature point moving speeds of the target student, B c denoted as the sum of image feature point moving speeds of other students except the target student, and alpha, beta and gamma are all constant coefficients with different values considering the student category.

[0012] In which:

[0013] In the formula: x i denoted as the moving distance of each feature point each time the portrait is matched, t i denoted as the moving time length of each feature point each time the portrait is matched, m c denoted as the feature area surrounded by the portrait feature points of the target student, n denoted as the number of feature points of the target student;

[0014]

[0015] In the formula: x i,j denoted as the moving distance of each feature point of each student except the target student each time the portrait is matched, t i,j denoted as the moving time length of each feature point of each student except the target student each time the portrait is matched, v denoted as the total number of students participating in the live broadcast;

[0016] Step five: when the students participating in the live broadcast are distracted and the distraction index exceeds the standard value set in advance, the head portrait frame of the distracted student on the live screen turns red, and the teacher is prompted below the screen with the distraction index of the student and the student's name;

[0017] Step six: according to the situation of each student in class, the distraction index score of each student in the live class is recorded, which provides a basis for the teacher to score the usual part of the score.

[0018] As a further scheme of the present application, the basis formula for the teacher to score the usual part of the score in step six is:

[0019]

[0020] In the formula: P c denoted as the usual part score of the student for a semester, 30 denoted as the total score of the usual part, Z c,hThe data is recorded as the student's distraction index for each live stream, h is the live stream number, and d is the number of live stream teaching sessions per semester.

[0021] As a further aspect of the present invention, the student in step one is one of a primary school student, a middle school student, or a university student.

[0022] As a further aspect of the present invention, in step two, the number of feature points in the triplet portrait image G of each student is set to 15 when the student in step two is a primary school student, 20 when the student in step two is a middle school student, and 25 when the student in step two is a university student.

[0023] As a further embodiment of the present invention, in step three, a human face matching task is set to be performed every ten minutes, or a human face matching task can be set to be performed every five minutes. When a human face matching task is set to be performed every five minutes, the duration of each matching task is set to 1 minute, and the number of matching images is set to 4.

[0024] As a further aspect of the present invention, the constant values ​​of α, β, and γ in step four are set to consider the different ages of the student categories and are respectively:

[0025] When the student category is primary school students, α = 1.25, β = 0.23, γ = 1;

[0026] When the student category is middle school students, α = 1.18, β = 0.21, γ = 0.85;

[0027] When the student category is university students, α = 1.01, β = 0.15, and γ = 0.66.

[0028] As a further embodiment of the present invention, step five can also be set as follows:

[0029] Step 5: When a student participating in the live stream becomes distracted and the distraction index exceeds the pre-set standard value, the distracted student's window on the live stream screen will shake, and the lecturer's teaching window will also shake to alert the teacher. The distracted student's name will then appear in the teaching window via a barrage of comments.

[0030] As a further aspect of the present invention, the standard value set in step five is determined based on the playback of the live teaching video. By comparing the student's performance with the distraction index, and setting the distraction index based on multiple live playbacks, the numerical range of the student's distraction is displayed.

[0031] The beneficial technical effects of this invention are as follows: According to the live-streaming teaching analysis method based on image point pattern matching, this invention uses image point matching patterns combined with the Hungarian algorithm to match features of student images. This accurately matches students after changes in head posture, calculates changes in student head posture during live-streaming teaching, and then evaluates student performance during live-streaming teaching based on these changes. The formula in step four allows for the calculation of each student's distraction index, facilitating direct calculation of student distraction levels during live-streaming teaching. It also allows for monitoring students' learning status while teachers present teaching materials and provides prompts to teachers explaining the materials, promptly reminding them to monitor students' learning status and improving the interactivity of live-streaming teaching. Furthermore, it enhances the learning effectiveness of students in live-streaming classes. The distraction index also provides a basis for teachers to grade end-of-term teaching performance, facilitating the recording of student participation in each live-streaming session. This solves the problem that existing internet live-streaming learning platforms cannot combine real-time analysis of student focus based on their facial images during live-streaming. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall structure of the live teaching analysis method based on human face point pattern matching according to the present invention. Detailed Implementation

[0033] To enable those skilled in the art to understand the technical solution of the present invention more clearly, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0034] like Figure 1 As shown in this embodiment, the live teaching analysis method based on human face point pattern matching includes:

[0035] Step 1: Capture student images at the start of the live teaching session;

[0036] Step 2: Extract the triplet portrait image G for each student based on the characteristics of each student's portrait. Construct a set of points to be matched for each student's triplet portrait image G. Then, construct a Laplace matrix for each set of points to be matched. Perform singular decomposition on the student portrait image to obtain the eigenvalues ​​and eigenvectors of the student portrait image. Construct an initial matching relation matrix using the decomposed eigenvalues ​​and eigenvectors. Combine the Hungarian algorithm to minimize the matching cost between the point sets and obtain a new matching relation matrix, thus realizing the matching of student portrait feature points.

[0037] Step 3: Set up a portrait matching task to be performed every 10 minutes, with each matching session lasting 2 minutes and the number of images matched set to 4.

[0038] Step 4: When matching each student's image, set the formula for the student's distraction index as follows:

[0039]

[0040] In the formula: Z c Let A be the student distraction index for image matching. c Let B be the sum of the moving velocities of the image feature points of the target student. c Let α be the sum of the moving speeds of the image feature points of all students except the target student, and let α, β and γ be constant coefficients that take into account the different values ​​of the student categories.

[0041] in:

[0042] In the formula: x i Let t be the distance moved by each feature point during each portrait matching process. i Let m be the movement time of each feature point during each portrait matching process. c Let n be the feature area enclosed by the feature points of the target student's face, and let n be the number of feature points of the target student.

[0043]

[0044] In the formula: x i,j Let t be the distance moved by each feature point of a single student other than the target student during each matching process. i,j Let v be the movement time of each feature point of each student other than the target student when matching portraits, and v is the total number of students participating in the live broadcast.

[0045] Step 5: When a student participating in the live broadcast becomes distracted and their distraction index exceeds the pre-set standard value, the border of the distracted student's avatar on the live broadcast screen turns red, and a message is displayed at the bottom of the screen indicating the student's distraction index and name.

[0046] Step Six: Record the students' distraction index scores for each live class to provide a basis for teachers to grade participation scores in the academic credit assessment.

[0047] By combining image point matching with the Hungarian algorithm to match student features, it can accurately identify students after changing their head posture. It calculates changes in student head posture during live teaching and uses these changes to evaluate student performance. The formula in step four calculates a distraction index for each student, facilitating direct monitoring of distraction levels during live teaching. This allows for monitoring student learning status while teachers present materials and provides prompts to teachers, enhancing interactivity and improving learning outcomes. Furthermore, the distraction index provides a basis for teachers to grade end-of-term performance, facilitating the recording of student participation in each live lesson. This addresses the shortcomings of existing internet live learning platforms that cannot analyze student focus in real-time using their facial images.

[0048] The formula used in step six to provide the basis for teachers' assessment of class participation grades is as follows:

[0049]

[0050] In the formula: P c This is recorded as the student's participation score for the semester, with 30 being the total participation score. c,h The data is recorded as the student's distraction index for each live stream, h is the live stream number, and d is the number of live stream teaching sessions per semester.

[0051] By setting up a formula for assigning participation marks, teachers can easily redefine them based on students' performance in each live class. This avoids the limitations of a simple attendance system that evaluates students' participation based solely on their presence or absence. It improves the comprehensiveness and scientific nature of participation marks, enabling teachers to motivate and encourage students to concentrate during live classes. This helps improve the effectiveness of live teaching and fosters a positive learning attitude among students during live classes.

[0052] The students in step one are primary school students, middle school students, or university students.

[0053] By using elementary, middle, or university students in step one, this method of assessing distractibility can be easily applied to elementary, middle, and university levels, making the assessment method universal and clarifying its scope of application.

[0054] In step two, the number of feature points in the triplet portrait image G for each student is set to 15 when the student is a primary school student, 20 when the student is a middle school student, and 25 when the student is a university student.

[0055] The number of feature points in the image G for elementary school students is set to 15 because their faces occupy a smaller area compared to middle and high school students, resulting in smaller feature points, edges, and movement distances. Five feature points are allocated to the crown / hat area to capture different hairstyles, hair accessories, and hat wearing patterns. Ten different feature points are used in the face area to capture the unique features of different students, facilitating differentiation and identification. The number of feature points in the image G for middle school students is set to 20 because their faces occupy a larger area compared to elementary school students, resulting in larger feature points, edges, and movement distances. Ten feature points are allocated to the crown / hat area to capture the head features of middle school students. The study focuses on the hair accessories, hairstyles, and hat wearing of middle school students. Since the head details of middle school students are richer than those of primary school students, it's easier to distinguish their facial features. Ten feature points are set on the faces of middle school students to differentiate them based on their facial characteristics. Twenty-five feature points are set on the heads of university students because their hairstyles and face shapes offer more differences from middle and primary school students. Furthermore, the facial contour area of ​​university students is larger than that of primary and middle school students. Fifteen feature points are set in the crown and hat area of ​​university students to extract more detailed feature descriptions, facilitating the differentiation of features in this area among different students. Fifteen feature points are also set on the face to provide more detailed descriptions of university students' facial features, making it easier to identify different student faces through graphic transformations and distinguish the facial features of different students in live-streamed teaching.

[0056] In step three, it is possible to set a face matching task to be performed every ten minutes or every five minutes. When setting a face matching task to be performed every five minutes, the duration of each matching task is set to 1 minute and the number of matching images is set to 4.

[0057] By setting the time interval for the human image matching task in step three to once every five minutes, it is possible to set different matching durations according to the duration of the live teaching. This allows for setting the interval of the image matching task to the duration of the live broadcast, improving the personalization and adaptability of the matching time. By setting the time interval for the matching task to five minutes and the duration of each matching to one minute, the matching can be performed continuously for the full duration, which is conducive to capturing the students' learning and class status in real time.

[0058] In step four, the constant values ​​of α, β, and γ are set to consider the different ages of the student categories and are respectively:

[0059] When the student category is primary school students, α = 1.25, β = 0.23, γ = 1;

[0060] When the student category is middle school students, α = 1.18, β = 0.21, γ = 0.85;

[0061] When the student category is university students, α = 1.01, β = 0.15, and γ = 0.66.

[0062] By setting different α, β, and γ values ​​in step four for different students, the formula for the distraction index can be adjusted according to the students' age. For elementary school students, middle school students, and university students, α gradually decreases because elementary school students have shorter attention spans compared to middle school and university students. Furthermore, due to their younger age, their range of motion is smaller compared to middle school and university students. Therefore, the first item of the distraction index... A larger constant coefficient is set to expand the value so that the distraction index numerically meets the generally established empirical threshold for classroom concentration. The value of β decreases sequentially from elementary school students to middle school students to university students because the range of motion of students other than the target student increases accordingly; therefore, the second term needs to be reduced. The value of γ is used as a constant adjustment value to supplement the overall parameters. For elementary school students, the supplemented value is significantly larger than that for middle school and university students. The purpose of this setting is not only to consider the distractibility of the target student, but also to consider the classmates who are learning together in the live broadcast. By referring to the action status of other students, the distractibility index will be lower when other students are moving, which means that everyone is discussing questions or taking a break. When other students are swaying or moving slightly, if one student moves more, it means that student is distracted. This setting makes the distractibility index more diversified and referential, avoiding blindly and simplistically evaluating the state of the target student, which would make the evaluation of distractibility index unscientific and inaccurate.

[0063] Step five can also be set as follows:

[0064] Step 5: When a student participating in the live stream becomes distracted and the distraction index exceeds the pre-set standard value, the distracted student's window on the live stream screen will shake, and the lecturer's teaching window will also shake to alert the teacher. The distracted student's name will then appear in the teaching window via a barrage of comments.

[0065] By setting different steps in step five, teachers can be given diverse prompts to pay attention to students' learning status, thus avoiding the situation where teachers are so focused on explaining the materials that they cannot pay attention to students' learning status.

[0066] The standard value set in step five is determined based on the playback of the live teaching video. By comparing the students' performance with the distraction index, and setting the distraction index based on multiple live playbacks, the range of students' distraction values ​​is displayed.

[0067] By reviewing historical live video replays, teachers can compare students' learning status with distraction indicators to identify different thresholds for these indicators. This also helps avoid analytical errors caused by image matching and capture mistakes. By setting different distraction indicator thresholds, teachers can easily develop specific scoring standards and assign different scores based on different distraction indicator ranges, thus improving the comprehensiveness and rationality of daily scores.

[0068] In summary, this embodiment, based on the live-stream teaching analysis method using facial feature pattern matching, allows teachers to redefine attendance scores by considering students' performance in each live stream. This avoids the simplistic evaluation of attendance based solely on attendance, improving the comprehensiveness and scientific rigor of the attendance score system. This approach incentivizes and encourages students to concentrate during live-stream teaching, enhancing its effectiveness and fostering a more positive learning attitude. By including elementary, middle, or university students in step one, this distraction assessment method can be applied across elementary, middle, and university levels, making it universally applicable and clearly defining its scope. The number of feature points in the image G of primary school students is set to 15 because the facial area of ​​primary school students is smaller than that of middle school and university students, and the movement distance of the monitored feature points, edges, and feature points is also smaller. Five feature points are set in the crown and hat area to capture different hairstyles, hair accessories, and hat wearing patterns of primary school students. Ten different feature points are set in the facial area to capture the different features of different students, making it easier to distinguish and identify primary school students. The number of feature points in the image G of middle school students is set to 20 because the facial area of ​​middle school students is larger than that of primary school students, and the movement distance of the monitored feature points, edges, and feature points is also larger. Ten feature points are allocated to the crown and hat area to capture the hairstyles, hair accessories, and hat wearing patterns of middle school students. Since the head details of middle school students are richer than those of primary school students, it is easier to distinguish facial features. Ten feature points are also set in the facial area of ​​middle school students to differentiate them based on their facial features. The number of feature points in the image G of university students is set to 25. The reason for using 15 feature points is that university students' hairstyles and facial shapes differ more from those of middle and primary school students. Additionally, the facial contour area of ​​university students is larger than that of middle and primary school students. Setting 15 feature points in the university student's hat / headwear area allows for more detailed feature descriptions, facilitating the differentiation of different students' hat / headwear features. By setting 15 feature points on the face, more feature descriptions of university students' facial characteristics can be obtained, making it easier to identify different student faces through image transformation and distinguish different student facial features in live teaching. Furthermore, by setting the image matching task time interval in step three to once every five minutes, it is possible to set different matching durations according to the duration of the live teaching. This allows for setting the interval of the image matching task to the duration of the live broadcast, improving the personalization and adaptability of the matching time. By setting the matching task time interval to five minutes and the matching duration to one minute, continuous matching can be performed for the full duration, facilitating real-time capture of students' learning and class status.By setting different α, β, and γ values ​​in step four for different students, it is easy to adjust the distraction index formula according to the students' age. For elementary school students, middle school students, and university students, α gradually decreases because elementary school students have a shorter attention span compared to middle school and university students, and because they are younger, their range of motion is smaller than that of middle school and university students. Therefore, the first item of the distraction index is reduced. A larger constant coefficient is set to expand the value so that the distraction index numerically meets the generally established empirical threshold for classroom concentration. The value of β decreases sequentially from elementary school students to middle school students to university students because the range of motion of students other than the target student increases accordingly; therefore, the second term needs to be reduced. The value of γ is used as a constant adjustment value to supplement the overall parameters. For elementary school students, the supplemented value is significantly larger than that for middle school and university students. This setting aims not only to consider the distractibility of the target student but also to consider the surrounding classmates who are also learning live. By referring to the action status of other students, the distractibility index will be lower when other students are moving, indicating that everyone is discussing questions or taking a break. When other students are swaying or moving slightly, if one student moves more significantly, it indicates that this student is distracted. This setting makes the distractibility index more diverse and referential, avoiding blindly evaluating the state of the target student in a single way, which would make the evaluation of distractibility index unscientific and inaccurate. By setting different prompts in step five, it is easier to provide diverse prompts for teachers to pay attention to the students' learning status, avoiding the situation where teachers are focused on explaining the materials and cannot pay attention to the students' learning status. By reviewing historical live video replays, teachers can compare students' learning status with distraction indicators to identify different thresholds for these indicators. This also helps avoid analytical errors caused by image matching and capture mistakes. By setting different distraction indicator thresholds, teachers can easily develop specific scoring standards and assign different scores based on different distraction indicator ranges, thus improving the comprehensiveness and rationality of daily scores.

[0069] The above description is merely a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A live-streaming teaching analysis method based on human face point pattern matching, characterized in that, The method includes: Step 1: Capture student images at the start of the live teaching session; Step 2: Extract the triplet portrait image G for each student based on the characteristics of each student's portrait. Construct a set of points to be matched for each student's triplet portrait image G. Then, construct a Laplace matrix for each set of points to be matched. Perform singular decomposition on the student portrait image to obtain the eigenvalues ​​and eigenvectors of the student portrait image. Construct an initial matching relation matrix using the decomposed eigenvalues ​​and eigenvectors. Combine the Hungarian algorithm to minimize the matching cost between the point sets and obtain a new matching relation matrix, thus realizing the matching of student portrait feature points. Step 3: Set up a portrait matching task to be performed every 10 minutes, with each matching session lasting 2 minutes and the number of images matched set to 4. Step 4: When matching each student's image, set the formula for the student's distraction index as follows: ， In the formula: This is denoted as the student distraction index for image matching. Let be the sum of the moving speeds of the image feature points of the target student. Let be the sum of the moving velocities of the image feature points of all students except the target student. , and All are denoted as constant coefficients that take into account the different numerical values ​​of student categories; in: , In the formula: This is denoted as the distance moved by each feature point during each portrait matching process. This is denoted as the movement time of each feature point during each portrait matching process. Let the area enclosed by the facial feature points of the target student be denoted as . The number of feature points of the target student is recorded. ， In the formula: Let this be the distance moved by each feature point of a single student (excluding the target student) during each matching process. Let v be the movement time of each feature point of each student other than the target student when matching portraits, and v is the total number of students participating in the live broadcast. Step 5: When a student participating in the live broadcast becomes distracted and their distraction index exceeds the pre-set standard value, the border of the distracted student's avatar on the live broadcast screen turns red, and a message is displayed at the bottom of the screen indicating the student's distraction index and name. Step Six: Record the students' distraction index scores for each live class to provide a basis for teachers to grade participation scores in the academic credit assessment. Step five can be set as follows: When a student participating in the live broadcast becomes distracted and the distraction index exceeds the pre-set standard value, the distracted student's window on the live broadcast screen shakes, and the teacher's teaching window shakes as a notification. The teacher's teaching window displays the distracted student's name via a barrage of comments. The pre-set standard value in step five is determined based on the replay of the live teaching video. By comparing the student's performance with the distraction index, and setting the range of distraction index values ​​for displaying student distraction based on multiple live broadcast replays, the system can effectively prevent such distraction.

2. The live teaching analysis method based on human face point pattern matching as described in claim 1, characterized in that, The formula used in step six to provide the basis for teachers' assessment of class participation grades is as follows: ， In the formula: This score is recorded as the student's participation grade for the semester, with 30 being recorded as the total participation grade. The data is recorded as the student's distraction index for each live stream, h is the live stream number, and d is the number of live stream teaching sessions per semester.

3. The live teaching analysis method based on human face point pattern matching as described in claim 1, characterized in that, The students in step one are primary school students, middle school students, or university students.

4. The live teaching analysis method based on human face point pattern matching as described in claim 1, characterized in that, In step two, the number of feature points in the triplet portrait image G for each student is set to 15 when the student is a primary school student, 20 when the student is a middle school student, and 25 when the student is a university student.

5. The live teaching analysis method based on human face point pattern matching as described in claim 1, characterized in that, In step three, it is possible to set a face matching task to be performed every ten minutes or every five minutes. When setting a face matching task to be performed every five minutes, the duration of each matching task is set to 1 minute and the number of matching images is set to 4.

6. The live teaching analysis method based on human face point pattern matching as described in claim 1, characterized in that, In step four , and The constant values ​​are set to take into account the different ages of the student categories, and are respectively set as follows: When the student category is primary school students , , ; When the student category is secondary school student , , ; When the student category is university student , , .

Images