Classroom effect evaluation method and system based on multi-modal attention data analysis

Abstract

The invention discloses a classroom effect evaluation method and system based on multi-modal attention data analysis, relates to the technical field of classroom education systems, and specifically comprises the following steps: constructing a mind map of a classroom explanation scheme, presetting understanding difficulty and an explanation sequence for each map node, and forming an explanation sequence line; extracting a plurality of explanation keywords, and associating the explanation keywords with the corresponding mapping nodes; voice information is obtained, real-time explanation characters are extracted and analyzed, real-time explanation keywords are determined, and corresponding map nodes are positioned in the mind map; by analyzing the facial expressions and limb actions of the students in the real-time classroom video and combining the understanding difficulty of different mapping nodes, the initial state evaluation of the students is corrected to obtain the final state evaluation, and the sum of the final state evaluation is used as the classroom effect evaluation. And a basis is provided for improvement of subsequent classroom explanation schemes.

Description

Technical Field

[0001] The present invention relates to the technical field of classroom education systems, and in particular to a classroom effect evaluation method and system based on multimodal attention data analysis. Background Art

[0002] In the traditional education environment, the evaluation of classroom effectiveness has always been a complex and multidimensional problem. For a long time, teachers have mainly relied on their own teaching experience, students' feedback and simple test scores to evaluate classroom effectiveness. Although these methods can reflect the classroom situation to a certain extent, they often lack comprehensiveness and objectivity. Especially in large-scale teaching environments, how to accurately and efficiently evaluate classroom effectiveness has become a technical problem that needs to be solved in the field of education. Summary of the invention

[0003] The purpose of the present invention is to provide an evaluation method and system that can accurately evaluate classroom effects.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] Classroom effectiveness evaluation method based on multimodal attention data analysis, including:

[0006] Construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, connect the map nodes in sequence to obtain the explanation sequence line;

[0007] Conduct semantic analysis on the classroom explanation plan, identify several explanation keywords, and associate the explanation keywords with corresponding map nodes;

[0008] Acquire real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and determine corresponding map nodes in the mind map based on a number of real-time explanation keywords that appear in a preset time period;

[0009] Obtain real-time classroom video information, analyze students' facial expressions and body movements in the real-time classroom information, determine the initial status evaluation of a single student, and make the first correction to the initial status evaluation based on the difficulty of understanding different map nodes to obtain the student's final status evaluation. The cumulative sum of all students' final status evaluations is recognized as the classroom effect evaluation.

[0010] Based on the corresponding part of the classroom teaching content in the mind map, a classroom knowledge point test answer sheet is generated, and several students answer it. Based on the relationship between the answer sheet scores and the classroom effectiveness evaluation, the credibility of the classroom effectiveness evaluation is determined.

[0011] In some embodiments disclosed in the present invention, the method for semantically analyzing a classroom explanation program includes:

[0012] Predefine the keywords of the knowledge points involved in the classroom explanation plan;

[0013] Step S202, based on the position of the knowledge point keyword in the text of the classroom explanation plan, determining the secondary keywords with semantic association before and after the knowledge point keyword, and combining the knowledge point keyword and the secondary keyword based on the semantic association to obtain an explanation keyword group;

[0014] The text range corresponding to the explanation keyword group is determined, and other supplementary keywords in the text information whose occurrence frequency is greater than or equal to a preset value are determined, and the supplementary keywords are added to the explanation keyword group, and the explanation keyword group is associated with the corresponding map node.

[0015] In some embodiments disclosed in the present invention, based on a number of explanation keywords appearing in a preset time period, a method for generating corresponding map nodes in a mind map includes:

[0016] Determine the map nodes that have been explained in the mind map, and use the map nodes that have just been explained as the current positioning nodes, and select several map nodes after the current positioning nodes as comparison map nodes;

[0017] Compare the real-time explanation keywords that appear in the preset time period with the explanation keyword groups corresponding to the comparison map nodes, determine the mapping quantity of the real-time explanation keywords and different explanation keyword groups, and calculate the ratio of the mapping quantity to the keyword quantity of the real-time explanation keywords in the preset time period, which is recorded as the comprehensive mapping ratio;

[0018] Based on the comprehensive mapping ratio and the ratio of various keywords mapped in the explanation keyword group, the mapping matching parameters of the real-time explanation keywords and the explanation keyword group appearing in the preset time period are determined, and based on the size of the mapping matching parameters, the explanation keyword group corresponding to the real-time keyword is determined, and the map node corresponding to the explanation keyword group is identified as the real-time keyword as the corresponding map node.

[0019] In some embodiments disclosed in the present invention, the method for determining the mapping matching parameters of the real-time explanation keywords and the explanation keyword groups appearing within a preset time period includes:

[0020] Determine a first mapping number of knowledge point keywords mapped in the explanation keyword group, a second mapping number of secondary keywords mapped, and a third mapping number of complementary keywords mapped;

[0021] Calculating a first mapping ratio of the first mapping quantity to the number of real-time keywords within a preset time period, calculating a second mapping ratio of the second mapping quantity to the number of real-time keywords within a preset time period, and calculating a third mapping ratio of the third mapping quantity to the number of real-time keywords within a preset time period;

[0022] Determining a mapping matching parameter based on the first mapping ratio, the second mapping ratio, the third mapping ratio, and the comprehensive mapping ratio;

[0023] The expression for calculating the mapping matching parameter is:

[0024] Y=R×U zong ×exp{[K1×U1+K2×U2+K3×U3]+b};

[0025] Among them, Y is the mapping matching parameter, R is the mapping matching parameter conversion coefficient, and U zong is the comprehensive mapping ratio, U1 is the first mapping ratio, U2 is the second mapping ratio, U3 is the third mapping ratio, K1 is the knowledge point keyword weight adjustment coefficient, K2 is the secondary keyword weight adjustment coefficient, K3 is the supplementary keyword weight adjustment coefficient, and b is the mapping ratio adjustment constant.

[0026] In some embodiments disclosed in the present invention, a method for analyzing facial expressions and body movements of students in real-time classroom video information to determine an initial status evaluation of a single student includes:

[0027] Using the preset facial recognition model and body recognition model, the facial expressions and body movements of students in the real-time classroom video information are recognized and analyzed, wherein the facial expressions include focused learning expressions, doubtful expressions, and unfocused learning expressions, and the body movements include static and corrected states, static and uncorrected states, and active and uncorrected states, wherein each facial expression and each body movement is configured with a specific unit evaluation parameter;

[0028] Continuous time node analysis is performed on each student’s facial expressions and body movements, and based on the analysis results, the initial status assessment of a single student is determined.

[0029] In some embodiments disclosed in the present invention, the method for first correcting the initial state assessment in combination with the difficulty of understanding different map nodes includes:

[0030] Step S403, for the course of classroom teaching, a time process reference line is established, the combination of facial expression factors and body movement factors determined at different time nodes is recorded as a state factor group, and the state factor group is mapped on the time process reference line according to the corresponding time node;

[0031] Based on the time nodes corresponding to the map nodes explained during the class, mark the time segments on the time process reference line, and associate the understanding difficulty corresponding to the map nodes with the corresponding time segments;

[0032] An initial sub-state evaluation is performed on each state factor, and based on the time segment to which the time node corresponding to the state factor belongs, a correction feature for the initial sub-state evaluation is determined.

[0033] In some implementations of the present disclosure, the expression for calculating the initial status evaluation of a single student is:

[0034]

[0035] Among them, P is the initial state evaluation of a single student, α t is the unit evaluation parameter of facial expression at the i-th time node, β t is the unit evaluation parameter of the limb action at the i-th time node, δ[t] is the judgment function of the continuous time nodes, if there are α corresponding to each of the continuous time nodes t +β t If the difficulty of the tth time node is greater than or equal to the preset value, the corresponding number of continuous time nodes is determined, and the maximum number of continuous time nodes is selected. δ[t] determines the preset continuity correction parameter according to the maximum number of continuous time nodes. H[...] is the difficulty judgment function. If the difficulty of the tth time node is greater than or equal to the preset value, and α t +β t If it is a negative value, H[...] outputs 0. If the difficulty of the tth time node is less than or equal to the preset value, H[...] outputs α t +β t , if the difficulty of the tth time node is greater than or equal to the preset value, then H[...] outputs [α t +β t ]×h, where h is the preset difficulty multiplier factor.

[0036] In some embodiments disclosed in the present invention, based on the relationship between the answer sheet score and the classroom effect evaluation, the method for determining the credibility of the classroom effect evaluation includes:

[0037] In view of the relationship between the answer sheet scores and the classroom effect evaluation, a score interval group and a classroom effect evaluation interval group are established, and the score interval and the classroom effect evaluation interval correspond one-to-one to form an interval corresponding group;

[0038] Determine the score range to which the answer sheet score belongs and the classroom effectiveness range to which the classroom effectiveness evaluation belongs respectively. If the respective intervals are consistent with the interval corresponding groups, the classroom effectiveness evaluation is deemed to be credible.

[0039] In some embodiments disclosed in the present invention, there is also disclosed a classroom effect evaluation system based on multimodal attention data analysis, including:

[0040] The first module is used to construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, the map nodes are connected in sequence to obtain the explanation sequence line;

[0041] The second module is used to perform semantic analysis on the classroom explanation plan, determine several explanation keywords, and associate the explanation keywords with corresponding map nodes;

[0042] The third module is used to obtain real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and determine corresponding map nodes in the mind map based on a number of real-time explanation keywords that appear in a preset time period;

[0043] The fourth module is used to obtain real-time classroom video information, analyze the facial expressions and body movements of students in the real-time classroom information, determine the initial status evaluation of a single student, and make the first correction to the initial status evaluation based on the difficulty of understanding different map nodes to obtain the final status evaluation of the student. The cumulative sum of the final status evaluations of all students is recognized as the classroom effect evaluation;

[0044] The fifth module is used to generate a classroom knowledge point test answer sheet based on the corresponding part of the classroom teaching content in the mind map, and a number of students will answer it. Based on the relationship between the answer sheet scores and the classroom effectiveness evaluation, the credibility of the classroom effectiveness evaluation is determined.

[0045] The present invention discloses a classroom effect evaluation method and system based on multimodal attention data analysis, which relates to the technical field of classroom education systems. Specifically, a mind map of a classroom explanation plan is constructed, and the difficulty of understanding and the order of explanation are preset for each map node to form an explanation order line; a number of explanation keywords are extracted and associated with corresponding map nodes; voice information is acquired, real-time explanation text is refined and analyzed, real-time explanation keywords are determined, and corresponding map nodes are located in the mind map; by analyzing students' facial expressions and body movements in real-time classroom videos, combined with the difficulty of understanding different map nodes, the students' initial state evaluation is corrected to obtain a final state evaluation, and the cumulative sum thereof is used as a classroom effect evaluation. The present invention achieves a more accurate evaluation of classroom effects through the above technical solution, and provides a basis for the improvement of subsequent classroom explanation plans. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1This is a method step diagram of a classroom effectiveness evaluation method based on multimodal attention data analysis disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0048] The purpose of the present invention is to provide an evaluation method and system that can accurately evaluate classroom effects.

[0049] In order to achieve the above object, the present invention adopts the following technical solutions:

[0050] See also Figure 1 , a classroom effect evaluation method based on multimodal attention data analysis, including:

[0051] Step S100, construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, the map nodes are connected in sequence to obtain the explanation sequence line.

[0052] The core purpose of constructing a mind map for classroom explanation plans is not only to provide a clear knowledge point structure, but more importantly to provide the system with the function of locating explanations. By presenting complex knowledge points in a structured way in the mind map, the system can clearly identify the location of each knowledge point and the relationship between them. In this way, in a real-time classroom environment, the system can quickly and accurately locate the corresponding node in the mind map according to the teacher's explanation content. At the same time, the difficulty of understanding each map node and the order of explanation are preset to ensure that the system can locate knowledge points in a reasonable order and depth, so as to better support classroom effect evaluation and provide teachers with targeted teaching feedback.

[0053] Step S200, semantic analysis is performed on the classroom explanation plan to determine a number of explanation keywords, and the explanation keywords are associated with corresponding map nodes.

[0054] In some embodiments disclosed in the present invention, the method for semantically analyzing a classroom explanation program includes:

[0055] Step S201, pre-define the knowledge point keywords involved in the classroom teaching plan.

[0056] Step S202, based on the position of the knowledge point keywords in the text of the classroom explanation plan, the secondary keywords with semantic association before and after the knowledge point keywords are determined, and based on the semantic association, the knowledge point keywords and the secondary keywords are combined to obtain the explanation keyword group.

[0057] Determine the secondary keywords that have semantic associations before and after them. The principle of this step is to use semantic analysis technology to identify other keywords that are closely related to the knowledge point keywords in semantics. Although these secondary keywords may not be the core of the explanation content, they are of great significance for a comprehensive understanding of the knowledge points and the construction of a complete knowledge system. By combining these secondary keywords with the knowledge point keywords, a more comprehensive and accurate explanation keyword group can be obtained.

[0058] Step S203, determine the text range corresponding to the explanation keyword group, and determine other supplementary keywords in the text information whose occurrence frequency is greater than or equal to a preset value, add the supplementary keywords to the explanation keyword group, and associate the explanation keyword group with the corresponding map node.

[0059] The principle of this step is to identify additional keywords that are important for understanding knowledge points, based on an in-depth understanding and analysis of the content of the lecture. These additional keywords may be recurring concepts, terms or phrases in the content of the lecture.

[0060] Step S300, obtain real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and based on a number of real-time explanation keywords that appear in a preset time period, determine the corresponding map nodes in the mind map.

[0061] The purpose of acquiring real-time classroom voice information and extracting text is to capture the teacher's explanation in real time and convert it into analyzable text data. The implementation of this step relies on advanced speech recognition and text extraction technology to ensure that the extracted text data is accurate. The purpose of semantic analysis of the extracted real-time explanation text information is to determine the keywords and themes of the teacher's current explanation, so as to judge the progress and focus of the explanation. The system can determine whether the content of the teacher's current explanation corresponds to the preset knowledge point structure by analyzing the keywords and sentence structures in the real-time explanation text information.

[0062] In some embodiments disclosed in the present invention, based on a number of explanation keywords appearing in a preset time period, a method for generating corresponding map nodes in a mind map includes:

[0063] Step S301, determining the map nodes that have been explained in the mind map, and taking the map nodes that have just been explained as the current positioning nodes, and selecting several map nodes after the current positioning nodes as comparison map nodes;

[0064] Step S302, comparing the real-time explanation keywords that appear in a preset time period with the explanation keyword groups corresponding to the comparison map nodes, determining the mapping quantity of the real-time explanation keywords and the different explanation keyword groups, and calculating the ratio of the mapping quantity to the keyword quantity of the real-time explanation keywords in the preset time period, which is recorded as the comprehensive mapping ratio;

[0065] Step S303, based on the comprehensive mapping ratio and the ratio of each type of keywords mapped in the explanation keyword group, determine the mapping matching parameters between the real-time explanation keywords and the explanation keyword group that appear within the preset time period, and based on the size of the mapping matching parameters, determine the explanation keyword group corresponding to the real-time keyword, and identify the map node corresponding to the explanation keyword group as the corresponding map node.

[0066] In some embodiments disclosed in the present invention, the method for determining the mapping matching parameters of the real-time explanation keywords and the explanation keyword groups appearing within a preset time period includes:

[0067] Step S3031, determining a first mapping number of knowledge point keywords mapped in the explanation keyword group, a second mapping number of secondary keywords mapped, and a third mapping number of complementary keywords mapped.

[0068] Step S3032, calculate a first mapping ratio of the first mapping quantity to the number of real-time keywords within a preset time period, calculate a second mapping ratio of the second mapping quantity to the number of real-time keywords within a preset time period, and calculate a third mapping ratio of the third mapping quantity to the number of real-time keywords within a preset time period.

[0069] Step S3033, determining a mapping matching parameter based on the first mapping ratio, the second mapping ratio, the third mapping ratio and the comprehensive mapping ratio.

[0070] The expression for calculating the mapping matching parameter is:

[0071] Y=R×U zong ×exp{[K1×U1+K2×U2+K3×U3]+b}

[0072] Among them, Y is the mapping matching parameter, R is the mapping matching parameter conversion coefficient, and U zongis the comprehensive mapping ratio, U1 is the first mapping ratio, U2 is the second mapping ratio, U3 is the third mapping ratio, K1 is the knowledge point keyword weight adjustment coefficient, K2 is the secondary keyword weight adjustment coefficient, K3 is the supplementary keyword weight adjustment coefficient, and b is the mapping ratio adjustment constant.

[0073] Based on the mapping ratios of various keywords and the comprehensive mapping ratios, combined with the preset weight adjustment coefficients and mapping ratio adjustment constants, the mapping matching parameter is calculated. This parameter comprehensively considers the matching degree of various keywords and their importance in the explanation, and can fully reflect the matching between the real-time explanation content and the preset explanation keyword group. Through this parameter, the system can quantitatively evaluate the teacher's explanation effect and provide teachers with targeted teaching feedback.

[0074] Step S400, obtain real-time classroom video information, analyze the facial expressions and body movements of students in the real-time classroom information, determine the initial state evaluation of a single student, and make a first correction to the initial state evaluation based on the difficulty of understanding different map nodes to obtain the student's final state evaluation. The cumulative sum of all students' final state evaluations is recognized as the classroom effectiveness evaluation.

[0075] Acquiring real-time classroom video information and analyzing students' facial expressions and body movements is to obtain clues about their learning status from their non-verbal behaviors. Students' facial expressions and body movements can reflect their understanding of the content, interest, and fatigue, which are crucial for evaluating students' learning status. Correcting students' initial status assessment based on the difficulty of understanding different map nodes is to more accurately judge students' learning status at different knowledge points. The system can comprehensively evaluate students' learning status based on the difficulty of understanding map nodes and students' non-verbal behavior, and give more accurate judgments.

[0076] In some embodiments disclosed in the present invention, a method for analyzing facial expressions and body movements of students in real-time classroom video information to determine an initial status evaluation of a single student includes:

[0077] Step S401, using a preset facial recognition model and body recognition model, recognize and analyze the facial expressions and body movements of students in real-time classroom video information, wherein facial expressions include focused learning expressions, puzzled expressions, and unfocused learning expressions, and body movements include a static and correct state, a static and uncorrected state, and an active and uncorrected state, wherein each facial expression and each body movement is configured with a specific unit evaluation parameter.

[0078] Using advanced facial recognition and body recognition models, students' facial expressions and body movements in real-time classroom videos are accurately identified and analyzed. These models are based on deep learning technology and can accurately identify students' various facial expressions (such as focused learning, doubtful, unfocused learning, etc.) and body movements (such as still and upright, still and not upright, active and not upright, etc.). Each facial expression and body movement is assigned a specific unit evaluation parameter, which reflects the possible learning status of students under different expressions and movements.

[0079] Step S402, performing continuous time node analysis on the facial expressions and body movements of each student, and determining the initial state evaluation of a single student based on the analysis results.

[0080] The facial expressions and body movements of each student are analyzed at continuous time nodes. By analyzing the changes in students' expressions and movements over a period of time, the system can capture the dynamic changes in students' learning status and determine the initial status evaluation of a single student based on these analysis results. This evaluation comprehensively considers the performance of students in facial expressions and body movements, and can fully reflect the students' learning status in class, providing an important basis for subsequent classroom effect evaluation.

[0081] In some embodiments disclosed in the present invention, the method for first correcting the initial state assessment in combination with the difficulty of understanding different map nodes includes:

[0082] Step S403, a time process reference line is established for the classroom explanation process, the combination of facial expression factors and body movement factors determined at different time nodes is recorded as a state factor group, and the state factor group is mapped onto the time process reference line according to the corresponding time node.

[0083] Step S404, based on the time nodes corresponding to the map nodes explained during the classroom explanation, time segments are marked on the time process reference line, and the difficulty of understanding corresponding to the map nodes is associated with the corresponding time segments.

[0084] Step S405 , performing an initial sub-state evaluation on each state factor, and determining a correction feature for the initial sub-state evaluation based on the time segment to which the time node corresponding to the state factor belongs.

[0085] In some implementations of the present disclosure, the expression for calculating the initial status evaluation of a single student is:

[0086]

[0087] Among them, P is the initial state evaluation of a single student, α t is the unit evaluation parameter of facial expression at the i-th time node, βt is the unit evaluation parameter of the limb action at the i-th time node, δ[t] is the judgment function of the continuous time nodes, if there are α corresponding to each of the continuous time nodes t +β t If the difficulty of the tth time node is greater than or equal to the preset value, the corresponding number of continuous time nodes is determined, and the maximum number of continuous time nodes is selected. δ[t] determines the preset continuity correction parameter according to the maximum number of continuous time nodes. H[...] is the difficulty judgment function. If the difficulty of the tth time node is greater than or equal to the preset value, and α t +β t If it is a negative value, H[...] outputs 0. If the difficulty of the tth time node is less than or equal to the preset value, H[...] outputs α t +β t , if the difficulty of the tth time node is greater than or equal to the preset value, then H[...] outputs [α t +β t ]×h, where h is the preset difficulty multiplier factor.

[0088] Step S500, based on the corresponding part of the classroom teaching content in the mind map, a classroom knowledge point test answer sheet is generated, and a number of students answer it. Based on the relationship between the answer sheet score and the classroom effect evaluation, the credibility of the classroom effect evaluation is determined.

[0089] The purpose of generating a classroom knowledge point test answer sheet based on the corresponding part of the classroom explanation content in the mind map is to test students' mastery of knowledge points through testing. This test method can directly reflect students' understanding and memory of knowledge points, so it is one of the important means to evaluate classroom effectiveness. Asking students to answer the answer sheet and compare it based on the relationship between the answer sheet score and the classroom effect evaluation is to verify the accuracy and credibility of the classroom effect evaluation. If the answer sheet score is consistent or similar to the classroom effect evaluation result, it means that the classroom effect evaluation has a high credibility; if there is a large difference, it is necessary to further analyze the cause and adjust the evaluation method.

[0090] In some embodiments disclosed in the present invention, based on the relationship between the answer sheet score and the classroom effect evaluation, the method for determining the credibility of the classroom effect evaluation includes:

[0091] Step S501, according to the relationship between the answer sheet score and the classroom effect evaluation, a score interval group and a classroom effect evaluation interval group are established, and the score interval and the classroom effect evaluation interval are one-to-one corresponding to form an interval corresponding group.

[0092] Step S502, respectively determine the score interval to which the answer sheet score belongs and the classroom effect interval to which the classroom effect evaluation belongs. If the intervals to which they belong conform to the interval corresponding group, the classroom effect evaluation is deemed to be credible.

[0093] In some embodiments disclosed in the present invention, there is also disclosed a classroom effect evaluation system based on multimodal attention data analysis, including:

[0094] The first module is used to construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, the map nodes are connected in sequence to obtain the explanation sequence line;

[0095] The second module is used to perform semantic analysis on the classroom explanation plan, determine several explanation keywords, and associate the explanation keywords with corresponding map nodes;

[0096] The third module is used to obtain real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and determine corresponding map nodes in the mind map based on a number of real-time explanation keywords that appear in a preset time period;

[0097] The fourth module is used to obtain real-time classroom video information, analyze the facial expressions and body movements of students in the real-time classroom information, determine the initial status evaluation of a single student, and make the first correction to the initial status evaluation based on the difficulty of understanding different map nodes to obtain the final status evaluation of the student. The cumulative sum of the final status evaluations of all students is recognized as the classroom effect evaluation;

[0098] The fifth module is used to generate a classroom knowledge point test answer sheet based on the corresponding part of the classroom teaching content in the mind map, and a number of students will answer it. Based on the relationship between the answer sheet scores and the classroom effectiveness evaluation, the credibility of the classroom effectiveness evaluation is determined.

[0099] The present invention discloses a classroom effect evaluation method and system based on multimodal attention data analysis, which relates to the technical field of classroom education systems. Specifically, a mind map of a classroom explanation plan is constructed, and the difficulty of understanding and the order of explanation are preset for each map node to form an explanation order line; a number of explanation keywords are extracted and associated with corresponding map nodes; voice information is acquired, real-time explanation text is refined and analyzed, real-time explanation keywords are determined, and corresponding map nodes are located in the mind map; by analyzing students' facial expressions and body movements in real-time classroom videos, combined with the difficulty of understanding different map nodes, the students' initial state evaluation is corrected to obtain a final state evaluation, and the cumulative sum thereof is used as a classroom effect evaluation. The present invention achieves a more accurate evaluation of classroom effects through the above technical solution, and provides a basis for the improvement of subsequent classroom explanation plans.

[0100] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A classroom effect evaluation method based on multimodal attention data analysis, characterized in that: include: Construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, connect the map nodes in sequence to obtain the explanation sequence line; Conduct semantic analysis on the classroom explanation plan, identify several explanation keywords, and associate the explanation keywords with corresponding map nodes; Acquire real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and determine corresponding map nodes in the mind map based on a number of real-time explanation keywords that appear in a preset time period; Obtain real-time classroom video information, analyze students' facial expressions and body movements in the real-time classroom information, determine the initial status evaluation of a single student, and make the first correction to the initial status evaluation based on the difficulty of understanding different map nodes to obtain the student's final status evaluation. The cumulative sum of all students' final status evaluations is recognized as the classroom effect evaluation. Based on the corresponding part of the classroom teaching content in the mind map, a classroom knowledge point test answer sheet is generated, and several students answer it. Based on the relationship between the answer sheet scores and the classroom effectiveness evaluation, the credibility of the classroom effectiveness evaluation is determined.

2. The classroom effect evaluation method based on multimodal attention data analysis according to claim 1 is characterized in that: Methods for semantic analysis of classroom explanation plans include: Predefine the keywords of the knowledge points involved in the classroom explanation plan; Based on the position of the knowledge point keywords in the text of the classroom explanation plan, the secondary keywords with semantic association before and after the knowledge point keywords are determined, and based on the semantic association, the knowledge point keywords and the secondary keywords are combined to obtain the explanation keyword group; The text range corresponding to the explanation keyword group is determined, and other supplementary keywords in the text information whose occurrence frequency is greater than or equal to a preset value are determined, and the supplementary keywords are added to the explanation keyword group, and the explanation keyword group is associated with the corresponding map node.

3. The classroom effect evaluation method based on multimodal attention data analysis according to claim 2 is characterized in that: Based on a number of explanation keywords that appear in a preset time period, the methods for generating corresponding map nodes in a mind map include: Determine the map nodes that have been explained in the mind map, and use the map nodes that have just been explained as the current positioning nodes, and select several map nodes after the current positioning nodes as comparison map nodes; Compare the real-time explanation keywords that appear in the preset time period with the explanation keyword groups corresponding to the comparison map nodes, determine the mapping quantity of the real-time explanation keywords and different explanation keyword groups, and calculate the ratio of the mapping quantity to the keyword quantity of the real-time explanation keywords in the preset time period, which is recorded as the comprehensive mapping ratio; Based on the comprehensive mapping ratio and the ratio of various keywords mapped in the explanation keyword group, the mapping matching parameters of the real-time explanation keywords and the explanation keyword group appearing in the preset time period are determined, and based on the size of the mapping matching parameters, the explanation keyword group corresponding to the real-time keyword is determined, and the map node corresponding to the explanation keyword group is identified as the real-time keyword as the corresponding map node.

4. The classroom effect evaluation method based on multimodal attention data analysis according to claim 3 is characterized in that: The method for determining the mapping matching parameters of the real-time explanation keywords and the explanation keyword groups appearing within the preset time period includes: Determine a first mapping number of knowledge point keywords mapped in the explanation keyword group, a second mapping number of secondary keywords mapped, and a third mapping number of complementary keywords mapped; Calculating a first mapping ratio of the first mapping quantity to the number of real-time keywords within a preset time period, calculating a second mapping ratio of the second mapping quantity to the number of real-time keywords within a preset time period, and calculating a third mapping ratio of the third mapping quantity to the number of real-time keywords within a preset time period; Determining a mapping matching parameter based on the first mapping ratio, the second mapping ratio, the third mapping ratio, and the comprehensive mapping ratio; The expression for calculating the mapping matching parameter is: Y=R×U zong ×exp{[K1×U1+K2×U2+K3×U3]+b}; Among them, Y is the mapping matching parameter, R is the mapping matching parameter conversion coefficient, and U zong is the comprehensive mapping ratio, U1 is the first mapping ratio, U2 is the second mapping ratio, U3 is the third mapping ratio, K1 is the knowledge point keyword weight adjustment coefficient, K2 is the secondary keyword weight adjustment coefficient, K3 is the supplementary keyword weight adjustment coefficient, and b is the mapping ratio adjustment constant.

5. The classroom effect evaluation method based on multimodal attention data analysis according to claim 1 is characterized in that: Methods for analyzing students' facial expressions and body movements in real-time classroom video information to determine the initial status assessment of a single student include: Using the preset facial recognition model and body recognition model, the facial expressions and body movements of students in the real-time classroom video information are recognized and analyzed, wherein the facial expressions include focused learning expressions, doubtful expressions, and unfocused learning expressions, and the body movements include static and corrected states, static and uncorrected states, and active and uncorrected states, wherein each facial expression and each body movement is configured with a specific unit evaluation parameter; Continuous time node analysis is performed on each student’s facial expressions and body movements, and based on the analysis results, the initial status assessment of a single student is determined.

6. The classroom effect evaluation method based on multimodal attention data analysis according to claim 5 is characterized in that: Combined with the difficulty of understanding different map nodes, the methods for making the first correction to the initial status assessment include: According to the course of classroom explanation, a time process reference line is established, and the combination of facial expression factors and body movement factors determined at different time nodes is recorded as a state factor group, and the state factor group is mapped on the time process reference line according to the corresponding time node; Based on the time nodes corresponding to the map nodes explained during the class, mark the time segments on the time process reference line, and associate the understanding difficulty corresponding to the map nodes with the corresponding time segments; An initial sub-state evaluation is performed on each state factor, and based on the time segment to which the time node corresponding to the state factor belongs, a correction feature for the initial sub-state evaluation is determined.

7. The classroom effect evaluation method based on multimodal attention data analysis according to claim 6 is characterized in that: The expression for calculating the initial status evaluation of a single student is: Among them, P is the initial state evaluation of a single student, α t is the unit evaluation parameter of facial expression at the i-th time node, β t is the unit evaluation parameter of the limb action at the i-th time node, δ[t] is the judgment function of the continuous time nodes, if there are α corresponding to each of the continuous time nodes t +β t If the difficulty of the tth time node is greater than or equal to the preset value, the corresponding number of continuous time nodes is determined, and the maximum number of continuous time nodes is selected. δ[t] determines the preset continuity correction parameter according to the maximum number of continuous time nodes. H[...] is the difficulty judgment function. If the difficulty of the tth time node is greater than or equal to the preset value, and α t +β t If it is a negative value, H[...] outputs 0. If the difficulty of the tth time node is less than or equal to the preset value, H[...] outputs α t +β t , if the difficulty of the tth time node is greater than or equal to the preset value, then H[...] outputs [α t +β t ]×h, where h is the preset difficulty multiplier factor.

8. The classroom effect evaluation method based on multimodal attention data analysis according to claim 1 is characterized in that: Based on the relationship between the test scores and the classroom effectiveness evaluation, the methods for determining the credibility of the classroom effectiveness evaluation include: In view of the relationship between the answer sheet scores and the classroom effect evaluation, a score interval group and a classroom effect evaluation interval group are established, and the score interval and the classroom effect evaluation interval correspond one-to-one to form an interval corresponding group; Determine the score range to which the answer sheet score belongs and the classroom effectiveness range to which the classroom effectiveness evaluation belongs respectively. If the respective intervals are consistent with the interval corresponding groups, the classroom effectiveness evaluation is deemed to be credible.

9. Classroom effect evaluation system based on multimodal attention data analysis, characterized by: include: The first module is used to construct an explanation mind map for the classroom explanation plan, preset the difficulty of understanding for each map node, and define the explanation sequence of the map nodes of the explanation mind map. Based on the constructed explanation sequence, the map nodes are connected in sequence to obtain the explanation sequence line; The second module is used to perform semantic analysis on the classroom explanation plan, determine several explanation keywords, and associate the explanation keywords with corresponding map nodes; The third module is used to obtain real-time classroom voice information, extract real-time explanation text information from the real-time classroom voice information, and perform semantic analysis on the extracted real-time explanation text information to determine real-time explanation keywords, and determine corresponding map nodes in the mind map based on a number of real-time explanation keywords that appear in a preset time period; The fourth module is used to obtain real-time classroom video information, analyze the facial expressions and body movements of students in the real-time classroom information, determine the initial status evaluation of a single student, and make the first correction to the initial status evaluation based on the difficulty of understanding different map nodes to obtain the final status evaluation of the student. The cumulative sum of the final status evaluations of all students is recognized as the classroom effect evaluation; The fifth module is used to generate a classroom knowledge point test answer sheet based on the corresponding part of the classroom teaching content in the mind map, and a number of students will answer it. Based on the relationship between the answer sheet scores and the classroom effectiveness evaluation, the credibility of the classroom effectiveness evaluation is determined.

Images