Student attention assessment method and system based on big data analysis

Abstract

The invention provides a student attention assessment method and system based on big data analysis, and the method comprises the steps: capturing a class image of a student through a camera, collecting the facial expression of the student from the class image of the student, and recognizing the expression type of the student according to the facial expression of the student; acquiring student posture images from the student class images, and judging whether the students raise hands to answer questions or not according to the student posture images; attention scores are calculated from changes in the facial expression types and the body features of the students, and whether the attention of the students is concentrated is determined based on the attention scores. Through the method and the corresponding system, the attention of the student can be evaluated by using a plurality of sequences recorded by the video, and a more real result can be obtained.

Description

Technical Field

[0001] The present invention proposes a student attention assessment method and system based on big data analysis, and relates to the field of computer technology. Background Art

[0002] Classroom attention reveals the ability of students to focus on something for a certain period of time in class. It is also the basic quality for students to spontaneously and actively acquire information, improve knowledge and master skills in class. The quality of classroom attention has a very important impact on the effectiveness of lectures, learning outcomes and other evaluation methods. However, in traditional teaching scenarios, the popularization of compulsory education has increased the number of students. Without scientific quantitative standards and technical support, teachers need a long time span to observe and evaluate the attention status of all students, and more remain at the subjective level.

[0003] Moreover, the current dataset focuses on single images rather than image sequences, which are instantaneous features. When a person is in a certain state, his facial and body movements are a continuous process of change. The picture at a certain moment may not be able to significantly represent the state, and it is not possible to use several recorded sequences for evaluation to obtain more realistic results. Summary of the invention

[0004] The present invention provides a student attention assessment method and system based on big data analysis to solve the above-mentioned problems:

[0005] The present invention proposes a student attention assessment method based on big data analysis, the method comprising:

[0006] Capturing images of students in class through a camera, collecting students' facial expressions from the images of students in class, and identifying students' expression types based on their facial expressions;

[0007] Collect student posture images from students' class images, and judge whether students raise their hands to answer questions based on the student posture images;

[0008] The attention score is calculated from the changes in the students' facial expression types and body characteristics, and the students' concentration is determined based on the attention score.

[0009] Furthermore, the camera is used to capture images of students in class, and students' facial expressions are collected from the images of students in class. The types of students' expressions are identified according to their facial expressions, including:

[0010] Define students' expressions in class;

[0011] Label the facial expression dataset according to the definition of students’ facial expressions in class;

[0012] Divide the annotated expression dataset into training set, test set and validation set;

[0013] Build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model;

[0014] The image of the student in class is captured by a camera, the facial expression of the student is collected from the image of the student in class, and the facial expression of the student is input into the expression recognition model to obtain the expression type of the student.

[0015] Furthermore, we define students' expressions in class, including:

[0016] Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library;

[0017] When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection;

[0018] When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt;

[0019] Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening.

[0020] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding.

[0021] Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion.

[0022] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

[0023] Furthermore, collecting student posture images from the images of students in class, and judging whether the students raise their hands to answer questions based on the student posture images, including:

[0024] Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates;

[0025] The angle model is used to determine whether the student raises his hand. Specifically, the angle model is:

[0026]

[0027] When 45°≤α≤135°, the student is judged to be in the hand-raising state.

[0028] Furthermore, the attention score is calculated from the changes in the student's facial expression type and body characteristics, and whether the student's attention is focused is determined based on the attention score, including:

[0029] The attention score is calculated from the changes in the student's facial expression type and body features through the attention model. Specifically, the attention model is:

[0030]

[0031] Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered;

[0032] When A(t)≤0.3, the student’s attention is judged to be in a low state;

[0033] When 0.3<A(t)≤0.7, the student’s attention is judged to be in a general state;

[0034] When 0.7<A(t), it is judged that the student's attention is focused.

[0035] The present invention proposes a student attention assessment system based on big data analysis, the system comprising:

[0036] The module for identifying student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and identify students' expression types based on their facial expressions;

[0037] A hand-raising module is used to collect student posture images from students in class and judge whether students raise their hands to answer questions based on the student posture images;

[0038] The attention score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics, and determine whether the student is focused based on the attention score.

[0039] Furthermore, the module for identifying student expression types includes:

[0040] Define the expression module, which is used to define students' expressions in class;

[0041] The annotation module is used to annotate the expression dataset according to the definition of students' expressions in class;

[0042] The data set partitioning module is used to partition the annotated expression data set into a training set, a test set, and a validation set;

[0043] The training model module is used to build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model;

[0044] The module for obtaining student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and input students' facial expressions into the expression recognition model to obtain students' expression types.

[0045] Furthermore, the expression definition module includes:

[0046] Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library;

[0047] When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection;

[0048] When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt;

[0049] Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening.

[0050] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding.

[0051] Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion.

[0052] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

[0053] Furthermore, the module for determining whether to raise the hand includes:

[0054] Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates;

[0055] Angle judgment module, used to judge whether the student raises his hand through the angle model. Specifically, the angle model is:

[0056]

[0057] When 45°≤α≤135°, the student is judged to be in the hand-raising state.

[0058] Furthermore, the calculating of the attention score includes:

[0059] The score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics through the attention model. Specifically, the attention model is:

[0060]

[0061] Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered;

[0062] The low state determination module is used to determine that the student's attention is in a low state when A(t)≤0.3;

[0063] The module for judging the general state is used to judge that the student's attention is in the general state when 0.3<A(t)≤0.7;

[0064] The attention concentration determination module is used to determine that the student's attention is concentrated when 0.7<A(t).

[0065] The beneficial effects of the present invention are as follows: improving the quality of education, teachers can adjust teaching strategies in real time according to the feedback provided by the system, better attract and maintain students' attention, and thus improve teaching effectiveness; personalized teaching support, by analyzing the changes in the attention of individual students, teachers can provide more personalized guidance and support, especially for students who are easily distracted or need additional stimulation; data-supported educational decision-making, long-term collection and analysis of such data can help educators understand which parts of teaching activities are most able to attract students' attention and which parts may need improvement; improving student participation, by identifying behaviors such as raising hands, teachers can more fairly distribute speaking opportunities, increase student participation and classroom interaction; this technical solution uses advanced image analysis and artificial intelligence technology to provide real-time and accurate student attention and participation information in a real teaching environment, supporting more effective teaching and learning processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of a student attention assessment method based on big data analysis according to the present invention;

[0067] Figure 2 This is a map of 68 key points on the student's face. DETAILED DESCRIPTION

[0068] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. The embodiments described are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0071] One embodiment of the present invention is a method for evaluating student attention based on big data analysis, characterized in that the method comprises:

[0072] Capturing images of students in class through a camera, collecting students' facial expressions from the images of students in class, and identifying students' expression types based on their facial expressions;

[0073] Collect student posture images from students' class images, and judge whether students raise their hands to answer questions based on the student posture images;

[0074] The attention score is calculated from the changes in the students' facial expression types and body characteristics, and the students' concentration is determined based on the attention score.

[0075] The working principle and effect of the above technical solution are as follows: the camera system installed in the classroom is used to capture images of students in class, and these images are analyzed using advanced facial recognition technology (such as convolutional neural networks in deep learning) to detect and identify key points on the students' faces; these key points are analyzed to identify the types of students' expressions (such as happiness, anxiety, concentration, etc.); similarly, the posture images of students are captured through the camera, and the posture recognition algorithm (such as OpenPose or AlphaPose) is used to analyze the students' body posture, paying special attention to the position of the arms and the relative position of the head to determine whether the students have raised their hands; combining facial expression and posture data, a fusion algorithm (including weight distribution, Sigmoid and other mathematical models) is used to calculate the attention score of each student; this algorithm can integrate facial expression and body language data to more comprehensively evaluate the students' attention level; based on the calculated attention score, it is determined whether the students are focused, including setting a threshold, and when the attention score is lower than a certain threshold, the student is considered to be inattentive. Improve the quality of education. Teachers can adjust teaching strategies in real time based on the feedback provided by the system to better attract and maintain students' attention, thereby improving teaching effectiveness. Personalized teaching support. By analyzing the changes in attention of individual students, teachers can provide more personalized guidance and support, especially for students who are easily distracted or need additional stimulation. Data-supported educational decision-making. Long-term collection and analysis of such data can help educators understand which parts of teaching activities are most attractive to students and which parts may need improvement. Improve student participation. By identifying behaviors such as raising hands, teachers can distribute speaking opportunities more fairly, increase student participation and classroom interaction. This technical solution uses advanced image analysis and artificial intelligence technology to provide real-time and accurate student attention and participation information in a real teaching environment, supporting more effective teaching and learning processes.

[0076] An embodiment of the present invention is a method for evaluating student attention based on big data analysis, which captures images of students in class through a camera, collects students' facial expressions from the images of students in class, and identifies students' expression types based on their facial expressions, including:

[0077] Define students' expressions in class;

[0078] Label the facial expression dataset according to the definition of students’ facial expressions in class;

[0079] Divide the annotated expression dataset into training set, test set and validation set;

[0080] Build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model;

[0081] The image of the student in class is captured by a camera, the facial expression of the student is collected from the image of the student in class, and the facial expression of the student is input into the expression recognition model to obtain the expression type of the student.

[0082] The working principle and effect of the above technical solution are as follows: First, it is necessary to determine and define the types of expressions that may appear in students during class; these definitions are clear and specific to ensure the consistency and accuracy of the annotation of the expression data set; the annotation of the expression data set, collect image or video data containing students' expressions in class; manually annotate the images in the data set according to the pre-defined expression categories, and point out the type of expression of the students in each picture; divide the annotated data set into a training set, a test set, and a validation set; the training set is used to train the neural network model, the test set is used to evaluate the model performance, and the validation set is used to verify the model and adjust parameters during the training process; build a convolutional neural network model, design a suitable convolutional neural network architecture, which usually contains multiple convolutional layers, pooling layers, and fully connected layers; use the training set data to train the model to learn how to recognize different types of expressions; model evaluation and optimization, use the test set and validation set to evaluate the trained model; analyze the recognition accuracy and error rate of the model on various expressions, and adjust the parameters or structure of the model as needed for optimization; real-time expression recognition application, capture the image of students in class in real time through the camera. Extract the student's facial expression from the image and input it into the trained expression recognition model. The model outputs the student's expression type and provides real-time feedback. Improve the quality of teaching interaction. By identifying students' expressions, teachers can understand students' emotional state and participation level, and then adjust teaching strategies and rhythm to improve the effectiveness of teaching interaction. Early identification of students' learning difficulties and timely identification of students showing expressions of confusion or misunderstanding may help teachers intervene early and provide necessary guidance and support. Online learning or adaptive learning systems can use expression recognition technology to adjust the difficulty and presentation of content to match students' emotions and cognitive needs. This technical solution makes the education process more personalized and interactive by using deep learning and computer vision technology, improving educational effects and students' learning experience. At the same time, it also provides valuable references for the application of technology in emotion recognition and human-computer interaction.

[0083] One embodiment of the present invention defines student expressions during class, including:

[0084] Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library;

[0085] When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection;

[0086] When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt;

[0087] Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening.

[0088] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding.

[0089] Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion.

[0090] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

[0091] The working principle and effect of the above technical solution are as follows: using the 68-point facial key point detection technology based on the Dlib library, this technology is based on a machine learning model and can accurately find 68 key position points of the human face (such as eyebrows, eyes, nose, mouth and other areas) in a given image. These points provide accurate basic data for subsequent expression analysis; combining the mouth key points, by calculating the angles of the mouth (such as the angle between point 48, point 54, point 66 and the horizontal line), the inner emotional state of the student is judged. The positive and negative angles intuitively reflect the upward or downward trend of the mouth, which is used to define different types of expressions (such as rejection, contempt, confusion, happiness, etc.); further analyzing the emotional state of the student through the aspect ratio of the eyes and the angle between the eyebrows. The aspect ratio of the eyes can reveal the degree of opening and closing of the eyes, while the angle of the eyebrows can reflect the attention and emotional state of the person. Combining the data of the eyes and eyebrows, expressions such as listening and understanding can be defined in more detail; according to the extracted facial features, such as the mouth angle, the aspect ratio of the eyes, the eyebrow angle, etc., the current expression category is judged according to the pre-defined rules. Improve classroom management efficiency. By automatically identifying students' expressions, teachers can understand students' emotions and attitudes in real time, such as whether they are happy, confused or uninterested. This helps teachers adjust teaching methods or content in a timely manner, thereby improving teaching effectiveness. Enhance student interaction. The system can help teachers better understand students' non-verbal feedback, such as expressions of understanding and listening, so that teachers can encourage students to participate more actively in classroom discussions. For distance education or online classes, this technology can help teachers monitor students' participation and reactions from a distance to ensure the interactivity and effectiveness of distance teaching. Real-time expression recognition provides instant feedback on students' emotional state, which can be used to support the research and analysis of emotional cognition and help improve students' emotional intelligence. This technical solution uses advanced face recognition technology to analyze students' emotions and reactions, which can not only promote communication and interaction between teachers and students, but also help optimize teaching strategies and improve education quality through accurate feedback. In addition, the application of this technology can also be extended to other fields that require emotion and reaction analysis, such as customer service and psychological counseling.

[0092] An embodiment of the present invention is a method for evaluating student attention based on big data analysis, which collects student posture images from images of students in class, and determines whether the students raise their hands to answer questions based on the student posture images, including:

[0093] Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates;

[0094] The angle model is used to determine whether the student raises his hand. Specifically, the angle model is:

[0095]

[0096] When 45°≤α≤135°, the student is judged to be in the hand-raising state.

[0097] The working principle and effect of the above technical solution are as follows: posture image acquisition, using a camera or other image acquisition equipment to collect students' posture images in the classroom; using OpenPose, a deep learning posture estimation algorithm, to process the acquired images. OpenPose can identify and mark the key points of the human body, including the coordinates of the elbow and wrist; using the coordinates of the elbow and wrist to calculate the angle (α) between the two. This is done through the cosine theorem in geometry: the first step is to calculate the difference between the wrist and elbow in the horizontal direction (x-axis) and the vertical direction (y-axis); the second step is to use these differences to calculate the Euclidean distance between the two points; the third step is to use the cosine theorem to calculate the angle (α) formed by the line from the elbow to the wrist and the horizontal line of the arm part, and the angle model determines the hand-raising state: according to the calculated angle α, it is determined whether it is between 45° and 135°. If so, the student is considered to be in the hand-raising state. Improve teachers' response efficiency. Automatically identifying students who raise their hands can help teachers quickly respond to students' questions or requests for participation, ensuring that every student's questions can be answered in a timely manner and enhancing classroom interaction. Optimize classroom management. Automatically detecting the status of raised hands reduces the need for teachers to distract their attention to observe which students need help, allowing teachers to focus more on explaining and interacting with teaching content. Data-driven teaching participation analysis. Data collected on hand-raising behavior can be used to analyze student participation and provide feedback to teachers to help them understand which students are more active and which need more incentives or support. Improve students' learning enthusiasm. Knowing that all raised hands will be fairly identified and responded to may encourage more students to participate in class discussions, thereby improving their learning motivation and classroom participation. The technical solution not only improves the efficiency and effectiveness of teaching through automated hand-raising recognition, but also promotes more equitable and active student participation. The solution uses advanced machine vision technology to accurately and quickly analyze and respond to student behavior, thereby optimizing the educational process and results.

[0098] In one embodiment of the present invention, an attention score is calculated from changes in the student's facial expression type and body features, and whether the student is concentrating is determined based on the attention score, including:

[0099] The attention score is calculated from the changes in the student's facial expression type and body features through the attention model. Specifically, the attention model is:

[0100]

[0101] Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered;

[0102] When A(t)≤0.3, the student’s attention is judged to be in a low state;

[0103] When 0.3<A(t)≤0.7, the student’s attention is judged to be in a general state;

[0104] When 0.7<A(t), it is judged that the student's attention is focused.

[0105] The working principle and effect of the above technical solution are as follows: the above technical solution evaluates the attention level of students at a specific time point by combining the changes in their facial expressions and body movements. The specific calculation process is as follows: facial expression and body posture data are integrated, and computer vision and posture estimation technology are used to extract relevant data of facial expression (Fi) and body posture (Pi) from video or image sequences. Weights (α_i and β_i) are applied to facial and body data in different time windows. These weights indicate the contribution of each feature to attention in different situations. The facial and body data are integrated using a mathematical model to calculate the attention score (A(t)). The model consists of two parts. The first part processes facial expression data by applying a sigmoid function to express the smooth contribution to attention; the second part processes body movement data and expresses the effect of these features decaying over time through an exponential decay function; the two parts are fused through weights (ω_1 and ω_2) to balance the influence of facial and body features on the attention score; according to the calculated attention score, the student's attention level is classified as low, general or concentrated, and this classification is determined by a preset threshold. Personalized teaching support: teachers can adjust teaching methods and rhythms according to each student's attention score, and provide more guidance and motivation for students with low attention. Improve learning outcomes: by real-time monitoring of students' attention status, teachers can adjust classroom interactions immediately to ensure that students can learn in the best cognitive state, thereby improving teaching quality and students' learning outcomes. Data-driven teaching improvements: long-term collection and analysis of students' attention data can help educational researchers and teachers understand the key factors that affect students' attention and further optimize teaching content and environment design. Enhance students' self-management ability: students can understand their attention changes through feedback, learn how to self-regulate to maintain a good learning state, and cultivate good learning habits and self-management skills. Focus on image sequences, which are instantaneous features. When a person is in a certain state, his facial and body movements are a continuous process of change. The picture at a certain moment may not be able to significantly represent the state. Here, several recorded sequences are used for evaluation to obtain more realistic results.

[0106] In order to better understand the specific implementation of the attention score function, a virtual example can be used to demonstrate how to calculate the attention score for a single student within a time window. Assume that the model has extracted key facial features and body posture data from the video and has qualitatively analyzed the importance coefficients of these features. The specific numerical settings and calculation process are given below.

[0107] The assumed parameter setting time window length N = 5, that is, considering the data of 5 consecutive frames in the video, weights ω1 = 0.6 and ω2 = 0.4, respectively reflecting the importance of facial features and body movements in the attention score, facial feature importance coefficient αi and body posture importance coefficient βi. For simplification, it is assumed that the importance coefficients of each feature are facial features: [0.8, 0.7, 0.9, 0.6, 0.85], body features: [0.9, 0.75, 0.65, 0.8, 0.7], facial feature values ​​Fi [0.2, 0.3, 0.5, 0.1, 0.4], body posture values ​​[0.5, 0.6, 0.4, 0.3, 0.7], and the weighted sum of facial features is calculated: Compute the weighted sum of limb poses:

[0108]

[0109] Apply the attention score formula:

[0110] A(t)=ω1·sigmoid(1.22)+ω2·(1-exp(-1.89))=0.8028, where we assume that sigmoid(1.22)≈0.772 and exp(-1.89)≈0.151. Therefore, based on the above parameters and data, the student’s attention score in this time window is 0.803, which means that the student’s attention level is relatively high in this short period of time.

[0111] Here, when a student is judged to have raised his hand, Pi is calculated as:

[0112] When it is determined that the student did not raise his hand, the value of Pi is 0;

[0113] The value of Fi can be given a specific numerical value according to the student's expression. When the student's expression is judged to be rejection, Fi=0.1; when the student's expression is judged to be contemptuous, Fi=0.3; when the student's expression is judged to be listening, Fi=0.5; when the student's expression is judged to be confused, Fi=0.7; when the student's expression is judged to be happy, Fi=0.9; when the student's expression is judged to be understanding, Fi=1.

[0114] One embodiment of the present invention is a student attention assessment system based on big data analysis, the system comprising:

[0115] The module for identifying student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and identify students' expression types based on their facial expressions;

[0116] A hand-raising module is used to collect student posture images from students in class and judge whether students raise their hands to answer questions based on the student posture images;

[0117] The attention score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics, and determine whether the student is focused based on the attention score.

[0118] In one embodiment of the present invention, the module for identifying student expression types includes:

[0119] Define the expression module, which is used to define students' expressions in class;

[0120] The annotation module is used to annotate the expression dataset according to the definition of students' expressions in class;

[0121] The data set partitioning module is used to partition the annotated expression data set into a training set, a test set, and a validation set;

[0122] The training model module is used to build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model;

[0123] The module for obtaining student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and input students' facial expressions into the expression recognition model to obtain students' expression types.

[0124] In one embodiment of the present invention, the expression definition module includes:

[0125] Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library;

[0126] When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection;

[0127] When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt;

[0128] Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening.

[0129] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding.

[0130] Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion.

[0131] Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

[0132] In one embodiment of the present invention, the module for determining whether to raise the hand includes:

[0133] Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates;

[0134] Angle judgment module, used to judge whether the student raises his hand through the angle model. Specifically, the angle model is:

[0135]

[0136] When 45°≤α≤135°, the student is judged to be in the hand-raising state.

[0137] In one embodiment of the present invention, calculating the attention score includes:

[0138] The score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics through the attention model. Specifically, the attention model is:

[0139]

[0140] Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered;

[0141] The low state determination module is used to determine that the student's attention is in a low state when A(t)≤0.3;

[0142] The module for judging the general state is used to judge that the student's attention is in the general state when 0.3<A(t)≤0.7;

[0143] The attention concentration determination module is used to determine that the student's attention is concentrated when 0.7<A(t).

[0144] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A student attention assessment method based on big data analysis, characterized in that: The method comprises: Capturing images of students in class through a camera, collecting students' facial expressions from the images of students in class, and identifying students' expression types based on their facial expressions; Collect student posture images from students’ class images, and judge whether students raise their hands to answer questions based on the student posture images; The attention score is calculated from the changes in the students' facial expression types and body characteristics, and the students' concentration is determined based on the attention score.

2. According to claim 1, a student attention assessment method based on big data analysis is characterized in that: Capture images of students in class through a camera, collect students' facial expressions from the images, and identify students' expression types based on their facial expressions, including: Define students' expressions in class; Label the facial expression dataset according to the definition of students’ facial expressions in class; Divide the annotated expression dataset into training set, test set and validation set; Build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model; The image of the student in class is captured by a camera, the facial expression of the student is collected from the image of the student in class, and the facial expression of the student is input into the expression recognition model to obtain the expression type of the student.

3. According to claim 2, a method for evaluating student attention based on big data analysis is characterized in that: Define student expressions in class, including: Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library; When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection; When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt; Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening. Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding. Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion. Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

4. According to claim 1, a method for evaluating student attention based on big data analysis is characterized in that: Collect student posture images from student class images, and determine whether the student raises his hand to answer questions based on the student posture images, including: Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates; The angle model is used to determine whether the student raises his hand. Specifically, the angle model is: When 45°≤α≤135°, the student is judged to be in the hand-raising state.

5. According to claim 1, a method for evaluating student attention based on big data analysis is characterized in that: The attention score is calculated from the changes in the student's facial expression type and body characteristics, and the student's concentration is determined based on the attention score, including: The attention score is calculated from the changes in the student's facial expression type and body characteristics through the attention model. Specifically, the attention model is: Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered; When A(t)≤0.3, the student’s attention is judged to be in a low state; When 0.3<A(t)≤0.7, the student's attention is judged to be in a general state; When 0.7<A(t), it is judged that the student's attention is focused.

6. A student attention assessment system based on big data analysis, characterized in that: The system comprises: The module for identifying student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and identify students' expression types based on their facial expressions; A hand-raising module is used to collect student posture images from students in class and judge whether students raise their hands to answer questions based on the student posture images; The attention score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics, and determine whether the student is focused based on the attention score.

7. A student attention assessment system based on big data analysis according to claim 6, characterized in that: The module for identifying student expression types includes: Define the expression module, which is used to define students' expressions in class; The annotation module is used to annotate the expression dataset according to the definition of students' expressions in class; The data set partitioning module is used to partition the annotated expression data set into a training set, a test set, and a validation set; The training model module is used to build a convolutional neural network model, use the training set to train the convolutional neural network model, and use the test set and validation set to evaluate and optimize the trained convolutional neural network model to obtain an expression recognition model; The module for obtaining student expression types is used to capture images of students in class through a camera, collect students' facial expressions from the images of students in class, and input the students' facial expressions into the expression recognition model to obtain students' expression types.

8. The student attention assessment system based on big data analysis according to claim 6 is characterized in that: The expression definition module includes: Extract and return the coordinates of the key points of the student's face through the 68-point detection technology in the Dlib library; When the angle between the line connecting the coordinates of the 48th, 54th and 66th points of the mouth and the horizontal line is negative, the expression is defined as rejection; When the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line is a positive number, the expression is defined as contempt; Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, and subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25 and the angle between the two lines of the left and right eyebrows is greater than 120°, the expression is defined as listening. Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value. Subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value. The same is true for the right eye. Connect the left eyebrow points 17 and 21 with 19 to obtain two lines. The same is true for the right eyebrow. When the angle is a positive number, the ratio of the left and right eye height values ​​to the eye width values ​​is greater than 0.25, and the angle between the two lines connecting the left and right eyebrows is greater than 120°, define the expression as understanding. Obtain the angle between the lines connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line. Connect the left eyebrow points 17 and 21 with the line 19 to obtain two lines. If the angle is negative and the angle between the two lines connecting the left and right eyebrows is less than 120°, the expression is defined as confusion. Obtain the angle between the line connecting the coordinates of points 48, 54 and 66 of the mouth and the horizontal line, the height of the line connecting points 66 and 62 of the mouth, subtract the ordinate at point 37 of the left eye from the ordinate at point 41 to obtain the eye height value, subtract the abscissa at point 36 from the abscissa at point 39 to obtain the eye width value, and obtain two lines by connecting left eyebrow points 17 and 21 with 19 respectively. When the angle of the mouth is positive, the height of the line connecting the two points is greater than 0.03, the ratio of the eye height to eye width of the left and right eyes is greater than 0.25, and the angle between the left and right eyebrows is greater than 120 degrees, the expression is defined as happy.

9. The student attention assessment system based on big data analysis according to claim 6, characterized in that: The module for judging whether to raise hands includes: Collect student posture images, analyze and process the images using OpenPose, a posture estimation algorithm in deep learning, to identify and mark the student's elbow and wrist coordinates; The angle judgment module is used to judge whether the student raises his hand through the angle model. Specifically, the angle model is: When 45°≤α≤135°, the student is judged to be in the hand-raising state.

10. The student attention assessment system based on big data analysis according to claim 6, characterized in that: The calculation of the attention score includes: The score calculation module is used to calculate the attention score from the changes in the student's facial expression type and body characteristics through the attention model. Specifically, the attention model is: Among them, A(t) represents the attention score at time t, ω1 and ω2 are the weights for adjusting the influence of facial features and body movements on the final attention score, and α i and β i It is a facial feature F i and body posture P i The importance coefficient of , N is the length of the time window considered; The low state determination module is used to determine that the student's attention is in a low state when A(t)≤0.3; The module for judging the general state is used to judge that the student's attention is in the general state when 0.3<A(t)≤0.7; The attention concentration determination module is used to determine that the student's attention is concentrated when 0.7<A(t).

Images