Method and system for analyzing learning state based on student emotion
By performing feature analysis on student video images and combining course content with emotional changes, an emotion score is calculated, which solves the problem of inaccurate assessment of students' learning attitudes in existing technologies and achieves a more accurate assessment of learning status.
Patent Information
- Application Number
- CN202511242265.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-02
- Publication Date
- 2025-12-16
AI Technical Summary
Existing AI-based methods struggle to accurately assess students' learning attitudes, neglecting dynamic changes in emotions and their contextual relationship with course content, leading to biased and inaccurate assessment results.
By acquiring training video images, processing them into frame images and assembling them into an image set, performing feature analysis, and combining course content, emotional change time points, emotional type and duration, an emotional score is calculated to comprehensively judge the student's learning status.
It enables accurate assessment of students' learning status, and by combining emotional changes, course content, and emotional type, it provides a more objective and granular assessment of learning attitude, reducing the influence of subjective judgment.
Smart Images
Figure CN121147992A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence education technology, specifically to a method and system for analyzing students' learning status based on their emotions. Background Technology
[0002] When students are learning at school, it is difficult for teachers to pay attention to each student at the same time during class, and it is impossible to accurately grasp the students' learning attitude during class. Even when evaluating students, it is only based on the teacher's subjective observation or after-class test scores, which lacks real-time, objective and granular data support.
[0003] Most existing AI-based methods only statically classify students' expressions (such as happy, neutral, and sad), ignoring the dynamic changes and duration of emotions, as well as their contextual relationship with specific course content (such as lectures, Q&A sessions, and self-study), resulting in biased and inaccurate assessment results. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a method and system for analyzing students' learning status based on their emotions. This method does not simply judge the learning status based on students' emotions, but rather makes a comprehensive judgment by combining course content, emotional changes, and emotional types.
[0005] A method for analyzing students' learning status based on their emotions includes:
[0006] Acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order;
[0007] The image set is processed to obtain a processed image set;
[0008] Feature analysis is performed on the processed image set to obtain the students' emotions at different time points in the image set, forming a student emotion matrix;
[0009] The student emotion matrix was analyzed to obtain the time points of emotion changes, the duration of each emotion, and the emotion type.
[0010] Retrieve the current course content, which includes new lessons, reviews, self-study materials, and Q&A.
[0011] An emotion score is obtained based on the course content, the timing of the emotion change, the type of emotion, and the duration of the emotion.
[0012] Based on the emotion score, the student's current learning status in the course can be obtained.
[0013] Optionally, the step of performing feature analysis on the processed image set to obtain student emotions at different time points in the processed image set, forming a student emotion matrix, includes:
[0014] The processed image set is input into a pre-trained deep learning model in chronological order to obtain the emotion probability vector corresponding to each frame image.
[0015] The emotion corresponding to the largest emotion probability vector is selected as the student emotion, and the student emotions corresponding to all frames are sorted by time to form a student emotion matrix.
[0016] Optionally, obtaining the emotion score based on the course content, the time point of the emotion change, the emotion type, and the duration of the emotion includes:
[0017] The emotion types include positive and negative types;
[0018] Obtain an emotion-course association weight matrix based on course content and emotion type;
[0019] Detect the timing of emotional changes and the duration of different emotions;
[0020] An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions.
[0021] Optionally, the calculation of the emotion score based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions includes:
[0022] Based on the emotion type, obtain the ideal duration of the emotion.
[0023] A duration score is obtained based on the ideal duration of the emotion, the duration of the emotion, and the duration function.
[0024] The duration function is expressed as:
[0025]
[0026] Where t is the duration, t type For the ideal duration of the emotion, k is a parameter that adjusts the steepness of the function;
[0027] An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the score of the time point and duration of emotion change.
[0028] Optionally, the step of calculating the emotion score based on the emotion type, the emotion course association weight matrix, and the scores for the time points and duration of emotion changes includes:
[0029] Set basic value scores for different emotions;
[0030] The base value score is corrected based on the emotion association weight matrix to obtain the corrected score;
[0031] An emotion score is calculated based on the corrected score, duration score, and time point of emotion change.
[0032] Optionally, the calculation of the emotion score based on the corrected score, duration score, and time point of emotion change includes:
[0033] Obtain the emotions before and after the point in time of the emotional change;
[0034] Determine whether there is a correlation between the emotions before and after the stated emotional change point;
[0035] If the emotions before and after the emotional change are related, then the first weight is obtained;
[0036] An emotion score is calculated based on the corrected score, the duration score, and the first weight.
[0037] Optional, also includes:
[0038] If the emotions before and after the emotional change time are not related, then obtain the duration of the emotional change after the emotional change time point.
[0039] If the duration of the emotional change exceeds a preset threshold, a second weight is obtained;
[0040] An emotion score is calculated based on the corrected score, the duration score, and the second weight.
[0041] A system for analyzing students' learning states based on their emotions includes:
[0042] The first acquisition module is used to acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order.
[0043] The processing module is used to process the image set to obtain a processed image set;
[0044] The first analysis module is used to perform feature analysis on the processed image set to obtain the student emotions corresponding to different time points in the image set and form a student emotion matrix.
[0045] The second analysis module is used to analyze the student emotion matrix to obtain the time points of emotion changes, the duration of each emotion, and the emotion type.
[0046] The second acquisition module is used to acquire the current course content, which includes new courses, review sessions, self-study sessions, and Q&A.
[0047] The calculation module is used to obtain an emotion score based on the course content, the time point of the emotion change, the emotion type, and the duration of the emotion.
[0048] The evaluation module is used to determine the student's current learning status based on the emotion score.
[0049] A terminal device includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it employs a method based on student emotion analysis of learning status.
[0050] A computer-readable storage medium storing a computer program that, when loaded and executed by a processor, employs a method based on student emotion analysis of learning status.
[0051] The beneficial effects of this invention are:
[0052] The process involves acquiring training video images, processing them into frame images, and arranging several frame images into an image set in chronological order. This image set is then processed to obtain a processed image set. Feature analysis is performed on the processed image set to obtain student emotions at different time points, forming a student emotion matrix. The student emotion matrix is analyzed to obtain the time points of emotion changes, the duration of each emotion, and the emotion type. The current course content, including new lessons, review, self-study, and Q&A, is then acquired. Based on the course content, the time points of emotion changes, the emotion type, and the duration of the emotion, an emotion score is obtained. Based on the emotion score, the student's current learning status in the course is determined. Compared to subjective judgment by teachers or static classification, this application combines student emotion changes, course content, emotion type, and duration for a comprehensive assessment, resulting in a more accurate emotion score and a more precise evaluation of the student's learning status, thus providing a more accurate assessment of the student's learning attitude. Attached Figure Description
[0053] Figure 1 This is a flowchart illustrating a method for analyzing students' learning states based on their emotions, as proposed in this invention. Detailed Implementation
[0054] A method for analyzing students' learning status based on their emotions, such as Figure 1 As shown, the present invention includes:
[0055] S1. Acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order.
[0056] Specifically, the training video images are videos taken of students during class. The videos are divided into several frames, resulting in frame images, which are then assembled into an image set in chronological order. The video resolution is generally at least 720P, and the frame rate is no less than 25fps to ensure sufficient image quality for micro-expression analysis.
[0057] S2. Process the image set to obtain a processed image set;
[0058] Specifically, not all frames per second need to be processed. An adaptive frame extraction algorithm is used to forcibly extract a frame when the image difference between consecutive frames exceeds a threshold (e.g., pixel change exceeds 5%). This ensures that keyframes are captured when expressions change, reduces computation when images are still, clears some blurry images, and crops all images to the same standard size.
[0059] S3. Perform feature analysis on the processed image set to obtain the student emotions at different time points in the image set, and form a student emotion matrix.
[0060] Specifically, a deep learning model (such as VGG, ResNet, or EfficientNet) pre-trained on a large dataset (such as AffectNet or FER+) is used as the backbone, followed by a classification layer. Training can involve extracting facial features representing different emotions, inputting them into the deep learning model, obtaining output labels, comparing the output labels with the correct labels, calculating the loss, and minimizing the loss to obtain the optimal deep learning model.
[0061] Students' emotions include: anger, disgust, fear, happiness, sadness, surprise, and neutrality.
[0062] For each frame of an image, the model outputs a sentiment probability vector, and the sentiment with the highest probability is taken as the predicted sentiment for that frame.
[0063] S4. Analyze the student emotion matrix to obtain the time points of emotion changes, the duration of each emotion, and the emotion type;
[0064] The process involves performing feature analysis on the processed image set to obtain student emotions at different time points within the processed image set, forming a student emotion matrix, which includes:
[0065] The processed image set is input into a pre-trained deep learning model in chronological order to obtain the emotion probability vector corresponding to each frame image.
[0066] The emotion corresponding to the largest emotion probability vector is selected as the student emotion, and the student emotions corresponding to all frames are sorted by time to form a student emotion matrix.
[0067] S5. Obtain the current course content, which includes new courses, review sessions, self-study sessions, and Q&A.
[0068] Specifically, students will experience different emotions when learning different course content. Negative emotions are not necessarily bad. For example, when learning a new course, a student may be surprised because they are attracted by the new knowledge, or sad because they did not understand it. However, this does not mean that the student did not study hard.
[0069] S6. Based on the course content, the time points of emotional changes, the type of emotion, and the duration of the emotion, obtain an emotion score;
[0070] The emotional score, calculated based on the course content, the timing of the emotional change, the type of emotion, and the duration of the emotion, includes:
[0071] The emotion types include positive and negative types;
[0072] Specifically, positive types include happiness and surprise, while negative types include anger, disgust, and fear.
[0073] Obtain an emotion-course association weight matrix based on course content and emotion type;
[0074] Specifically, the emotion course association weight matrix is represented as follows:
[0075] Emotion Types Emotional weight positive emotions 1 negative emotions 0.5
[0076] Detect the timing of emotional changes and the duration of different emotions;
[0077] Specifically, the emotional change point is the time when a user changes from one emotion to another, and the duration of different emotions is the duration of a user's current emotion until the change to another emotion.
[0078] For example, happiness: V = +2.
[0079] Surprise: V = +1 (Surprise can be positive or negative, but it is usually associated with concentration, so we will tentatively define it as weakly positive).
[0080] Neutral: V = 0 (indicates focus or distraction, which needs to be judged in context).
[0081] Sadness: V = -1.
[0082] Anger, Disgust, Fear: V = -2.
[0083] An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions.
[0084] The emotion score, calculated based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions, includes:
[0085] Based on the emotion type, obtain the ideal duration of the emotion.
[0086] A duration score is obtained based on the ideal duration of the emotion, the duration of the emotion, and the duration function.
[0087] The duration function is expressed as:
[0088]
[0089] Where t is the duration, t type For the ideal duration of the emotion, k is a parameter that adjusts the steepness of the function;
[0090] An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the score of the time point and duration of emotion change.
[0091] The emotion score calculated based on the emotion type, the emotion course association weight matrix, and the scores for the time points and duration of emotion changes includes:
[0092] Set basic value scores for different emotions;
[0093] Specifically, the baseline value score refers to the baseline score of a certain emotion. Generally speaking, the baseline value score of positive emotions is higher than that of negative emotions.
[0094] The base value score is corrected based on the emotion association weight matrix to obtain the corrected score;
[0095] Specifically, the purpose of adjusting the base score is to avoid directly defining negative emotions as bad. Therefore, when negative emotions exist, the base score needs to be adjusted accordingly based on the course content.
[0096] For example, anger or aversion may arise under the new curriculum because students haven't understood the material, or fear may occur during question-and-answer sessions because they don't know how to respond. Therefore, these reactions need to be addressed.
[0097] In this example, the complete emotion-course association weight matrix (after correction) is as follows:
[0098]
[0099]
[0100] An emotion score is calculated based on the corrected score, duration score, and time point of emotion change.
[0101] The calculation of the emotion score based on the corrected score, duration score, and time point of emotion change includes:
[0102] Obtain the emotions before and after the point in time of the emotional change;
[0103] Specifically, if a student experiences emotional shifts during the learning process, it is normal if these shifts are related to the course content. However, if they are caused by other factors, it indicates a decline in attention span.
[0104] Determine whether there is a correlation between the emotions before and after the stated emotional change point;
[0105] Specifically, the correlation between emotional changes and course content can be established in advance. For example, if a student's emotion changes from surprise to disgust, and the timing of this change coincides with a shift in course content from new material to review, this is a correlation. Conversely, if a student's emotion changes from surprise to neutral, but the course content remains unchanged, it indicates that the student may have been daydreaming, which is a change unrelated to the course content.
[0106] If the emotions before and after the emotional change are related, then the first weight is obtained;
[0107] An emotion score is calculated based on the corrected score, the duration score, and the first weight.
[0108] Specifically, the emotion score = adjusted score * duration score * weighted score (first weight or second weight).
[0109] Specifically, both the first and second weights are designed to correct for changes that occur during the student's learning process, resulting in a more accurate emotion score.
[0110] If the emotions before and after the emotional change time are not related, then obtain the duration of the emotional change after the emotional change time point.
[0111] If the duration of the emotional change exceeds a preset threshold, a second weight is obtained;
[0112] An emotion score is calculated based on the corrected score, the duration score, and the second weight.
[0113] Specifically, if a student consistently maintains certain negative or positive emotions without any relevant connection, it needs to be corrected.
[0114] The first and second weights can be set to fixed values depending on the situation, or they can be set to different values based on the corresponding course content and the different emotions involved in the emotional transformation.
[0115] S7. Based on the emotion score, obtain the student's current learning status in the course.
[0116] Specifically, learning status includes excellent, average, and poor, with different learning statuses corresponding to different score ranges.
[0117] Compared to subjective judgments or static classifications by teachers, this method can comprehensively assess students' emotional changes, course content, emotional type, and duration to obtain more accurate emotional scores and evaluate their classroom performance. This allows for a more accurate assessment of students' learning attitude and can be applied in several ways: objectively assigning students a grade for their participation in class, avoiding excessive subjectivity in teacher scoring; or combining their learning status with their actual learning outcomes to provide appropriate guidance. For example, if a student's learning status is good but their exam results are unsatisfactory, it may be due to poor learning methods or approaches, requiring timely guidance and communication.
[0118] A system for analyzing students' learning states based on their emotions includes:
[0119] The first acquisition module is used to acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order.
[0120] The processing module is used to process the image set to obtain a processed image set;
[0121] The first analysis module is used to perform feature analysis on the processed image set to obtain the student emotions corresponding to different time points in the image set and form a student emotion matrix.
[0122] The second analysis module is used to analyze the student emotion matrix to obtain the time points of emotion changes, the duration of each emotion, and the emotion type.
[0123] The second acquisition module is used to acquire the current course content, which includes new courses, review sessions, self-study sessions, and Q&A.
[0124] The calculation module is used to obtain an emotion score based on the course content, the time point of the emotion change, the emotion type, and the duration of the emotion.
[0125] The evaluation module is used to determine the student's current learning status based on the emotion score.
[0126] This application also discloses a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it employs a method based on student emotion analysis of learning status.
[0127] The terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server. The terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.
[0128] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it in this regard.
[0129] The memory can be an internal storage unit of the terminal device, such as a hard disk or RAM of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the terminal device. Furthermore, the memory can be a combination of internal storage units and external storage devices of the terminal device. The memory is used to store computer programs and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.
[0130] In this terminal device, a method for analyzing student emotions and learning status based on the above embodiments is stored in the terminal device's memory and loaded and executed on the terminal device's processor for convenient use.
[0131] This application also discloses a computer-readable storage medium, which stores a computer program, wherein when the computer program is executed by a processor, it employs a method based on student emotion analysis of learning status as described in the above embodiments.
[0132] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.
[0133] The method for analyzing student emotions based on learning status in the above embodiments is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above method.
[0134] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.
[0135] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.
Claims
1. A method for analyzing students' learning status based on their emotions, characterized in that, include: Acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order; The image set is processed to obtain a processed image set; Feature analysis is performed on the processed image set to obtain the students' emotions at different time points in the image set, forming a student emotion matrix; The student emotion matrix was analyzed to obtain the time points of emotion changes, the duration of each emotion, and the emotion type. Retrieve the current course content, which includes new lessons, reviews, self-study materials, and Q&A. An emotion score is obtained based on the course content, the timing of the emotion change, the type of emotion, and the duration of the emotion. Based on the emotion score, the student's current learning status in the course can be obtained.
2. The method as described in claim 1, characterized in that, The process involves performing feature analysis on the processed image set to obtain student emotions at different time points within the processed image set, forming a student emotion matrix, which includes: The processed image set is input into a pre-trained deep learning model in chronological order to obtain the emotion probability vector corresponding to each frame image. The emotion corresponding to the largest emotion probability vector is selected as the student emotion, and the student emotions corresponding to all frames are sorted by time to form a student emotion matrix.
3. The method as described in claim 1, characterized in that, The emotional score, calculated based on the course content, the timing of the emotional change, the type of emotion, and the duration of the emotion, includes: The emotion types include positive and negative types; Obtain an emotion-course association weight matrix based on course content and emotion type; Detect the timing of emotional changes and the duration of different emotions; An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions.
4. The method as described in claim 3, characterized in that, The emotion score, calculated based on the emotion type, the emotion course association weight matrix, the time point of emotion change, and the duration of different emotions, includes: Based on the emotion type, obtain the ideal duration of the emotion. A duration score is obtained based on the ideal duration of the emotion, the duration of the emotion, and the duration function. The duration function is expressed as: Where t is the duration, t type For the ideal duration of the emotion, k is a parameter that adjusts the steepness of the function; An emotion score is calculated based on the emotion type, the emotion course association weight matrix, the score of the time point and duration of emotion change.
5. The method as described in claim 4, characterized in that, The emotion score calculated based on the emotion type, the emotion course association weight matrix, and the scores for the time points and duration of emotion changes includes: Set basic value scores for different emotions; The base value score is corrected based on the emotion association weight matrix to obtain the corrected score; An emotion score is calculated based on the corrected score, duration score, and time point of emotion change.
6. The method as described in claim 1, characterized in that, The calculation of the emotion score based on the corrected score, duration score, and time point of emotion change includes: Obtain the emotions before and after the point in time of the emotional change; Determine whether there is a correlation between the emotions before and after the stated emotional change point; If the emotions before and after the emotional change are related, then the first weight is obtained; An emotion score is calculated based on the corrected score, the duration score, and the first weight.
7. The method of claim 6, characterized in that, it further... include: If the emotions before and after the emotional change time are not related, then obtain the duration of the emotional change after the emotional change time point. If the duration of the emotional change exceeds a preset threshold, a second weight is obtained; An emotion score is calculated based on the corrected score, the duration score, and the second weight.
8. A system for analyzing students' learning states based on their emotions, characterized in that: include: The first acquisition module is used to acquire training video images, process the video images into frame images, and assemble a set of several frame images in chronological order. The processing module is used to process the image set to obtain a processed image set; The first analysis module is used to perform feature analysis on the processed image set to obtain the student emotions corresponding to different time points in the image set and form a student emotion matrix. The second analysis module is used to analyze the student emotion matrix to obtain the time points of emotion changes, the duration of each emotion, and the emotion type. The second acquisition module is used to acquire the current course content, which includes new courses, review sessions, self-study sessions, and Q&A. The calculation module is used to obtain an emotion score based on the course content, the time point of the emotion change, the emotion type, and the duration of the emotion. The evaluation module is used to determine the student's current learning status based on the emotion score.
9. A terminal device, comprising a memory and a processor, characterized in that, The memory stores a computer program that can run on a processor, and when the processor loads and executes the computer program, it employs the method described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the method described in any one of claims 1 to 7.