Student online learning evaluation method and system based on image psychological sentiment analysis

Abstract

The invention belongs to the technical field of online education, and provides a student online learning evaluation method and system based on image psychological sentiment analysis, and the method comprises the steps: collecting a face image and a limb movement image of a student in real time during online learning through a camera; preprocessing the acquired image by using an image recognition technology, and extracting facial feature points, expression features and limb action features; according to the method, the face image and the limb action image of the student during online learning are collected at the same time, multiple features are extracted for psychological emotion analysis, and the learning effect is evaluated in combination with learning behavior data, so that a single evaluation mode which purely depends on scores in the past is changed, the learning state of the student is comprehensively known from multiple dimensions, and the learning efficiency is improved. After the system identifies the emotional state of the student, detailed feedback information can be sent to the teacher or the assistant in real time, so that the teacher can master the psychological emotional dynamic state of each student in real time, and the requirements of the student can be better met.

Description

Technical Field

[0001] The present invention belongs to the technical field of online education, and specifically relates to a method and system for evaluating students' online learning based on image psychological emotion analysis. Background Art

[0002] With the rapid development of Internet technology, online learning has become an indispensable and important part of the current education field. Compared with traditional offline learning, online learning has the advantages of high flexibility and rich resources, and can meet the diverse learning needs of different students. However, there are still some problems in the current online learning in evaluating students' learning status.

[0003] However, the existing online learning systems lack a real-time perception and feedback mechanism for students' learning status. Teachers cannot timely understand the real-time psychological and emotional changes of students during the learning process, so it is difficult to adjust teaching strategies targeted, and cannot meet the personalized needs of students. At the same time, when students encounter emotional problems during the learning process, they cannot get effective guidance and interaction in time, resulting in a poor learning experience and a frustrated learning enthusiasm. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for evaluating students' online learning based on image psychological emotion analysis to solve the problem that the real-time psychological and emotional changes of students during the learning process cannot be timely understood in the prior art.

[0005] A method for evaluating students' online learning based on image psychological emotion analysis includes:

[0006] S1. Real-time collect facial images and limb movement images of students during online learning through a camera;

[0007] S2. Use image recognition technology to preprocess the collected images, and extract facial feature points, expression features, and limb movement features;

[0008] S3. Input the extracted feature data into a pre-trained psychological emotion analysis model based on a deep learning algorithm to predict the psychological emotion state of students;

[0009] S4. After identifying the emotion state of students, send feedback information including a specific emotion state description of students to the terminal device of teachers or teaching assistants through an instant messaging interface, and automatically pop up interactive content corresponding to the emotion state on the students' learning interface;

[0010] S5. Comprehensively evaluate the online learning effect of students according to the psychological emotion state and learning behavior data of students, and use an intelligent recommendation algorithm to screen and push personalized learning resources or suggestions to students from a learning resource library.

[0011] Preferably, the facial feature points are extracted using an algorithm based on an active shape model (ASM) by iteratively optimizing an objective function

[0012]

[0013] Determine the location of facial landmarks, x i represents the current estimated feature point position, represents the position of the corresponding feature point in the average shape model, w x is the weight coefficient, and n is the number of feature points.

[0014] Preferably, the evaluation formula for comprehensively evaluating the online learning effect of students is:

[0015] S=α×P+β×A+γ×Q

[0016] Among them, S is the comprehensive evaluation score, P is the psychological and emotional state score, A is the learning behavior activity score, Q is the answer accuracy score, α, β, γ are weight coefficients, and α+β+γ=1.

[0017] Preferably, the students' emotional state data is continuously stored and analyzed, and emotional trend reports are generated by daily, weekly, and monthly time dimensions. When students' emotional states show persistent lows and abnormal fluctuations, early warnings containing details of the emotional state and duration are sent to teachers and parents via text messages or in-app notifications.

[0018] Preferably, the psychological emotion analysis model is based on a convolutional neural network (CNN) and adopts transfer learning technology, using a model pre-trained on a large-scale general image dataset combined with a small amount of student psychological emotion image data for fine-tuning.

[0019] A student online learning evaluation system based on image psychological sentiment analysis, including:

[0020] Image acquisition module, used to collect image information of students when they are learning online in real time;

[0021] Image preprocessing and feature extraction module, which uses image recognition technology to preprocess and extract features from the collected images;

[0022] The psychological and emotional analysis module contains a pre-trained deep learning model to analyze feature data and predict students' psychological and emotional states;

[0023] The learning assessment module comprehensively evaluates students’ online learning effects and generates an assessment report based on their psychological and emotional states and learning behavior data. The above comprehensive assessment formula S=α×P+β×A+γ×Q is used in the assessment process.

[0024] A data storage module for storing the collected image data, feature data, psychological emotion analysis results, learning assessment reports, etc.

[0025] A real-time feedback and interaction module that, after the system identifies the student's emotional state, sends feedback information to the teacher or teaching assistant and triggers the display of interactive content on the student's learning interface.

[0026] A personalized learning advice generation module that combines emotion analysis and learning behavior data and uses intelligent algorithms to generate and push personalized learning advice to students.

[0027] An emotion trend analysis and warning module that conducts long-term tracking and analysis of the student's emotional state data, generates an emotion trend report according to a set time period, and issues a warning to teachers and parents when an abnormal situation is detected.

[0028] Preferably, the image acquisition module is a camera and is installed at a suitable position on the student's learning device to clearly acquire images.

[0029] Preferably, the image preprocessing and feature extraction module is integrated with an edge computing chip to perform preliminary processing and feature extraction on the image locally.

[0030] Preferably, the psychological emotion analysis module adopts a distributed computing architecture to distribute the model calculation tasks to multiple computing nodes for parallel processing.

[0031] Preferably, the learning assessment module uses blockchain technology to ensure the security and immutability of the assessment data, and the assessment report is stored in the data storage module using cloud storage technology, and teachers and students can view it through the corresponding terminal devices.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The present invention simultaneously collects the facial images and limb movement images of students during online learning, extracts various features for psychological emotion analysis, and combines learning behavior data to evaluate the learning effect, changing the previous single evaluation mode that solely relies on grades, comprehensively understanding the learning status of students from multiple dimensions. After the system identifies the student's emotional state, it can immediately send detailed feedback information to the teacher or teaching assistant, enabling the teacher to grasp the psychological emotion dynamics of each student in real time and adjust the teaching rhythm, methods, or content in a timely manner to better meet the needs of students.

[0034] 2. By continuously storing and analyzing the emotional state data of students and generating emotional trend reports according to different time dimensions, the present invention helps teachers and parents comprehensively understand the long-term change trend of students' emotional states. When the emotional state of a student is abnormal, timely warnings are issued, enabling relevant personnel to intervene as early as possible and take targeted measures to help the student adjust the state, ensuring the physical and mental health of the student and the smooth progress of learning.

[0035] 3. By using intelligent recommendation algorithms to accurately screen and push personalized learning resources or suggestions from the learning resource library according to the unique psychological and emotional states and learning behavior data of students, the present invention can fully meet the individual differences of students, help students learn more efficiently based on their own foundation, tap their learning potential, improve learning effects, and enable each student to obtain the most suitable learning path in the online learning environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a block diagram of the method steps of the present invention;

[0037] Figure 2 It is a block diagram of the system modules of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0038] The following further describes in detail the embodiments of the present invention in conjunction with the drawings and examples. The following examples are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0039] As shown in the Figure 1 drawings:

[0040] Example 1:

[0041] A method for online learning assessment of students based on image psychological emotion analysis, comprising:

[0042] S1. Real-time collect the facial images and limb movement images of students during online learning through a camera; the camera is installed at a suitable position on the student's learning device, such as above the computer screen or the front camera of the tablet computer, to ensure that the facial expression changes and limb movement information of the student during the learning process can be clearly and comprehensively captured. The collected image data is temporarily stored in the device cache in the form of digital signals for subsequent processing.

[0043] S2. Use image recognition technology to preprocess the collected images and extract facial feature points, expression features, and limb movement features; among them, the facial feature points include, but are not limited to, the position information of the eyes, mouth, nose, etc., the expression features include, but are not limited to, expression features such as joy, anger, sadness, and happiness, and the limb movement features include, but are not limited to, features such as arm swinging and body posture.

[0044] S3. Input the extracted feature data into a pre-trained psychological emotion analysis model based on deep learning algorithms to predict the psychological emotion state of students; for example, determine whether students are focused, fatigued, bored, etc.

[0045] S4. After identifying the emotion state of students, send feedback information containing a description of the specific emotion state of students to the terminal devices of teachers or teaching assistants through an instant messaging interface, and automatically pop up interactive content corresponding to the emotion state on the student learning interface; enable teachers or teaching assistants to understand the student state in real time. When students show boredom, pop up interesting learning tips or interactive games to improve the learning enthusiasm of students.

[0046] S5. Comprehensively evaluate the online learning effect of students based on their psychological emotion state and learning behavior data, and use intelligent recommendation algorithms to screen and push personalized learning resources or suggestions to students from the learning resource library.

[0047] As can be seen from the above, by collecting students' facial images and limb movement images in real time through a camera, rich student state information is obtained, providing a comprehensive data basis for accurate psychological emotion analysis and learning evaluation, making the evaluation no longer limited to a single dimension and being more able to reflect the real learning state of students.

[0048] Use image recognition technology to extract facial feature points, expression features, and limb movement features, and then use a psychological emotion analysis model based on deep learning algorithms to predict the psychological emotion state of students, achieving accurate insight into students' psychological emotions, which helps to deeply understand the emotions and attitudes of students during the learning process.

[0049] After identifying the emotion state, send feedback information to teachers or teaching assistants through an instant messaging interface, and pop up corresponding interactive content on the student learning interface. On the one hand, teachers or teaching assistants can master the student state in real time and adjust teaching strategies in a timely manner; on the other hand, the interactive content targeted at the student emotion state can stimulate students' learning interest and improve learning enthusiasm and participation.

[0050] Comprehensively evaluate the learning effect based on the student psychological emotion state and learning behavior data, and use intelligent recommendation algorithms to push personalized learning resources or suggestions to meet the individual differences of students, help students learn more efficiently, improve the learning effect, and make online learning more targeted and adaptable.

[0051] Among them, the extraction of facial feature points adopts an algorithm based on the active shape model (ASM), and the position of facial feature points is determined by iteratively optimizing the objective function

[0052]

[0053] where \(x\) i represents the current estimated position of the feature point, represents the position of the corresponding feature point in the average shape model, w x is the weight coefficient, and n is the number of feature points.

[0054] Among them, the evaluation formula for comprehensively evaluating the online learning effect of students is

[0055] S = α×P + β×A + γ×Q

[0056] Among them, S is the comprehensive evaluation score, P is the psychological and emotional state score, A is the learning behavior activity score, Q is the answering correct rate score, α, β, and γ are weight coefficients, and α + β + γ = 1.

[0057] Specifically, continuously store and analyze the emotional state data of students, generate an emotional trend report according to the time dimensions of daily, weekly, and monthly. When the emotional state of students shows continuous depression and abnormal fluctuations, send a warning including the details of the emotional state and the duration to teachers and parents via text messages and in-app notifications.

[0058] As can be seen from the above, when the emotional state of students shows continuous depression (such as being in a negative emotional state for several consecutive days) and abnormal fluctuations (such as frequent changes in the emotional state in a short period of time), send a warning to teachers and parents via text messages, in-app notifications, etc. The warning content includes the details of the emotional state and the duration, so that teachers and parents can pay attention to the situation of students in time and take corresponding measures.

[0059] Specifically, the psychological and emotional analysis model is based on a convolutional neural network (CNN) and uses transfer learning technology. It fine-tunes the model pre-trained on a large-scale general image dataset combined with a small amount of student psychological and emotional image data.

[0060] Among them, CNN has powerful capabilities in image feature extraction.

[0061] As can be seen from the above, by using transfer learning technology and the model pre-trained on a large-scale general image dataset, these pre-trained models have learned rich general image features. On this basis, fine-tuning is carried out in combination with a small amount of student psychological and emotional image data, which not only reduces the demand for training data volume but also enables the model to quickly adapt to the specific task of student psychological and emotional analysis, improving the model training efficiency and accuracy.

[0062] As shown in the Figure 2 appendix:

[0063] Embodiment 2

[0064] A student online learning evaluation system based on image psychological and emotional analysis includes:

[0065] The image acquisition module is used to collect the image information of students during online learning in real time, including facial images and limb movement images, and is the data source of the entire system;

[0066] The image preprocessing and feature extraction module uses image recognition technology to preprocess the collected images, such as removing noise, normalization, etc., and then extracts facial feature points, expression features, and limb movement features to provide data support for subsequent psychological and emotional analysis.

[0067] The psychological and emotional analysis module contains a pre-trained deep learning model, receives the feature data from the image preprocessing and feature extraction module, and analyzes and predicts the psychological and emotional state of students through the model;

[0068] The learning evaluation module comprehensively evaluates the online learning effect of students and generates an evaluation report based on the psychological and emotional state and learning behavior data; in the evaluation process, the above comprehensive evaluation formula S = α×P + β×A + γ×Q is used to provide detailed feedback on the learning situation for teachers and students;

[0069] The data storage module is used to store various data generated during the operation of the system, including the collected image data, the extracted feature data, the psychological and emotional analysis results, and the learning evaluation report, etc., to ensure the traceability of the data and subsequent analysis and use.

[0070] The real-time feedback and interaction module, after the system identifies the emotional state of students, on the one hand, sends feedback information to teachers or teaching assistants, and on the other hand, triggers the display of interactive content on the student learning interface to achieve real-time communication and interactive guidance.

[0071] The personalized learning suggestion generation module combines the emotional analysis results and learning behavior data of students, uses intelligent algorithms to generate personalized learning suggestions, and pushes them to students to meet the personalized learning needs of students.

[0072] The emotional trend analysis and early warning module conducts long-term tracking and analysis on the emotional state data of students, and generates an emotional trend report according to the set time period (such as daily, weekly, monthly). At the same time, when detecting abnormal situations in the emotional state of students, it issues early warnings to teachers and parents to pay attention to the emotional changes of students in a timely manner.

[0073] Specifically, the image acquisition module is a camera, and is installed at a suitable position on the student learning device to clearly collect images.

[0074] As can be seen from the above, the specific device of the image acquisition module is a camera, and the importance of its installation position is emphasized. It needs to be installed at a suitable position on the student learning device to ensure that the facial images and limb movement images of students can be clearly collected, providing high-quality data for subsequent image analysis.

[0075] Specifically, the image preprocessing and feature extraction module is integrated with an edge computing chip to perform preliminary image processing and feature extraction locally.

[0076] As can be seen from the above, the module integrated with an edge computing chip enables the preliminary processing and feature extraction of images to be carried out locally, reducing the data transmission volume, improving the processing efficiency, reducing the dependence on network bandwidth, and also protecting the privacy of students to a certain extent because some data can be preprocessed without being uploaded to a remote server.

[0077] Specifically, the psychological emotion analysis module adopts a distributed computing architecture to distribute the model computing tasks to multiple computing nodes for parallel processing.

[0078] As can be seen from the above, by adopting a distributed computing architecture and distributing the model computing tasks to multiple computing nodes for parallel processing. This method can make full use of the computing resources of multiple computing nodes, improve the computing speed and processing ability of the psychological emotion analysis model. Especially when processing a large amount of image data and feature analysis of students, it can quickly obtain the analysis results and meet the real-time requirements of the system.

[0079] Specifically, the learning evaluation module uses blockchain technology to ensure the security and immutability of evaluation data, and the evaluation reports are stored in the data storage module using cloud storage technology, which can be viewed by teachers and students through corresponding terminal devices.

[0080] As can be seen from the above, the learning evaluation module uses blockchain technology to ensure the security and immutability of evaluation data. The distributed ledger and encryption technology of blockchain make it difficult to tamper with evaluation data, ensuring the fairness and credibility of evaluation results. The evaluation reports are stored in the data storage module using cloud storage technology, and cloud storage has the characteristics of strong scalability and high data reliability. Teachers and students can conveniently view the evaluation reports through corresponding terminal devices, realizing the convenient access and sharing of data.

[0081] Importantly, it should be noted that the construction and arrangement of the present application shown in multiple different exemplary embodiments are merely illustrative. Although only a few embodiments are described in detail in this disclosure, those who refer to this disclosure should easily understand that many modifications are possible without materially departing from the novel teachings and advantages of the subject matter described in this application (e.g., changes in the dimensions, scales, structures, shapes and proportions of various elements, as well as parameter values (such as temperature, pressure, etc.), installation arrangements, use of materials, colors, orientations, etc.). For example, an element shown as integrally formed may be composed of multiple parts or elements, the position of the element may be inverted or otherwise changed, and the nature, number or position of discrete elements may be altered or changed. Accordingly, all such modifications are intended to be included within the scope of the present invention. The order or sequence of any process or method steps may be changed or reordered according to alternative embodiments. In the claims, any "means-plus-function" clause is intended to cover the structures that perform the recited function herein, and not only structural equivalents but also equivalent structures. Other substitutions, modifications, changes and omissions may be made in the design, operating conditions and arrangement of the exemplary embodiments without departing from the scope of the present invention. Accordingly, the present invention is not limited to a particular embodiment, but extends to various modifications that still fall within the scope of the appended claims.

[0082] In addition, in order to provide a concise description of the exemplary embodiments, not all features of the actual embodiments may be described (i.e., those features that are not relevant to the currently contemplated best mode of carrying out the present invention, or those features that are not relevant to the implementation of the present invention).

[0083] It should be understood that in the development of any actual implementation, as in any engineering or design project, numerous specific implementation decisions may be made. Such development efforts may be complex and time-consuming, but for those of ordinary skill in the art who benefit from this disclosure, without undue experimentation, such development efforts will be a routine task of design, manufacturing and production.

[0084] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention may be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A student online learning evaluation method based on image psychological sentiment analysis, characterized in that: include: S1, collect the facial images and body movement images of students in real time when they are learning online through the camera; S2. Use image recognition technology to pre-process the collected images and extract facial feature points, expression features and body movement features; S3, input the extracted feature data into a pre-trained psychological emotion analysis model based on a deep learning algorithm to predict the psychological emotion state of the students; S4. After identifying the student's emotional state, feedback information containing a description of the student's specific emotional state is sent to the teacher's or teaching assistant's terminal device through the instant messaging interface, and interactive content corresponding to the emotional state is automatically popped up on the student's learning interface; S5. Comprehensively evaluate students’ online learning outcomes based on their psychological and emotional states and learning behavior data, and use intelligent recommendation algorithms to filter and push personalized learning resources or suggestions to students from the learning resource library.

2. According to claim 1, a student online learning evaluation method based on image psychological emotion analysis is characterized by: The facial feature points are extracted by using an algorithm based on the active shape model (ASM) by iteratively optimizing the objective function Determine the location of facial landmarks, x i represents the current estimated feature point position, represents the position of the corresponding feature point in the average shape model, w x is the weight coefficient, and n is the number of feature points.

3. According to claim 1, a student online learning evaluation method based on image psychological emotion analysis is characterized by: The evaluation formula for comprehensively evaluating students' online learning effects is: S=α×P+β×A+γ×Q Among them, S is the comprehensive evaluation score, P is the psychological and emotional state score, A is the learning behavior activity score, Q is the answer accuracy score, α, β, γ are weight coefficients, and α+β+γ=1.

4. According to claim 1, a student online learning evaluation method based on image psychological emotion analysis is characterized by: Continuously store and analyze students' emotional state data, generate emotional trend reports by day, week, and month. When students' emotional states show persistent lows and abnormal fluctuations, early warnings containing emotional state details and duration will be sent to teachers and parents via text messages and in-app notifications.

5. According to claim 1, a student online learning evaluation method based on image psychological emotion analysis is characterized by: The psychological emotion analysis model is based on a convolutional neural network (CNN) and adopts transfer learning technology, using a model pre-trained on a large-scale general image dataset combined with a small amount of student psychological emotion image data for fine-tuning.

6. A student online learning evaluation system based on image psychological sentiment analysis, characterized in that: include: Image acquisition module, used to collect image information of students when they are learning online in real time; Image preprocessing and feature extraction module, which uses image recognition technology to preprocess and extract features from the collected images; The psychological and emotional analysis module contains a pre-trained deep learning model to analyze feature data and predict students' psychological and emotional states; The learning assessment module comprehensively evaluates students’ online learning effects and generates an assessment report based on their psychological and emotional states and learning behavior data. The above comprehensive assessment formula S=α×P+β×A+γ×Q is used in the assessment process. Data storage module, used to store collected image data, feature data, psychological emotion analysis results, and learning evaluation reports; Real-time feedback and interaction module, after the system recognizes the student's emotional state, it sends feedback information to the teacher or teaching assistant and triggers the display of interactive content on the student learning interface; The personalized learning suggestion generation module combines sentiment analysis and learning behavior data, uses intelligent algorithms to generate and push personalized learning suggestions to students; The emotional trend analysis and early warning module conducts long-term tracking and analysis of students' emotional status data, generates emotional trend reports according to set time periods, and issues early warnings to teachers and parents when abnormal situations are detected.

7. The student online learning evaluation system based on image psychological emotion analysis according to claim 6 is characterized by: The image acquisition module is a camera and is installed at a suitable position of the student learning device to clearly capture images.

8. The student online learning evaluation system based on image psychological emotion analysis according to claim 6 is characterized by: The image preprocessing and feature extraction module is integrated with an edge computing chip to perform preliminary processing and feature extraction on the image locally.

9. The student online learning evaluation system based on image psychological emotion analysis according to claim 6 is characterized by: The psychological emotion analysis module adopts a distributed computing architecture to distribute model computing tasks to multiple computing nodes for parallel processing.

10. The student online learning evaluation system based on image psychological emotion analysis according to claim 6 is characterized by: The learning assessment module uses blockchain technology to ensure the security and non-tamperability of the assessment data, and the assessment report is stored in a data storage module using cloud storage technology, which can be viewed by teachers and students through corresponding terminal devices.

Images