Student real-time feedback and teaching system based on multi-modal emotion calculation

By combining facial expressions, voice, and physiological signal data, a multimodal emotion computing system can identify students' emotional states in real time and provide personalized feedback, solving the problem that emotional changes are difficult to capture in traditional teaching and improving teaching effectiveness and student participation.

CN121329028APending Publication Date: 2026-01-13河南开放大学
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511473377.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-10-15
Publication Date
2026-01-13

AI Technical Summary

Technical Problem

In traditional teaching models, it is difficult for teachers to understand students' emotional changes in real time and comprehensively, and single-modal affective computing methods cannot accurately reflect students' true emotional state.

Method used

A multimodal emotion computing system is adopted to identify students' emotional states in real time by collecting facial expression, voice and physiological signal data, combined with deep learning models and weighted fusion algorithms, and to provide personalized feedback and teaching adjustments.

Benefits of technology

This approach enabled a comprehensive and accurate understanding of students' emotions, improved the relevance and effectiveness of teaching, enhanced students' learning enthusiasm and classroom interaction, and optimized the teaching content and pace.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121329028A_ABST
    Figure CN121329028A_ABST
Patent Text Reader

Abstract

The invention provides a student real-time feedback and teaching system based on multi-modal emotion calculation, and the system comprises a multi-modal data collection module which is used for collecting multi-modal data of a student in a learning process in real time, and an emotion calculation module which is used for receiving the multi-modal data, carrying out the fusion analysis of the multi-modal data through employing a multi-modal fusion algorithm, and carrying out the teaching of the multi-modal data. The real-time emotional state of the student is identified; the student feedback module is used for generating corresponding student feedback information according to the real-time emotional state, recognized by the emotion calculation module, of the student; the teaching adjustment module is used for adjusting the teaching content, the teaching method or the teaching rhythm according to the real-time emotional state of the student and the feedback information of the student, and sending the adjusted teaching scheme to the user side; the user side is used for receiving the feedback information of the students and the adjusted teaching plans, and according to the real-time emotional states and the learning conditions of the students, the teaching effect and the academic records of the students can be improved, and personalized teaching feedback is provided.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of teaching systems, and more particularly to a real-time student feedback and teaching system based on multimodal affective computing. Background Technology

[0002] In traditional teaching models, teachers primarily rely on limited information such as students' classroom performance and homework completion to understand their learning status, making it difficult to grasp each student's emotional changes in a real-time and comprehensive manner. However, emotional state has a crucial impact on students' learning outcomes; positive emotions help increase learning interest and participation, while negative emotions may lead to a decline in learning motivation.

[0003] With the rapid development of artificial intelligence technology, affective computing is gradually emerging in the field of education. However, most existing affective computing methods are based on single-modal information, such as facial expression recognition or voice emotion analysis. Due to the differences and complementarities between different modalities in emotional expression, a single modality is difficult to accurately and comprehensively reflect the student's true emotional state.

[0004] Therefore, it is necessary to provide a new student real-time feedback and teaching system based on multimodal affective computing to solve the above-mentioned technical problems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a real-time student feedback and teaching system based on multimodal affective computing.

[0006] The real-time student feedback system based on multimodal emotion computing provided by this invention includes: The multimodal data acquisition module is used to collect students' multimodal data in real time during the learning process. The multimodal data includes facial expression data, voice data, and physiological signal data. An emotion computing module, connected to the multimodal data acquisition module, is used to receive the multimodal data and perform fusion analysis on the multimodal data using a multimodal fusion algorithm to identify the student's real-time emotional state. The student feedback module, connected to the emotion computing module, is used to generate corresponding student feedback information based on the real-time emotional state of the student identified by the emotion computing module. The student end is used to receive teaching content and interact with the learning process.

[0007] Furthermore, the multimodal data acquisition module includes: Facial expression capture unit: Uses a high-definition camera to capture students' facial image data in real time. Voice acquisition unit: Uses a microphone array to acquire students' voice data in real time. Physiological signal acquisition unit: The wearable device is used to collect students' physiological signal data in real time. The physiological signal data includes heart rate, skin conductance response and electroencephalogram (EEG) signals. The collected physiological signal data is sent to the emotion computing module in real time.

[0008] Furthermore, the multimodal fusion algorithm employed by the emotion computing module includes the following steps: The facial expression data, voice data, and physiological signal data are preprocessed respectively, and the preprocessing includes data cleaning, feature extraction, and normalization. Deep learning models were used to classify the preprocessed facial expression data, speech data, and physiological signal data into emotions, and the emotion classification results for each modality of data were obtained. A weighted fusion method is used to fuse the sentiment classification results corresponding to each modality of data to obtain the student's comprehensive sentiment state. The weight coefficients of the weighted fusion method are dynamically adjusted according to the importance and reliability of different modalities in sentiment expression.

[0009] Furthermore, the emotion classification results corresponding to the facial expression data include happiness, sadness, anger, surprise, disgust, and neutral; the emotion classification results corresponding to the voice data include positive, negative, and neutral; and the emotion classification results corresponding to the physiological signal data include excitement, calmness, and tension.

[0010] Furthermore, the student feedback information generated by the student feedback module includes at least one of the following: Emotional state feedback: Display students' real-time emotional state to students and teachers in graphical or textual form; Learning suggestions and feedback: Based on students' real-time emotional state and historical learning data, provide students with personalized learning suggestions, such as adjusting the learning pace, increasing learning resources, or conducting targeted exercises; Classroom interaction feedback: Based on students' real-time emotional state, encourage students to actively participate in classroom interactions, such as asking questions, discussions, or group activities.

[0011] A teaching system based on multimodal affective computing includes: The teacher's end is used to receive the student feedback information; The teaching adjustment module, connected to the emotion computing module and the student feedback module, is used to adjust the teaching content, teaching methods or teaching pace based on the students' real-time emotional state and the students' feedback information, and send the adjusted teaching plan to the teacher's end.

[0012] Furthermore, the teaching adjustment module adjusts the teaching content, teaching methods, or teaching pace based on the student's real-time emotional state and student feedback information in at least one of the following ways: Adjusting teaching content: Based on students' real-time emotional state and learning needs, increase or decrease the difficulty and quantity of teaching content, or adjust the way the teaching content is presented; Adjusting teaching methods: Selecting more suitable teaching methods based on students' real-time emotional state and learning style; Adjusting the pace of teaching: Adjust the teaching progress and pace according to the students' real-time emotional state and level of concentration.

[0013] Furthermore, the teacher and student terminals communicate with the multimodal data acquisition module, emotion computing module, student feedback module, and teaching adjustment module via wired or wireless networks, and the communication protocol adopts TCP / IP or HTTP protocol.

[0014] Furthermore, the teacher's and student's ends are equipped with a user interface, which includes the following functional areas: Data display area: Used to display students' real-time emotional state, student feedback information, and adjusted teaching plans; Operation control area: used by teachers to input teaching instructions and content, and by students to perform learning interactions and feedback operations; Settings area: Used by teachers and students to configure system parameters.

[0015] Compared with related technologies, the real-time student feedback and teaching system based on multimodal affective computing provided by this invention has the following beneficial effects: 1. This invention, through multimodal data acquisition and fusion analysis, can comprehensively and accurately acquire students' emotional information, overcome the limitations of single-modal emotion computing, and accurately perceive students' emotional state.

[0016] 2. Based on students' real-time emotional state and learning situation, this invention provides students with personalized learning suggestions and feedback, provides teachers with a basis for adjusting teaching strategies, helps improve teaching effectiveness and student academic performance, and provides personalized teaching feedback.

[0017] 3. This invention can promptly encourage students to participate in classroom interaction, improve students' learning enthusiasm and participation, create a lively classroom atmosphere, enhance classroom interactivity, and teachers can adjust teaching content, teaching methods and teaching pace in real time based on the feedback information provided by the system, so that teaching is more in line with students' learning needs, improves the pertinence and effectiveness of teaching, and realizes dynamic adjustment of teaching. Attached Figure Description

[0018] Figure 1This invention provides a system block diagram of a real-time student feedback and teaching system based on multimodal affective computing. Figure 2 This is a structural block diagram of the multimodal data acquisition module provided by the present invention. Detailed Implementation

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Please refer to the following: Figure 1 as well as Figure 2 ,in, Figure 1 This invention provides a system block diagram of a real-time student feedback and teaching system based on multimodal affective computing. Figure 2 This is a structural block diagram of the multimodal data acquisition module provided by the present invention.

[0021] In the specific implementation process, such as Figures 1-2 As shown, the real-time student feedback and teaching system based on multimodal affective computing consists of a multimodal data acquisition module, an affective computing module, a student feedback module, a teaching adjustment module, a teacher's end, and a student's end. These modules collaborate to achieve real-time perception and feedback of students' emotional states and dynamic adjustment of teaching.

[0022] The multimodal data acquisition module includes: The facial expression capture unit uses a high-definition camera with autofocus and face recognition capabilities. Autofocus ensures clear capture of students' facial images at various distances, while face recognition quickly locates the student's face, improving the accuracy and efficiency of data acquisition. The capture frequency is set to 10-15 frames per second, ensuring the capture of subtle changes in facial expressions without generating excessive data and increasing the system's processing burden. The captured facial image data is transmitted in real-time to the emotion computing module via a USB 3.0 interface.

[0023] The voice acquisition unit uses a microphone array composed of multiple microphones and has a certain degree of spatial directivity. Its noise reduction function can effectively remove environmental noise, such as the sound of fans in the classroom and the conversations of other students, thus improving the quality of voice acquisition. The sampling rate is set to 16kHz, which can meet the audio quality requirements of voice emotion analysis. Voice data is sent to the emotion computing module via Wi-Fi or Bluetooth wireless transmission.

[0024] The physiological signal acquisition unit uses wearable devices, such as smart bracelets or wristbands, with multiple built-in sensors to collect students' physiological signal data in real time, including heart rate, skin conductance response, and electroencephalogram (EEG) signals. These physiological signals can reflect students' emotional fluctuations and physical state. The heart rate acquisition frequency is once per second, the skin conductance response acquisition frequency is five times per second, and the EEG signal acquisition frequency is set to an appropriate value according to the device performance and requirements, preferably 100-500 times per second. The collected physiological signal data is sent to the emotion computing module in real time via Bluetooth Low Energy.

[0025] It should be noted that the sentiment computing module includes data preprocessing, deep learning model training, and multimodal fusion algorithms; Facial expression data preprocessing: First, the acquired facial images are converted to grayscale to reduce data dimensionality; then, histogram equalization is used to enhance image contrast and improve the visibility of facial features; next, a face detection algorithm is used to locate key facial feature points, such as eyes, mouth, and eyebrows; finally, facial expression features, such as the degree of eye opening and closing and the shape of the mouth, are extracted and normalized to map the feature values ​​to the [0,1] interval. Speech data preprocessing: The acquired speech signal is pre-emphasized to enhance the high-frequency components; then, a frame-segmentation and windowing operation is performed to divide the speech signal into short-time frames, with each frame preferably 20-30ms in length and a frame shift of 10-15ms; next, speech features such as Mel frequency cepstral coefficients, short-time energy, and zero-crossing rate are extracted and normalized. Physiological signal data preprocessing: heart rate data is filtered to remove noise interference; skin conductance response data is smoothed to reduce short-term fluctuations; electroencephalogram (EEG) signal data is detrended and filtered to extract useful EEG features, such as alpha waves and beta waves, and then normalized.

[0026] It should be noted that the deep learning model is trained as follows: Facial expression classification model: A convolutional neural network is used. It is pre-trained using a publicly available facial expression dataset and then fine-tuned on locally collected student facial expression data. During training, the cross-entropy loss function and stochastic gradient descent optimization algorithm are used. With appropriate learning rate and batch size set, the model is trained through multiple iterations, enabling it to accurately recognize facial expressions such as happiness, sadness, anger, surprise, disgust, and neutrality.

[0027] The speech emotion classification model employs recurrent neural networks and their variants, long short-term memory networks or gated recurrent units. It is pre-trained using publicly available speech emotion datasets and then fine-tuned using locally collected student speech data. During training, the cross-entropy loss function and optimization algorithm are used to adjust the model parameters, enabling it to distinguish between positive, negative, and neutral speech emotions.

[0028] Physiological Signal Emotion Classification Model: A corresponding classification model is constructed based on the types of collected physiological signals. For heart rate and skin conductance response data, traditional machine learning algorithms such as support vector machines or decision trees can be used for classification; for electroencephalogram (EEG) signal data, deep learning models, such as one-dimensional CNNs or LSTMs, can be combined for classification. The model is trained using locally collected student physiological signal data and their corresponding emotion labels, enabling it to accurately determine students' physiological and emotional states, such as excitement, calmness, and tension.

[0029] It should be noted that the multimodal fusion algorithm is as follows: a weighted fusion method is used to fuse the sentiment classification results corresponding to the three modalities. First, based on the importance and reliability of different modalities in sentiment expression, corresponding weight coefficients are assigned to each modality. For example, facial expressions are usually more intuitive in sentiment expression and can be given a higher weight; vocal emotions can reflect a student's tone and intonation and also have a certain weight; physiological signals can reflect a student's emotional state at the physical level and also need to be given a certain weight. The weight coefficients can be determined through a combination of experimental analysis and expert experience.

[0030] During the fusion process, the sentiment classification results of the three modalities are weighted and summed according to the weight coefficients to obtain the student's comprehensive sentiment score. Based on the comprehensive sentiment score, it is mapped to the corresponding sentiment category, such as positive, negative or neutral, thereby obtaining the student's comprehensive sentiment state.

[0031] It should be noted that the student feedback module includes the following: Emotional state feedback: Display students' real-time emotional state in a graphical way on the user interface of both students and teachers. For example, use different colored emoji icons to represent different emotional categories, green for positive, red for negative, and yellow for neutral. At the same time, it can display the trend of emotional state changes, such as showing the fluctuation of students' emotional state over a period of time through a line graph. Learning suggestion feedback: Based on students' real-time emotional state and historical learning data, the system provides personalized learning suggestions. For example, if a student shows negative emotions and historical learning data shows that they have not mastered a certain knowledge point, the system will suggest that the student increase learning resources for the relevant knowledge point, such as watching instructional videos and doing specific practice questions. If the student is in a positive emotional state, the system can encourage the student to challenge themselves with more difficult learning tasks, and the learning suggestions will be displayed in a text pop-up prompt box on the student's end.

[0032] Classroom interaction feedback: Based on students' real-time emotional state, the system encourages students to actively participate in classroom interactions. When the system detects that a student is exhibiting positive emotions, it displays encouraging messages on the student's device; when a student is exhibiting negative emotions, it prompts the teacher to pay attention to that student and guide them to participate in some simple interactive activities, such as answering questions or group discussions, to improve student participation and enthusiasm.

[0033] It should be noted that the teaching adjustment module includes the following: Teaching content adjustment: The system adjusts the teaching content based on students' real-time emotional state and learning needs. If most students exhibit negative emotions, it indicates that the current teaching content may be too difficult or too boring. The system will suggest that teachers appropriately reduce the difficulty of the teaching content and add some vivid and interesting cases or examples. If students exhibit positive emotions and are learning quickly, the system will suggest that teachers add some extended teaching content to meet students' learning needs.

[0034] Adjustments to teaching methods: Based on students' real-time emotional state and learning style, select more suitable teaching methods. For example, for students who prefer intuitive learning, when they show positive emotions, case analysis or project-based learning can be used to allow students to deepen their understanding of knowledge through practical cases or projects. For students who prefer abstract thinking, lectures combined with discussions can be used to guide students to think and communicate in depth.

[0035] Teaching pace adjustment: The teaching progress and pace are adjusted according to students' real-time emotional state and concentration level. If students show inattention or negative emotions for a period of time, the system will suggest that teachers increase rest time or relieve students' fatigue through some interactive activities. If students show high concentration and positive emotions, teachers can appropriately speed up the teaching pace and improve teaching efficiency.

[0036] It should be noted that the teacher's and student's ends include the following: Communication method: The teacher and student terminals communicate with the multimodal data acquisition module, emotion computing module, student feedback module and teaching adjustment module through wired or wireless networks, and use TCP / IP or HTTP protocol to ensure the stability and reliability of data transmission.

[0037] User interface design, data display area: used to display students' real-time emotional status, student feedback information and adjusted teaching plans, presenting data in a clear and intuitive way for easy viewing by teachers and students.

[0038] Operation control area: The teacher's side provides an interface for inputting teaching instructions and teaching content, such as inputting teaching content through text input boxes and selecting teaching methods and teaching pace through buttons; the student's side provides an entry point for learning interaction and feedback operations, such as answering questions, submitting assignments, and providing feedback on learning experiences.

[0039] Settings Area: Teachers and students can configure system parameters such as data collection frequency, emotion classification threshold, and feedback method. Through the settings area, users can personalize the system according to their actual needs.

[0040] The overall process of this teaching system includes: (I) System Initialization Before the start of class, the teacher opens the teacher's software, and the students open the student's software. The system automatically activates the multimodal data acquisition module, and high-definition cameras, microphone arrays, and wearable devices begin collecting students' facial expressions, voice, and physiological signal data, transmitting the data to the emotion computing module in real time.

[0041] (II) Affective computing and analysis After receiving multimodal data, the emotion computing module performs real-time calculation and analysis of the student's emotional state according to the above steps of data preprocessing, deep learning model training and multimodal fusion algorithm, and obtains the student's comprehensive emotional state.

[0042] (III) Student Feedback and Teaching Adjustments Student Feedback: The student feedback module displays students' real-time emotional states in graphical and textual formats on both the student and teacher user interfaces. Simultaneously, it provides personalized learning suggestions and classroom interaction feedback based on students' emotional states and learning progress.

[0043] Teaching Adjustment: The teaching adjustment module provides teachers with suggestions for adjusting teaching content, methods, and pace based on students' real-time emotional state and feedback. Teachers can then adjust their teaching plans by making corresponding adjustments on their own devices based on these suggestions.

[0044] (iv) Continuous optimization of the teaching process During the teaching process, the system continuously collects students' multimodal data, updates students' emotional state and feedback information in real time, and teachers continuously adjust their teaching strategies based on the system's feedback, forming a dynamically optimized teaching process until the end of the class.

[0045] According to embodiments of the present invention, a computing device that can be used to implement the above method includes a processor and a memory; The processor can be a multi-core processor or include multiple processors. In some embodiments, the processor may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor may be implemented using custom circuitry, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0046] Memory can include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM can store static data or instructions required by the processor or other modules of the computer. Permanent storage devices can be read-write storage devices. Permanent storage devices can be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices can be removable storage devices (e.g., floppy disks, optical drives). System memory can be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory can store some or all of the instructions and data required by the processor during operation. Furthermore, memory can include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks can also be used. In some implementations, the memory may include removable storage devices that are readable and / or writable, such as laser discs (CDs), read-only digital versatile optical discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), read-only Blu-ray discs, ultra-high density optical discs, flash memory cards (e.g., SD cards, mini SD cards, Micro-SD cards, etc.), magnetic floppy disks, etc. Computer-readable storage media do not include carrier waves and transient electronic signals transmitted wirelessly or via wired connections.

[0047] It should be understood that, unless otherwise expressly stated herein, there is no strict order restriction on the execution of the above steps, and these steps may be executed in other orders. Moreover, at least some steps in the processes involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0048] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0049] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment includes only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A student real-time feedback system based on multi-modal affective computing, characterized in that, The application relates to a student emotion feedback system, which comprises the following: a multi-modal data acquisition module for acquiring multi-modal data of students in the learning process in real time, wherein the multi-modal data comprises facial expression data, voice data and physiological signal data; an emotion computing module connected with the multi-modal data acquisition module, for receiving the multi-modal data and performing fusion analysis on the multi-modal data by using a multi-modal fusion algorithm to identify the real-time emotional state of the student; a student feedback module connected with the emotion computing module, for generating corresponding student feedback information according to the real-time emotional state of the student identified by the emotion computing module; a student terminal for receiving teaching content and interacting with learning.

2. The multimodal sentiment computing based student real-time feedback system as claimed in claim 1, wherein, The multi-modal data acquisition module comprises: a facial expression acquisition unit for acquiring facial image data of the student in real time by using a high-definition camera a voice acquisition unit for acquiring voice data of the student in real time by using a microphone array a physiological signal acquisition unit for acquiring physiological signal data of the student in real time by using a wearable device, wherein the physiological signal data comprises heart rate, skin electric response and electroencephalogram signal, and the acquired physiological signal data is sent to the emotion computing module in real time.

3. The multimodal sentiment computing based student real-time feedback system as claimed in claim 1, wherein, The multi-modal fusion algorithm used by the emotion computing module comprises the following steps: preprocessing the facial expression data, voice data and physiological signal data respectively, wherein the preprocessing comprises data cleaning, feature extraction and normalization processing; performing emotion classification on the preprocessed facial expression data, voice data and physiological signal data respectively by using a deep learning model to obtain the emotion classification result corresponding to each kind of modal data; performing fusion on the emotion classification result corresponding to each kind of modal data by using a weighted fusion method to obtain the comprehensive emotional state of the student, wherein the weight coefficient of the weighted fusion method is dynamically adjusted according to the importance and reliability of different modal data in emotional expression.

4. The multimodal sentiment computing based student real-time feedback system as claimed in claim 1, wherein, The emotion classification result corresponding to the facial expression data comprises happiness, sadness, anger, surprise, disgust and neutrality; the emotion classification result corresponding to the voice data comprises positivity, negativity and neutrality; and the emotion classification result corresponding to the physiological signal data comprises excitement, calmness and tension.

5. The multimodal sentiment computing based student real-time feedback system as claimed in claim 1, wherein, The student feedback information generated by the student feedback module comprises at least one of the following: emotional state feedback: displaying the real-time emotional state of the student to the student terminal and the teacher terminal in graphical or textual form; learning suggestion feedback: providing personalized learning suggestions for the student according to the real-time emotional state of the student and historical learning data, such as adjusting the learning rhythm, increasing learning resources or conducting targeted exercises; classroom interaction feedback: encouraging the student to actively participate in classroom interaction, such as asking questions, discussing or group activities, according to the real-time emotional state of the student.

6. A teaching system based on multi-modal affective computing, suitable for the student real-time feedback system based on multi-modal affective computing according to any one of claims 1-5, characterized in that, The application also relates to a student emotion feedback system, which comprises the following: a teacher terminal for receiving the student feedback information; a teaching adjustment module connected with the emotion computing module and the student feedback module, for adjusting the teaching content, teaching method or teaching rhythm according to the real-time emotional state of the student and the student feedback information, and sending the adjusted teaching scheme to the teacher terminal.

7. The multimodal affective computing based student real-time feedback and teaching system as claimed in claim 6, wherein, The teaching adjustment module adjusts the teaching content, teaching method or teaching rhythm according to the real-time emotional state of the student and the student feedback information, and the adjustment mode includes at least one of the following: Teaching content adjustment: according to the real-time emotional state of the student and the learning needs, increase or decrease the difficulty and quantity of the teaching content, or adjust the presentation mode of the teaching content; Teaching method adjustment: according to the real-time emotional state of the student and the learning style, select a more suitable teaching method; Teaching rhythm adjustment: according to the real-time emotional state of the student and the degree of attention concentration, adjust the teaching progress and rhythm.

8. The student real-time feedback and teaching system based on multi-modal sentiment computing as claimed in claim 7, wherein, The teacher end and the student end communicate with the multi-modal data acquisition module, the emotional computing module, the student feedback module and the teaching adjustment module through wired or wireless network, and the communication protocol adopts TCP / IP protocol or HTTP protocol.

9. The multimodal sentiment computing based student real-time feedback and teaching system as claimed in claim 8, wherein, The teacher end and the student end are provided with user interface, and the user interface includes the following functional areas: Data display area: for displaying the real-time emotional state of the student, the student feedback information and the adjusted teaching scheme; Operation control area: for the teacher to input teaching instructions and teaching content, and for the student to learn interaction and feedback operation; Setting area: for the teacher and the student to set the parameters of the system.