A multi-modal ideological and political teaching evaluation system fusing electroencephalogram signals and eye movement signals

Abstract

The application relates to the field of artificial intelligence and multi-modal signal processing technology, and discloses a multi-modal ideological and political teaching evaluation system fusing electroencephalogram signals and eye movement signals, which is used for solving the problems that the existing teaching evaluation method lacks learning state explicit modeling, is difficult to depict the dynamic evolution process of classroom learning state, and lacks the adaptive adjustment ability of fusion weight. The system synchronously collects electroencephalogram signals and eye movement signals of students, constructs electroencephalogram feature vectors and eye movement feature vectors, and maps the electroencephalogram feature vectors and the eye movement feature vectors to a unified feature space, and generates a cross-modal correlation feature vector through a cross-modal correlation matrix; a learning state vector and state vectors of cognitive input, emotional identification and behavior participation are generated based on a neural network gating mechanism, and then electroencephalogram mode and eye movement mode weights are dynamically generated through a state control network, a state-driven fusion feature vector is obtained, and teaching effect scores and teaching state trends are output in combination with explicit evaluation feature vectors, so that the system is suitable for ideological and political classroom teaching dynamic evaluation and teaching feedback.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and multimodal signal processing technology, and in particular to a multimodal ideological and political education assessment system that integrates electroencephalogram (EEG) signals and eye-tracking signals. Background Technology

[0002] The evaluation of ideological and political education classroom teaching effectiveness typically involves multiple dimensions, including knowledge comprehension, cognitive engagement, emotional identification, behavioral participation, and value internalization. Current assessment methods for ideological and political education classroom teaching largely rely on exam scores, classroom questionnaires, post-class interviews, or teachers' subjective observations to analyze students' learning status and teaching effectiveness. While these methods can reflect students' mastery of course content to some extent, they are primarily based on outcome-based evaluation or experiential judgment, making it difficult to continuously, objectively, and quantitatively depict changes in students' states during the classroom teaching process. In particular, they struggle to reflect students' intrinsic learning states, such as cognitive engagement, emotional identification, and behavioral participation.

[0003] With the development of artificial intelligence, educational data mining, and multimodal signal processing technologies, intelligent teaching assessment methods based on physiological signals, behavioral data, and classroom process data are increasingly being applied to classroom teaching analysis. Among these, electroencephalogram (EEG) signals can reflect students' cognitive load, attentional state, and changes in brain activity during the learning process, while eye-tracking (EMT) signals can characterize students' fixation areas, visual attention stability, saccadic behavior, and classroom attention distribution. Therefore, applying EEG and EMT signals to the assessment of ideological and political education classroom teaching helps to dynamically perceive students' learning process from both implicit physiological states and explicit behavioral manifestations.

[0004] In existing technologies, some multimodal teaching assessment methods jointly model data from different modalities through feature concatenation, fixed-weight fusion, or attention-based fusion, and then use classification or regression models to output learning state categories, attention levels, or teaching assessment results. However, existing methods still have the following shortcomings:

[0005] First, most existing methods directly output evaluation results based on multimodal features, lacking explicit modeling of students' learning status. They are difficult to express the ideological and political classroom learning process from semantic dimensions such as cognitive input, emotional identification, and behavioral participation, resulting in evaluation results lacking semantic support for teaching explanation and teaching feedback.

[0006] Secondly, existing methods usually focus on data analysis under a single time window or discrete classroom segments, lacking state evolution modeling of the learning state change process under continuous time windows, and are difficult to reflect the dynamic change trend of students' learning state in different teaching stages such as classroom teaching, interaction, and testing.

[0007] Furthermore, existing multimodal fusion methods mostly employ static fusion strategies, simple feature splicing, or fixed fusion relationships learned automatically by the model. They lack a mechanism to dynamically adjust the contribution of EEG and eye-tracking modal information based on the current learning state, making it difficult to adapt to the constantly changing cognitive load, attention distribution, and behavioral participation state in classroom scenarios, thus affecting the stability and robustness of multimodal fusion results.

[0008] Furthermore, existing teaching assessment methods often rely solely on implicit physiological signals or explicit teaching behavior data, lacking a mechanism to jointly assess implicit process data such as EEG and eye movement with explicit teaching behavior data such as attendance rate, homework completion, classroom test scores, and classroom interaction frequency. This makes it difficult to form a multi-dimensional, process-oriented, and continuous evaluation of classroom teaching effectiveness.

[0009] Therefore, how to construct a multimodal ideological and political education teaching evaluation system that can simultaneously integrate EEG and eye-tracking signals, explicitly model students' learning states, depict the dynamic evolution of learning states within a continuous time window, dynamically adjust the fusion weights of EEG and eye-tracking modalities based on learning states, and combine explicit teaching behavior data to achieve joint evaluation of teaching effectiveness has become a pressing technical problem in this field. Summary of the Invention

[0010] To address the shortcomings of existing ideological and political education classroom teaching evaluation methods, such as the lack of explicit modeling of students' learning status, difficulty in depicting the dynamic changes in learning status within a continuous time window, and lack of dynamic adjustment of the fusion process of EEG and eye-tracking modes based on learning status, this invention proposes a multimodal ideological and political education teaching evaluation system that integrates EEG and eye-tracking signals.

[0011] To achieve the above objectives, the present invention adopts the following technical solution:

[0012] A multimodal ideological and political education teaching evaluation system that integrates EEG signals and eye movement signals includes a data acquisition module, a feature construction module, a basic cross-modal representation module, a learning state modeling module, a state-driven dynamic fusion module, and a teaching evaluation module.

[0013] The data acquisition module is used to acquire the EEG and eye movement signals of students during ideological and political education classes, and to perform synchronous acquisition, time window division, and preprocessing of the EEG and eye movement signals based on a unified time reference.

[0014] The feature construction module is used to extract features from the EEG signal and the eye movement signal to construct EEG feature vectors and eye movement feature vectors under the corresponding time window;

[0015] The basic cross-modal representation module is used to map the EEG feature vector and the eye-tracking feature vector to a unified feature space to generate unified EEG feature vector and unified eye-tracking feature vector, and to construct a cross-modal correlation matrix based on the unified EEG feature vector and the unified eye-tracking feature vector. The module then performs correlation enhancement on the unified EEG feature vector and the unified eye-tracking feature vector according to the cross-modal correlation matrix to generate cross-modal correlation feature vector.

[0016] The learning state modeling module is used to take the cross-modal correlation feature vector corresponding to the current time window as the state input, and combine it with the learning state vector corresponding to the previous time window to generate the learning state vector corresponding to the current time window through a neural network gating mechanism; the learning state modeling module is also used to generate cognitive input state vector, emotional identification state vector and behavioral participation state vector based on the learning state vector.

[0017] The state-driven dynamic fusion module is used to generate a fusion mode control vector based on the cognitive engagement state vector, the emotional identification state vector, and the behavioral participation state vector, and to generate EEG modal weight response values ​​and eye-tracking modal weight response values ​​based on the fusion mode control vector; the state-driven dynamic fusion module is also used to normalize the EEG modal weight response values ​​and the eye-tracking modal weight response values ​​to obtain EEG modal dynamic weight coefficients and eye-tracking modal dynamic weight coefficients, so that the EEG modal dynamic weight coefficients and the eye-tracking modal dynamic weight coefficients satisfy the normalization constraints;

[0018] The state-driven dynamic fusion module is further used to perform state-driven weighted fusion of the unified EEG feature vector and the unified eye-tracking feature vector based on the dynamic weight coefficients of the EEG modality and the dynamic weight coefficients of the eye-tracking modality, so as to generate a state-driven fusion feature vector, which is used as the input of the teaching evaluation module.

[0019] The teaching evaluation module is used to perform joint feature evaluation based on the state-driven fusion feature vector, the learning state vector, and the explicit evaluation feature vector constructed by normalizing explicit teaching behavior data. It also generates teaching effectiveness scores under corresponding time windows and teaching state trends under continuous time windows through the teaching evaluation network.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] (1) This invention constructs a basic cross-modal representation module, maps EEG feature vectors and eye movement feature vectors to a unified feature space, and constructs a cross-modal correlation matrix based on the unified EEG feature vectors and unified eye movement feature vectors, thereby enhancing the correlation expression ability between different modal data and improving the effectiveness of multimodal feature representation;

[0022] (2) This invention constructs a learning state modeling module, which generates the learning state vector corresponding to the current time window based on the cross-modal correlation feature vector and the learning state vector corresponding to the previous time window using the neural network gating mechanism, and further generates cognitive input state vector, emotional identification state vector and behavioral participation state vector, thereby realizing explicit modeling of students' learning state and improving the interpretability of teaching evaluation results and the ability to express teaching semantics.

[0023] (3) The present invention can depict the dynamic evolution trend of students’ learning status in the classroom teaching process through the learning status update mechanism under continuous time window, and improve the ability to continuously analyze the changes in the classroom learning process.

[0024] (4) This invention constructs a state-driven dynamic fusion module, uses cognitive input state vector, emotional identification state vector and behavioral participation state vector to generate fusion mode control vector, and further generates EEG modal dynamic weight coefficient and eye movement modal dynamic weight coefficient, performs state-driven weighted fusion of EEG unified feature vector and eye movement unified feature vector, so as to dynamically adjust the information contribution of EEG modality and eye movement modality according to the student's current learning state, improve the stability and robustness of multimodal fusion results in complex classroom scenarios. Since the EEG modal dynamic weight coefficient and eye movement modal dynamic weight coefficient are jointly generated by cognitive input state vector, emotional identification state vector and behavioral participation state vector, it can avoid evaluation bias caused by fixed weight fusion in scenarios of cognitive load change, attention shift or behavioral participation fluctuation.

[0025] (5) This invention uses a teaching evaluation module to jointly evaluate the state-driven fusion feature vector, the learning state vector, and the explicit evaluation feature vector constructed by normalizing explicit teaching behavior data. It can simultaneously utilize implicit learning state information and explicit teaching behavior information to generate teaching effect scores and teaching state trends, thereby improving the comprehensiveness, continuity, and objectivity of ideological and political classroom teaching evaluation results. Attached Figure Description

[0026] Figure 1 This is a flowchart of the teaching assessment system according to Embodiment 1 of the present invention.

[0027] Figure 2 This is a flowchart of data preprocessing and feature construction in Embodiment 1 of the present invention.

[0028] Figure 3 This is a flowchart of the cross-modal correlation feature vector construction process in Embodiment 1 of the present invention.

[0029] Figure 4 This is a flowchart of the learning state modeling process in Embodiment 1 of the present invention.

[0030] Figure 5 This is a flowchart of the state-driven dynamic fusion process of Embodiment 1 of the present invention.

[0031] Figure 6 This is a flowchart of the teaching assessment and trend analysis in Embodiment 2 of the present invention. Detailed Implementation

[0032] Example 1:

[0033] refer to Figure 1 This embodiment provides a specific implementation of a multimodal ideological and political education teaching evaluation system that integrates electroencephalogram (EEG) signals and eye movement signals. It is applied in ideological and political education classroom teaching scenarios to dynamically evaluate students' classroom learning status and teaching effectiveness.

[0034] S1. Acquisition and preprocessing of EEG and eye movement signals;

[0035] This embodiment is applied to ideological and political education classroom teaching scenarios. During teacher lectures, classroom interactions, and classroom tests, it synchronously collects students' electroencephalogram (EEG) and eye-tracking signals during classroom learning to obtain multimodal physiological and behavioral data that reflects changes in students' classroom learning status. In this embodiment, the data acquisition module includes an EEG acquisition unit, an eye-tracking acquisition unit, a time synchronization unit, a windowing unit, and a preprocessing unit.

[0036] Among them, EEG signals are acquired through EEG acquisition equipment, and eye movement signals are acquired through eye movement tracking equipment; EEG acquisition equipment is used to record students' EEG activity information during classroom learning, and eye movement tracking equipment is used to record students' eye movement information, fixation behavior information, and salivation behavior information during classroom learning.

[0037] In one implementation, the EEG acquisition device uses a 32-channel EEG acquisition system with an EEG signal sampling frequency of 256Hz; the eye-tracking device has a sampling frequency of 120Hz to meet the needs of temporal behavior analysis during the continuous changes in classroom learning status.

[0038] To ensure the temporal correspondence between different modal data, this embodiment synchronously collects EEG signals and eye movement signals based on a unified time reference, and divides the time window according to a fixed time length to construct classroom learning process data under a continuous time window.

[0039] In one implementation, the time window length is set to 2 seconds, and a 50% time overlap ratio is set between adjacent time windows to improve the temporal continuity of the continuous learning state change process.

[0040] The acquired EEG signals were preprocessed, including power frequency interference removal, bandpass filtering, and artifact removal. The power frequency interference frequency was set to 50Hz; the bandpass filtering frequency range was set to 0.5Hz to 45Hz to retain EEG activity information related to classroom learning; artifact removal included electrooculography artifact removal, electromyography interference removal, and removal of abnormal high-amplitude signals.

[0041] In one implementation, independent component analysis is used to separate and remove electrooculogram artifacts in the electroencephalogram (EEG) signal to improve the quality of the EEG signal.

[0042] The eye movement signals are preprocessed, including gaze drift correction, abnormal fixation point removal, missing fixation point interpolation compensation, and saccade trajectory smoothing.

[0043] The criteria for determining abnormal gaze points include gaze coordinates exceeding the screen area and abrupt changes in distance between consecutive sampling points exceeding a preset threshold. For missing gaze point data, linear interpolation is used for compensation. For saccade trajectory data, a moving average filter is used for trajectory smoothing to improve the stability of eye movement behavior data.

[0044] After preprocessing, the corresponding EEG signal data and eye movement signal data under the continuous time window are obtained and used as input for the subsequent feature construction process.

[0045] S2. Construction of EEG feature vectors and eye-tracking feature vectors;

[0046] Feature extraction is performed on the preprocessed EEG and eye movement signals from step S1 to construct EEG feature vectors and eye movement feature vectors for the corresponding time windows. The data preprocessing and feature construction process is as follows: Figure 2 As shown

[0047] For electroencephalogram (EEG) signals, this embodiment first divides the EEG signals into frequency bands according to different frequency ranges; in one embodiment, the EEG signals are divided into... Frequency band (0.5Hz~4Hz) Frequency band (4Hz~8Hz) Frequency band (8Hz~13Hz) Frequency band (13Hz~30Hz) and Frequency band (30Hz~45Hz). Among them, Frequency bands are typically associated with changes in cognitive load during students' classroom learning process. Frequency bands are often associated with changes in classroom attention. Frequency bands are typically related to changes in classroom learning participation; therefore, this embodiment selects... frequency band frequency band and The frequency band is used as a target frequency band related to classroom learning status.

[0048] Subsequently, frequency domain analysis was performed on the EEG signals within the corresponding time window.

[0049] In one implementation, a fast Fourier transform is used to convert the time-domain EEG signal to the frequency domain, and the power spectral density characteristics at the corresponding frequency are calculated. The calculation method is as follows: ,in, This represents the power spectral density value corresponding to the frequency. Indicates frequency point, Indicates the time window number The amplitude of the EEG signal corresponding to each sampling point Indicates the sampling point number. This indicates the number of sampling points within the time window. Represents the imaginary unit. Represents the natural constant.

[0050] Subsequently, calculate separately frequency band frequency band and The average spectral energy corresponding to the frequency band is obtained, and features are spliced ​​according to the EEG channel dimension to generate EEG feature vectors under the corresponding time window.

[0051] In one implementation, the number of EEG acquisition channels is set to 32, and the number of target frequency bands is set to 3. Therefore, the EEG feature dimension generated under the corresponding time window is 96-dimensional. The EEG feature vector generated under the corresponding time window is represented as: ,in Indicates the first EEG feature vectors corresponding to each time window.

[0052] For eye-tracking signals, this embodiment constructs gaze behavior features based on students' fixation and saccade behaviors during classroom learning. Fixation behavior features include fixation area distribution characteristics and fixation duration characteristics; saccade behavior features include saccade direction change characteristics and saccade trajectory change characteristics.

[0053] In one implementation, the classroom teaching area is divided into a teacher lecturing area, a courseware display area, and a classroom interaction area. The percentage of time students spend focusing on each area within a given time window is statistically analyzed to construct a gaze area distribution characteristic. The calculation method is as follows: .in, Indicates the first Within the time window, students are in the first The percentage of eye contact in each classroom area This represents the classroom area number to be calculated. Indicates the first Within the time window, students are in the first The cumulative gaze duration in each classroom area Indicates the first The first time window The cumulative gaze duration in each classroom area Indicates to An index for summing up each classroom area. This indicates the total number of classroom areas.

[0054] In one implementation, the total number of classroom areas is set to three, corresponding to the teacher's lecture area, the courseware display area, and the classroom interaction area, respectively.

[0055] Simultaneously, the distance of gaze movement between consecutive fixation points within the corresponding time window is statistically analyzed to construct the characteristics of saccade trajectory changes. The calculation method is as follows: .in, Indicates the first The characteristics of the scanning trajectory changes corresponding to each time window. Indicates the fixation point number. and They represent the first The x and y coordinates corresponding to each fixation point and They represent the first The x and y coordinates corresponding to each fixation point This indicates the number of fixations within the corresponding time window.

[0056] Furthermore, the angle of change in saccade direction between consecutive fixation points is statistically analyzed to construct saccade direction change characteristics. The calculation method is as follows: ,in, Indicates the first Features of scanning direction changes corresponding to each time window and They represent the first The x and y coordinates corresponding to each fixation point This indicates the number of fixations within the corresponding time window. This represents the arctangent function.

[0057] Subsequently, the features of gaze region distribution, gaze duration, saccade direction change, and saccade trajectory change are concatenated to generate an eye movement feature vector for the corresponding time window.

[0058] In one implementation, the eye-tracking feature dimension is set to 16 dimensions, and the eye-tracking feature vector generated under the corresponding time window is represented as follows: ,in, Indicates the first The eye-tracking feature vectors corresponding to each time window.

[0059] After constructing the EEG feature vector and eye movement feature vector, the EEG feature vector and eye movement feature vector under the corresponding time window are time-aligned according to a unified time window and used as input for the subsequent cross-modal association feature vector construction process.

[0060] S3, Construction of cross-modal correlation feature vectors;

[0061] The EEG feature vectors and eye-tracking feature vectors obtained in step S2 are subjected to unified feature space mapping and cross-modal correlation modeling to generate cross-modal correlation feature vectors. The process of constructing cross-modal correlation feature vectors is as follows: Figure 3 As shown.

[0062] Since EEG feature vectors and eye movement feature vectors originate from different modal data, their feature dimensions and numerical distributions differ. Therefore, this embodiment first performs a unified feature space mapping on EEG feature vectors and eye movement feature vectors.

[0063] In one implementation, feature dimension mapping is performed on the EEG feature vector and the eye movement feature vector using linear mapping methods, and the calculation methods are as follows:

[0064]

[0065]

[0066] in, Indicates the first EEG feature vectors corresponding to each time window Indicates the first Eye-tracking feature vectors corresponding to each time window Indicates the first Each time window corresponds to a unified feature vector of EEG. Indicates the first Each time window corresponds to a unified eye-tracking feature vector. This represents the feature mapping matrix corresponding to the EEG modalities. This represents the feature mapping matrix corresponding to the eye-tracking modality. The bias vector representing the EEG modality. This represents the bias vector of the eye-tracking mode.

[0067] In one implementation, EEG feature vectors and eye-tracking feature vectors are uniformly mapped to a 128-dimensional feature space. Thus, the unified EEG feature vectors and unified eye-tracking feature vectors have the same dimension, so that subsequent cross-weighted and state-driven weighted fusion can be performed based on cross-modal correlation matrices.

[0068] After completing the unified feature space mapping, cross-modal correlation modeling is performed on the unified EEG feature vector and the unified eye-tracking feature vector to capture the correlation between different modalities.

[0069] In one implementation, a cross-modal correlation matrix is ​​constructed using a feature correlation calculation method, which is as follows: ,in, Indicates the first The cross-modal correlation matrix corresponding to each time window This indicates the transpose operation.

[0070] Subsequently, the EEG unified feature vector and the OMG unified feature vector are correlated and enhanced based on the cross-modal correlation matrix to generate a cross-modal correlated feature vector, which is calculated as follows: ,in, Indicates the first Cross-modal correlation feature vectors corresponding to each time window This represents the eye-to-EEG direction correlation response obtained by weighting the unified eye-tracking feature vector based on the cross-modal correlation matrix. This represents the EEG-to-eye movement direction correlation response obtained by weighting the unified EEG feature vector based on the transposed cross-modal correlation matrix. The cross-modal correlation feature vector is used to simultaneously characterize the bidirectional correlation response between the unified EEG feature vector and the unified eye movement feature vector.

[0071] After completing the construction of the cross-modal association feature representation, the cross-modal association feature vector under the corresponding time window is used as the input for the subsequent learning state modeling process.

[0072] S4. Learning state modeling and state evolution analysis;

[0073] The cross-modal correlation feature vectors generated in step S3 are used to model the learning state, so as to construct the learning state change process of students under continuous time windows. The learning state modeling process is as follows: Figure 4 As shown.

[0074] Because classroom learning states exhibit continuous and dynamic changes, cross-modal correlation feature vectors based solely on a single time window are insufficient to reflect the continuous evolution of students' learning states. Therefore, this embodiment uses a neural network gating mechanism to model the state evolution of learning states within continuous time windows, establishing a dynamic update relationship between the cross-modal correlation feature vector of the current time window and the learning state vector of the previous time window.

[0075] In one implementation, the cross-modal correlation feature vector generated in step S3 is set to 128 dimensions, i.e., the first... The cross-modal association feature vectors corresponding to each time window are: The learning state vector dimension is set to 64 dimensions, that is, the 64th dimension. Each time window corresponds to .

[0076] First, the state representation construction unit generates candidate learning state vectors based on the cross-modal correlation feature vectors corresponding to the current time window and the learning state vectors corresponding to the previous time window. The calculation method is as follows: ,in, Indicates the first Candidate learning state vectors corresponding to each time window Indicates the first Cross-modal correlation feature vectors corresponding to each time window This represents the learning state vector corresponding to the previous time window. The input mapping matrix represents the candidate state. Represents the candidate state history mapping matrix. This represents the candidate state bias vector. This represents the hyperbolic tangent activation function.

[0077] For the initial time window, the learning state vector corresponding to the previous time window is initialized to a zero vector, that is: .

[0078] Subsequently, the state evolution unit constructs a state update gate based on the cross-modal correlation feature vector corresponding to the current time window and the learned state vector corresponding to the previous time window, using a neural network gating mechanism. The calculation method is as follows: ,in, Indicates the first The state update gate vector corresponding to each time window This indicates updating the gate input mapping matrix. This indicates updating the gate history mapping matrix. This indicates updating the gate bias vector. This represents the Sigmoid activation function.

[0079] In one implementation, the state update gate vector dimension is set to 64 dimensions, i.e. , used to correspond element-wise with the learning state vector.

[0080] The state update gate controls the fusion ratio between the learned state vector from the previous time window and the current candidate learned state vector. When the state update gate value is small, the current learned state retains more historical state information from the previous time window; when the state update gate value is large, the current learned state incorporates more candidate learned state information corresponding to the current time window.

[0081] Subsequently, the state evolution unit generates the learning state vector corresponding to the current time window based on the state update gate, the candidate learning state vector, and the learning state vector corresponding to the previous time window. The calculation method is as follows: ,in, This represents the state update gate vector. Vectors consisting entirely of 1s of the same dimension Indicates the first The learning state vector corresponding to each time window This represents Hadamard element-wise multiplication.

[0082] After completing the learning state vector update, the state semantic mapping unit generates a cognitive engagement state vector, an emotional identification state vector, and a behavioral participation state vector based on the learning state vector. The calculation methods for these vectors are as follows:

[0083]

[0084]

[0085]

[0086] in, Indicates the first The cognitive input state vector corresponding to each time window Indicates the first The emotional identification state vector corresponding to each time window Indicates the first The behavior participation state vector corresponding to each time window , as well as These represent the cognitive engagement state mapping matrix, the emotional identification state mapping matrix, and the behavioral participation state mapping matrix, respectively. , as well as These represent the cognitive engagement state mapping bias vector, the emotional identification state mapping bias vector, and the behavioral participation state mapping bias vector, respectively. The cognitive engagement state vector characterizes the student's level of cognitive participation in the classroom learning process, the emotional identification state vector characterizes the student's emotional acceptance of the classroom teaching content, and the behavioral participation state vector characterizes the student's level of behavioral activity in the classroom learning process. These three vectors together constitute the semantic representation of the learning state within the current time window, serving as input to the generation of the fusion mode control vector in the subsequent state-driven dynamic fusion process. This allows the fusion weights of the subsequent EEG modality and eye-tracking modality to be adjusted according to the semantic representation of the current learning state.

[0087] In one implementation, the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are all set to 16 dimensions, which are used to represent the semantic subspaces related to cognition, emotion, and behavioral participation in the learning state vectors, respectively.

[0088] After completing the learning state modeling, the cognitive input state vector, emotional identification state vector, and behavioral participation state vector are used as inputs to the state-driven dynamic fusion process, and the learning state vector is used as inputs to the subsequent teaching evaluation process.

[0089] S5, State-driven dynamic fusion;

[0090] After completing the learning state modeling, the system obtains the first... The cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector correspond to each time window. Since students' attention to teaching content varies across different classroom learning states, the information contribution of EEG and eye-tracking modalities also changes at different learning stages. Therefore, this embodiment does not use fixed fusion weights to process multimodal information. Instead, based on the aforementioned cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector, a state control network dynamically generates a fusion mode control vector, and further generates dynamic fusion weights corresponding to the EEG and eye-tracking modalities. This improves the stability and adaptability of teaching evaluation results in complex classroom scenarios. The state-driven dynamic fusion process is as follows: Figure 5 As shown.

[0091] In one implementation, the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are all set to 16 dimensions, i.e.: , , ,in, Indicates the first The cognitive input state vector corresponding to each time window Indicates the first The emotional identification state vector corresponding to each time window Indicates the first The behavior participation state vector corresponding to each time window.

[0092] First, the state control unit generates a fusion pattern control vector based on the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector through the state control network. The calculation method is as follows: ,in, Indicates the first The fusion mode control vector corresponding to each time window This represents the cognitive input state mapping matrix. Represents the emotional identification state mapping matrix. The behavior-participation state mapping matrix represents the state mapping matrix. This represents the fusion mode control bias vector. This represents the hyperbolic tangent activation function.

[0093] In one implementation, the fusion mode control vector dimension is set to 32 dimensions, i.e. , , as well as All dimensions are 32×16. The dimension is 32.

[0094] The fusion mode control vector is used to characterize the fusion bias relationship between the EEG mode and the eye movement mode under the current time window, and to adjust the generation process of the subsequent EEG mode weight response value and eye movement mode weight response value.

[0095] Subsequently, the weighted response generation unit generates EEG modal weighted response values ​​and eye-tracking modal weighted response values ​​based on the fusion mode control vector, and their calculation methods are as follows:

[0096]

[0097]

[0098] in, Indicates the first The EEG modality weighted response values ​​corresponding to each time window Indicates the first The eye-tracking modal weighted response values ​​corresponding to each time window and These represent the EEG modal weighted response matrix and the eye-tracking modal weighted response matrix, respectively. and These represent the weighted response bias parameters of the EEG modality and the weighted response bias parameters of the eye-tracking modality, respectively.

[0099] In one implementation, and All are scalars. and All are scalars. and All dimensions are 1×32.

[0100] To ensure that the dynamic weight coefficients corresponding to the EEG and eye-tracking modalities satisfy the normalization constraint, the weight normalization unit performs Softmax normalization on the weight response values ​​of the EEG and eye-tracking modalities. The calculation methods are as follows:

[0101]

[0102]

[0103] in, Indicates the first The EEG dynamic weighting coefficients corresponding to each time window Indicates the first The eye-tracking dynamic weighting coefficients corresponding to each time window This represents the exponential mapping function.

[0104] The EEG dynamic weighting coefficients and the eye-tracking dynamic weighting coefficients satisfy the following normalization constraints: , ,and .

[0105] After dynamic weight generation is completed, the dynamic fusion unit performs state-driven dynamic fusion of the unified EEG feature vector and the unified eye-tracking feature vector in the unified feature space based on the EEG dynamic weight coefficient and the eye-tracking dynamic weight coefficient to generate a state-driven fused feature vector. The calculation method is as follows: ,in, Indicates the first The state-driven fusion feature vector corresponds to each time window. Indicates the first The unified EEG feature vector corresponding to each time window Indicates the first The eye-tracking unified feature vector corresponding to each time window.

[0106] The EEG dynamic weighting coefficients and the eye-tracking dynamic weighting coefficients change with the changes in the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector. This allows the state-driven fusion feature vector to adapt to the differences in information contribution between the EEG and eye-tracking modalities under different classroom learning states. Therefore, when the current learning state reflects changes in cognitive load, the system can increase the contribution of the EEG modality in the fusion process; when the current learning state reflects changes in visual attention or behavioral participation, the system can increase the contribution of the eye-tracking modality in the fusion process.

[0107] In one implementation, both the unified EEG feature vector and the unified eye-tracking feature vector are 128-dimensional, i.e. , Therefore, the state-driven fusion feature vector is 128-dimensional, i.e. .

[0108] Through the above methods, the system can dynamically adjust the contribution of EEG modality and eye-tracking modality in the fusion process based on changes in cognitive engagement, emotional identification, and behavioral participation, thereby improving the state adaptability and assessment stability in the classroom teaching evaluation process.

[0109] S6. Teaching evaluation and analysis of teaching status trends;

[0110] After completing state-driven dynamic fusion, the system obtains the first... The state-driven fusion feature vector corresponds to each time window. Since relying solely on EEG and eye-tracking signals is insufficient to fully reflect classroom teaching effectiveness, this embodiment further incorporates explicit teaching behavior data from the classroom teaching process to jointly evaluate classroom teaching effectiveness, thereby improving the objectivity and completeness of the evaluation results. Explicit teaching behavior data includes student attendance rates, homework completion status, classroom test scores, and the number of classroom interactions. This explicit teaching behavior data reflects the outward learning behavior performance during the classroom teaching process.

[0111] Since attendance, homework completion, classroom test scores, and classroom interaction frequency can be statistically analyzed for different classroom segments, teaching stages, or preset statistical periods, this embodiment normalizes the explicit teaching behavior data according to the classroom segment, teaching stage, or preset statistical period corresponding to the current time window, and maps it into an explicit evaluation feature vector under the corresponding time window.

[0112] The calculation method for the attendance rate evaluation index is as follows: ,in Indicates the first The sign-in rate evaluation index corresponds to each time window. Indicates the first The number of students actually signing in within each time window This indicates the number of students who should sign in.

[0113] The calculation method for the evaluation indicators of assignment completion is as follows: ,in, Indicates the first Evaluation indicators for task completion corresponding to each time window Indicates the first The number of students who complete the assignment within a time window This indicates the number of students who should submit assignments.

[0114] The calculation method for classroom test performance evaluation indicators is as follows: ,in, Indicates the first Evaluation indicators for classroom test scores corresponding to each time window Indicates the first The average score of classroom tests corresponding to each time window This indicates the full score for the classroom test.

[0115] The calculation method for the evaluation index of classroom interaction frequency is as follows: ,in, Indicates the first Evaluation indicators for the number of classroom interactions corresponding to each time window Indicates the first Number of classroom interactions within a time window This indicates the maximum number of interactions preset or the maximum number of interactions within a statistical period.

[0116] In one implementation, if the first If 50 students are expected to sign in during a given time window, and 48 students actually sign in, then the sign-in rate evaluation index is... =48 / 50=0.96; If the full score for a classroom test is 100 points, and the average score for a classroom test within the corresponding time window is 82 points, then the classroom test performance evaluation index is... =82 / 100=0.82.

[0117] Subsequently, the explicit indicator construction unit constructs explicit evaluation feature vectors for the corresponding time window based on the normalized explicit teaching behavior data. The calculation method is as follows: ,in, Indicates the first The explicit evaluation feature vector corresponding to each time window This represents a vector-level concatenation operation.

[0118] In one implementation, the explicit evaluation feature vector is set to 4 dimensions, i.e. .

[0119] Subsequently, the joint assessment unit performs vector-level feature concatenation on the state-driven fusion feature vector, the learning state vector, and the explicit evaluation feature vector to generate a joint teaching assessment feature vector, which is calculated as follows: ,in, Indicates the first The joint feature vector of teaching evaluation corresponding to each time window Indicates the first The state-driven fusion feature vector corresponds to each time window. Indicates the first The learning state vector corresponding to each time window Indicates the first The explicit evaluation feature vector corresponding to each time window.

[0120] Subsequently, the joint evaluation unit, based on the joint feature vector of teaching evaluation, performs a nonlinear mapping on the joint feature vector of teaching evaluation through a teaching evaluation network to generate a teaching effectiveness score for the corresponding time window. The calculation method is as follows: ,in, Indicates the first The teaching effectiveness score corresponds to each time window. This represents the weight matrix corresponding to the teaching evaluation network. Indicates the bias parameter. This represents the Sigmoid activation function.

[0121] In one implementation, teaching effectiveness scoring The value range is from 0 to 1, when The closer the value is to 1, the better the students' classroom learning status and the better the teaching effect within the corresponding time window.

[0122] To further reflect the dynamic changes in the classroom teaching process, the dynamic assessment output unit performs time series trend analysis on the teaching effectiveness scores within a continuous time window to generate teaching status trends. The calculation method is as follows: ,in, Indicates the first The teaching status trend corresponding to each time window This indicates the trend of teaching status corresponding to the previous time window. This indicates the teaching effectiveness score corresponding to the current time window. Represents the trend smoothing coefficient, and satisfies .

[0123] The teaching status trend is used to smooth the teaching effectiveness scores under continuous time windows, so as to reduce the impact of fluctuations in EEG signals, eye movement signals or explicit teaching behavior data on the teaching evaluation results within a single time window.

[0124] In one implementation, the trend smoothing coefficient Set to 0.8. For the initial time window, the teaching status trend is initialized to the teaching effectiveness score of the current time window, i.e.: ,in, This indicates the teaching status trend corresponding to the initial time window. This indicates the teaching evaluation effect corresponding to the initial time window.

[0125] When the trend of teaching status continues to rise over a continuous time window, it indicates that students' classroom learning status is gradually improving; when the trend of teaching status continues to decline, it indicates that students' classroom learning status is gradually weakening; when the trend of teaching status fluctuates little, it indicates that the overall classroom teaching status is relatively stable.

[0126] Through the above methods, the system can not only evaluate the effectiveness of classroom teaching within a single time window, but also continuously analyze the dynamic changes in students' learning status during the classroom teaching process, thereby providing a reference for optimizing classroom teaching and adjusting teaching strategies.

[0127] Before system deployment, the feature mapping matrix, state modeling parameters, state semantic mapping parameters, weight response matrix, and teaching evaluation network parameters can be trained offline based on historical ideological and political education classroom sample data. This historical ideological and political education classroom sample data includes EEG signals, eye-tracking signals, explicit teaching behavior data, and corresponding human evaluation results, classroom test results, or expert scoring results. After training, the obtained model parameters are embedded into the system for real-time evaluation during ideological and political education classroom teaching.

[0128] It should be noted that the above-mentioned EEG feature dimensions, eye movement feature dimensions, unified feature space dimensions, learning state vector dimensions, fusion mode control vector dimensions, and state semantic vector dimensions are only exemplary settings. Those skilled in the art can adjust the above dimensions according to the number of EEG acquisition channels, the number of eye movement sampling features, the number of students in the classroom, and the model complexity, without affecting the implementation of multimodal association modeling, learning state evolution modeling, and state-driven dynamic fusion described in this embodiment.

[0129] Example 2:

[0130] This embodiment, based on the multimodal ideological and political education teaching evaluation system described in Embodiment 1, further illustrates the application of the teaching evaluation results in classroom feedback and teaching adjustments. This application method generates corresponding classroom teaching feedback information and teaching adjustment suggestions based on changes in students' learning status, teaching effectiveness scores, and teaching status trends during the ideological and political education classroom teaching process. The flowchart of its teaching evaluation and trend analysis is as follows: Figure 6 As shown.

[0131] During ideological and political education classes, the system synchronously collects EEG and eye movement signals as described in Example 1, and performs time window segmentation, feature construction, cross-modal correlation modeling, learning state modeling, state-driven dynamic fusion, and teaching evaluation processing on the EEG and eye movement signals to obtain the first... Cognitive engagement state vector corresponding to each time window Emotional identification state vector Behavior-related state vector Teaching effectiveness evaluation and trends in teaching status .

[0132] In one implementation, the time window length is set to 2 seconds, and a 50% time overlap ratio is set between adjacent time windows; the cross-modal association feature vector dimension is set to 128 dimensions, the learning state vector dimension is set to 64 dimensions, and the dimensions of the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are all set to 16 dimensions; the teaching effect score ranges from 0 to 1, and the trend smoothing coefficient is set to 0.8.

[0133] The system scores teaching effectiveness based on a continuous time window. and trends in teaching status Assess the overall changes in the classroom teaching status. Specifically, when the teaching status trend... When the trend increases over multiple consecutive time windows, the current classroom teaching status is determined to be in an enhancing trend; when the teaching status trend... When the teaching status decreases over multiple consecutive time windows, it is determined that the current classroom teaching status is on a downward trend; when the teaching status trend... When the change is below a preset threshold within multiple consecutive time windows, the current classroom teaching status is determined to be stable.

[0134] In one implementation, the system uses five consecutive time windows as the trend judgment period. If the average increment of the teaching status trend within five consecutive time windows is greater than 0.05, the classroom teaching status is determined to be in an enhancing trend; if the average increment of the teaching status trend within five consecutive time windows is less than -0.05, the classroom teaching status is determined to be in a declining trend; if the average change in the teaching status trend within five consecutive time windows is less than 0.03, the overall classroom teaching status is determined to be stable.

[0135] Furthermore, the system is based on the cognitive input state vector Emotional identification state vector and the behavior participation state vector The system analyzes the reasons for the decline in classroom learning engagement. In one implementation, the system calculates the cognitive engagement state vector. Emotional identification state vector and the behavior participation state vector The mean is used to characterize the intensity of students' cognitive, emotional, and behavioral states within the corresponding time window.

[0136] When the intensity of cognitive engagement is lower than the preset cognitive threshold, the system determines that the student's understanding of the current teaching content is insufficient and generates teaching feedback suggestions such as increasing the explanation of knowledge points, reducing the density of concepts, or increasing case studies. When the intensity of emotional identification is lower than the preset emotional threshold, the system determines that the student's emotional acceptance of the current teaching content is insufficient and generates teaching feedback suggestions such as enhancing contextualized explanations, incorporating real-world cases, or increasing value-guided expression. When the intensity of behavioral participation is lower than the preset behavioral threshold, the system determines that the student's classroom participation is insufficient and generates teaching feedback suggestions such as increasing the frequency of questioning, setting up classroom interactive sessions, or increasing group discussions.

[0137] In one implementation, the preset cognitive threshold, preset emotional threshold, and preset behavioral threshold are all set to 0.6. When the intensity of a corresponding state is below 0.6, the system generates teaching adjustment suggestions for that dimension. When the intensity of multiple states is below 0.6 simultaneously, the system prioritizes generating primary feedback suggestions based on the state dimension with the largest decrease, and uses other state dimensions as supplementary feedback.

[0138] Furthermore, the system also incorporates EEG dynamic weighting coefficients. With eye-tracking dynamic weighting coefficient Analyze the contribution of different modalities to the current assessment results. When the EEG dynamic weighting coefficient... Higher and teaching status trend When the weighting coefficient of eye-tracking decreases, the system determines that the current teaching problem is mainly related to changes in students' cognitive input; when the eye-tracking dynamic weighting coefficient decreases... Higher and teaching status trend When a decline occurs, the system determines that the current teaching problem is primarily related to changes in student behavior, participation, or attention distribution. Therefore, the system can help teachers distinguish whether the decline in classroom learning is due to insufficient cognitive understanding or to distraction or insufficient classroom participation.

[0139] In one implementation, when the EEG dynamic weighting coefficient Greater than 0.65 and teaching status trend When the cognitive load of the current content is high, it is recommended to slow down the pace of explanation or provide supplementary examples; when the eye-tracking dynamic weight coefficient... Greater than 0.65 and teaching status trend When the visual attention stability of students decreases continuously, the system generates feedback information such as "Students' visual attention stability has decreased, it is recommended to increase classroom interaction or adjust the presentation of courseware"; when the EEG dynamic weighting coefficient... With eye-tracking dynamic weighting coefficient The difference is less than 0.1 and the teaching status trend When the situation is stable, the system generates feedback information stating that "the current classroom situation is generally stable and the current teaching pace can be maintained."

[0140] Subsequently, the system generates classroom teaching feedback information based on the aforementioned state analysis results and displays this feedback information to the teacher in a visual manner. The classroom teaching feedback information includes the teaching effectiveness score for the current time window, teaching status trends, changes in cognitive engagement, changes in emotional identification, changes in behavioral participation, and corresponding teaching adjustment suggestions.

[0141] In one implementation, the teacher's interface includes a teaching effectiveness rating curve, a teaching status trend curve, three types of learning status change curves, and teaching adjustment suggestion text. When the teaching effectiveness rating falls below a preset rating threshold, the system outputs a classroom status warning to assist teachers in adjusting their teaching strategies in a timely manner. In one implementation, the preset rating threshold is set to 0.6; when the teaching effectiveness rating is below 0.6 for three consecutive time windows, the system generates a classroom status warning.

[0142] Furthermore, the system stores teaching effectiveness scores, teaching status trends, and teaching adjustment suggestions during the classroom teaching process, forming a historical record of classroom teaching status for post-class teaching quality analysis and teaching strategy optimization. By statistically analyzing the changes in teaching status across multiple classroom segments, the system can help teachers identify teaching segments that are prone to causing a decline in student cognitive engagement, reduced emotional identification, or insufficient behavioral participation, thus providing a reference for subsequent teaching content design and classroom organization optimization.

[0143] Through the above methods, this embodiment can further realize dynamic feedback, status warning, and teaching strategy adjustment suggestions in the classroom teaching process based on the multimodal teaching evaluation results described in Embodiment 1, thereby improving the real-time nature, continuity, and teaching support capabilities of ideological and political classroom teaching evaluation.

[0144] Example 3:

[0145] This embodiment, based on the multimodal ideological and political education teaching evaluation system described in Embodiment 1, further illustrates the application of the system in group teaching status evaluation in multi-student classroom scenarios. This application method is used to dynamically analyze the learning status of student groups, classroom areas, and the overall teaching status of the class during the ideological and political education classroom teaching process.

[0146] In actual ideological and political education classroom teaching, the effectiveness of classroom teaching is not only related to the learning status of individual students, but also closely related to the overall learning atmosphere of the class, the participation of students in different areas of the classroom, and the trend of changes in the status of the student group. Therefore, this embodiment, based on the learning status modeling mechanism, state-driven dynamic fusion mechanism, and teaching evaluation mechanism in Embodiment 1, synchronously analyzes the EEG signals and eye movement signals of multiple students in the classroom and generates corresponding group teaching status evaluation results.

[0147] Specifically, during classroom teaching, the system simultaneously collects EEG and eye-tracking signals from multiple students. Based on a unified time reference, it performs synchronous acquisition, time window division, feature construction, cross-modal association modeling, learning state modeling, state-driven dynamic fusion, and teaching effectiveness scoring on the EEG and eye-tracking signals from multiple students to obtain the first... The first time window Learning state vector for each student Cognitive input state vector Emotional identification state vector Behavior-related state vector and teaching effectiveness evaluation .

[0148] In one implementation, the number of students in the classroom is set to... The time window length is set to 2 seconds, and the time overlap ratio between adjacent time windows is set to 50%. The learning state vector for each student is set to 64 dimensions, and the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are all set to 16 dimensions. The teaching effectiveness score ranges from 0 to 1.

[0149] First, the system calculates the overall class teaching effectiveness score based on the teaching effectiveness scores of multiple students. The calculation method is as follows: ,in, Indicates the first The overall teaching effectiveness score for each class corresponding to a time window. Indicates the first The first time window The teaching effectiveness score for each student. This indicates the total number of students in the class.

[0150] Subsequently, the system calculates the group learning state vector based on the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector corresponding to multiple students. The calculation methods are as follows:

[0151]

[0152]

[0153]

[0154] in, Indicates the first The group cognitive input state vector corresponding to each time window Indicates the first The group emotional identification state vector corresponding to each time window Indicates the first The group behavior participation state vector corresponding to each time window; , as well as They represent the first The first time window corresponds to the second The cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are corresponding to each student.

[0155] Furthermore, the system divides students into regions based on classroom seating distribution information. In one implementation, the classroom seating area is divided into front row, middle row, and back row areas, and the teaching effectiveness scores and learning status vectors of students in each classroom area are calculated to generate regional teaching status evaluation results.

[0156] For the The scoring method for the teaching effectiveness of each classroom area is as follows: ,in, Indicates the first The first time window The teaching effectiveness score corresponds to each classroom area. Indicates the first Students gathered within the classroom area. Indicates the first The number of students in each classroom area.

[0157] The system assesses differences in classroom status based on regional teaching effectiveness scores for different classroom areas. When a region's teaching effectiveness score consistently falls below the overall class teaching effectiveness score, the system determines that the region's learning status is declining. When the mean value of a region's cognitive engagement state vector is low, the system determines that students in that region lack sufficient understanding of the current teaching content. When the mean value of a region's behavioral participation state vector is low, the system determines that students in that region have insufficient classroom participation.

[0158] Furthermore, the system performs trend analysis on the overall teaching effectiveness scores of the class within a continuous time window to generate a trend of the overall teaching status of the class. The calculation method is as follows: ,in, Indicates the first The overall teaching status trend of the class corresponding to each time window. This indicates the overall teaching status trend of the class corresponding to the previous time window. This indicates the overall teaching effectiveness score for the class corresponding to the current time window. This represents the trend smoothing coefficient, which satisfies... .

[0159] In one implementation, the trend smoothing coefficient Set to 0.8. If the overall teaching status trend of the class continues to rise over multiple consecutive time windows, the system determines that the overall learning status of the current classroom is enhanced; if the overall teaching status trend of the class continues to decline over multiple consecutive time windows, the system determines that the overall learning status of the current classroom is weakened; if the fluctuation range of the overall teaching status trend of the class is lower than the preset threshold over multiple consecutive time windows, the system determines that the overall learning status of the current classroom is stable.

[0160] Furthermore, the system generates classroom group teaching feedback information based on the group cognitive engagement state vector, the group emotional identification state vector, and the group behavioral participation state vector. In one implementation, when the average value of the group cognitive engagement state vector is lower than a preset cognitive threshold, the system generates a teaching feedback suggestion: "The overall cognitive engagement in the current classroom is insufficient; it is recommended to reduce the density of knowledge points, increase case explanations, or slow down the pace of explanation." When the average value of the group emotional identification state vector is lower than a preset emotional threshold, the system generates a teaching feedback suggestion: "The emotional identification in the current classroom is insufficient; it is recommended to enhance contextualized expression or combine real-world cases for value guidance." When the average value of the group behavioral participation state vector is lower than a preset behavioral threshold, the system generates a teaching feedback suggestion: "The behavioral participation in the current classroom is insufficient; it is recommended to increase the frequency of questioning, group discussions, or classroom interaction."

[0161] In one implementation, the preset cognitive threshold, preset emotional threshold, and preset behavioral threshold are all set to 0.6. When the state intensity of a corresponding group is below 0.6, the system generates teaching feedback suggestions for the corresponding dimension; when the state intensity of multiple groups is simultaneously below 0.6, the system determines the main feedback direction based on the state dimension with the largest decrease.

[0162] Furthermore, the system can generate classroom area feedback information based on the teaching effectiveness scores and changes in the learning status of each area. When the teaching effectiveness score of the back area is consistently lower than that of the front and middle areas, the system generates a teaching feedback suggestion: "The classroom participation in the back area is low; it is recommended to increase interactive questioning by back students or adjust the organization of classroom activities." When the cognitive engagement in the middle area is consistently high, the system generates a teaching feedback message: "Students in the middle area have a good understanding of the current teaching content and can be used as a guidance area for classroom interaction."

[0163] Furthermore, the system visualizes the overall class teaching effectiveness score, the overall class teaching status trend, changes in group cognitive engagement, changes in group emotional identification, changes in group behavioral participation, and differences in classroom area status. In one implementation, the teacher's interface includes an overall class teaching effectiveness score curve, an overall class teaching status trend curve, curves showing changes in the learning status of the three groups, and a comparison chart of teaching effectiveness in different classroom areas, to assist teachers in observing changes in the student group status during classroom teaching.

[0164] Through the above methods, this embodiment can continuously evaluate the overall teaching status of the class, the learning status of different classroom areas, and the differences in the learning status of student groups, based on the multimodal ideological and political education teaching evaluation system described in Embodiment 1, thereby improving the ability to analyze group status and assist in teaching decision-making during the ideological and political education classroom teaching evaluation process.

Claims

1. A multimodal ideological and political education teaching evaluation system integrating electroencephalogram (EEG) signals and eye-tracking signals, characterized in that, It includes a data acquisition module, a feature construction module, a basic cross-modal representation module, a learning state modeling module, a state-driven dynamic fusion module, and a teaching evaluation module; The data acquisition module is used to acquire the EEG and eye movement signals of students during ideological and political education classes, and to perform synchronous acquisition, time window division, and preprocessing of the EEG and eye movement signals based on a unified time reference. The feature construction module is used to extract features from the EEG signals and eye movement signals to construct EEG feature vectors and eye movement feature vectors under the corresponding time window; The basic cross-modal representation module is used to map the EEG feature vector and the eye-tracking feature vector to a unified feature space to generate unified EEG feature vector and unified eye-tracking feature vector, and to construct a cross-modal correlation matrix based on the unified EEG feature vector and the unified eye-tracking feature vector. The module then performs correlation enhancement on the unified EEG feature vector and the unified eye-tracking feature vector according to the cross-modal correlation matrix to generate cross-modal correlation feature vector. The learning state modeling module is used to take the cross-modal correlation feature vector corresponding to the current time window as the state input, and combine it with the learning state vector corresponding to the previous time window to generate the learning state vector corresponding to the current time window through a neural network gating mechanism; the learning state modeling module is also used to generate cognitive input state vector, emotional identification state vector and behavioral participation state vector based on the learning state vector. The state-driven dynamic fusion module is used to generate a fusion mode control vector based on the cognitive engagement state vector, the emotional identification state vector, and the behavioral participation state vector, and to generate EEG modal weight response values ​​and eye-tracking modal weight response values ​​based on the fusion mode control vector; the state-driven dynamic fusion module is also used to normalize the EEG modal weight response values ​​and the eye-tracking modal weight response values ​​to obtain EEG modal dynamic weight coefficients and eye-tracking modal dynamic weight coefficients, so that the EEG modal dynamic weight coefficients and the eye-tracking modal dynamic weight coefficients satisfy the normalization constraints; The state-driven dynamic fusion module is further used to perform state-driven weighted fusion of the unified EEG feature vector and the unified eye-tracking feature vector based on the dynamic weight coefficients of the EEG modality and the dynamic weight coefficients of the eye-tracking modality, so as to generate a state-driven fusion feature vector, which is used as the input of the teaching evaluation module. The teaching evaluation module is used to perform joint feature evaluation based on the state-driven fusion feature vector, the learning state vector, and the explicit evaluation feature vector constructed by normalizing explicit teaching behavior data. It also generates teaching effectiveness scores under corresponding time windows and teaching state trends under continuous time windows through the teaching evaluation network.

2. The multimodal ideological and political education teaching evaluation system integrating electroencephalogram (EEG) signals and eye movement signals according to claim 1, characterized in that: The data acquisition module includes an EEG acquisition unit, an eye-tracking acquisition unit, a time synchronization unit, a window division unit, and a preprocessing unit; The EEG acquisition unit is used to acquire the EEG signals of students during ideological and political education classes through EEG acquisition equipment; The eye-tracking acquisition unit is used to acquire students' eye-tracking signals during ideological and political education classes through an eye-tracking device. The time synchronization unit is used to synchronize and align the EEG signal and the eye movement signal based on a unified time reference. The window division unit is used to divide the synchronized and aligned EEG signals and eye movement signals into continuous time windows according to a preset time length and a preset overlap ratio. The preprocessing unit is used to filter and denoise the EEG signal, suppress artifacts and remove abnormal signals, and remove abnormal fixation points, compensate for missing data and smooth the trajectory of the eye movement signal. The preprocessed EEG and eye movement signals are used as inputs to the feature construction module.

3. The multimodal ideological and political education assessment system integrating EEG signals and eye movement signals according to claim 1, characterized in that: The feature construction module is used to divide the EEG signal into frequency bands according to multiple preset EEG frequency bands, and select a target frequency band related to the classroom learning state from the multiple preset EEG frequency bands; The target frequency band includes frequency band frequency band and frequency band, the Frequency bands are used to characterize changes in cognitive load, the Frequency bands are used to characterize changes in attentional states, the Frequency bands are used to characterize changes in learning engagement. The feature construction module is used to calculate the spectral energy features of the target frequency band within the corresponding time window, and combine the spectral energy features according to the EEG channel dimension to construct an EEG feature vector; The eye movement signal includes fixation point coordinates, saccade trajectory, and fixation duration; The feature construction module is used to construct a gaze region distribution feature based on the gaze point coordinates within the corresponding time window, construct a saccade trajectory change feature and a saccade direction change feature based on the saccade trajectory, and construct a gaze duration feature based on the gaze duration. The feature construction module is used to combine the gaze region distribution features, the saccade trajectory change features, the saccade direction change features, and the gaze duration features to construct an eye movement feature vector; The EEG feature vector and the eye movement feature vector serve as inputs to the basic cross-modal representation module.

4. The multimodal ideological and political education assessment system integrating EEG signals and eye movement signals according to claim 1, characterized in that: The basic cross-modal representation module includes a multimodal alignment unit, a feature mapping unit, and a correlation modeling unit; The multimodal alignment unit is used to receive the time-synchronized EEG feature vector and eye movement feature vector, and to perform time-correspondence alignment of the EEG feature vector and the eye movement feature vector within the corresponding time window; The feature mapping unit is used to respectively process the first... Linear mapping is performed on the EEG feature vectors and eye movement feature vectors corresponding to each time window to obtain unified EEG feature vectors and unified eye movement feature vectors in a unified feature space. The unified EEG feature vectors and unified eye movement feature vectors have the same dimension, and their calculation methods are as follows: in, Indicates the first EEG feature vectors corresponding to each time window Indicates the first Eye-tracking feature vectors corresponding to each time window Indicates the first Each time window corresponds to a unified feature vector of EEG. Indicates the first Each time window corresponds to a unified eye-tracking feature vector. This represents the feature mapping matrix corresponding to the EEG modalities. This represents the feature mapping matrix corresponding to the eye-tracking modality. The bias vector representing the EEG modality. The bias vector representing the eye-tracking mode; The correlation modeling unit is used to construct a cross-modal correlation matrix based on the unified EEG feature vector and the unified eye-tracking feature vector, and its calculation method is as follows: ,in, Indicates the first The cross-membrane state correlation matrix corresponding to each time window Indicates the transpose operation; The correlation modeling unit is further configured to perform cross-weighting on the eye-tracking unified feature vector and the EEG unified feature vector based on the cross-modal correlation matrix, and to sum the cross-weighting results to generate a cross-modal correlation feature vector. The calculation method is as follows: ,in, Indicates the first Cross-modal correlation feature vectors corresponding to each time window This represents the eye-to-EEG direction correlation response obtained by weighting the unified eye-tracking feature vector based on the cross-modal correlation matrix. This represents the EEG-to-eye movement direction correlation response obtained by weighting the unified EEG feature vector based on the transposed cross-modal correlation matrix. The cross-modal correlation feature vector is used to simultaneously characterize the bidirectional correlation response between the unified EEG feature vector and the unified eye-tracking feature vector, and serves as the input to the learning state modeling module.

5. The multimodal ideological and political education assessment system integrating electroencephalogram (EEG) signals and eye movement signals according to claim 1, characterized in that: The learning state modeling module includes a state representation construction unit, a state evolution unit, and a state semantic mapping unit; The state representation construction unit is used to generate a candidate learning state vector based on the cross-modal association feature vector corresponding to the current time window and the learning state vector corresponding to the previous time window. For the initial time window, the learning state vector corresponding to the previous time window is initialized to a zero vector. The candidate learning state vector is calculated as follows: ,in, Indicates the first The candidate learning state vectors corresponding to each time window Indicates the first Cross-modal correlation feature vectors corresponding to each time window This represents the learning state vector corresponding to the previous time window. The input mapping matrix represents the candidate state. Represents the candidate state history mapping matrix. This represents the candidate state bias vector. Represents the hyperbolic tangent activation function; The state evolution unit is used to construct a state update gate based on the cross-modal correlation feature vector corresponding to the current time window and the learned state vector corresponding to the previous time window, through a neural network gating mechanism. Its calculation method is as follows: ,in, Indicates the first The state update gate vector corresponding to each time window This indicates updating the gate input mapping matrix. This indicates updating the gate history mapping matrix. This indicates updating the gate bias vector. This represents the Sigmoid activation function; The state evolution unit is used to generate the learning state vector corresponding to the current time window based on the state update gate, the candidate learning state vector, and the learning state vector corresponding to the previous time window. Its calculation method is as follows: ,in, This represents the state update gate vector. Vectors consisting entirely of 1s of the same dimension Indicates the first The learning state vector corresponding to each time window This represents Hadamard element-wise multiplication; The state semantic mapping unit is used to generate cognitive engagement state vectors, emotional identification state vectors, and behavioral participation state vectors based on the learned state vectors, and their calculation methods are as follows: in, Indicates the first The cognitive input state vector corresponding to each time window Indicates the first The emotional identification state vector corresponding to each time window Indicates the first The behavior participation state vector corresponding to each time window , as well as These represent the cognitive engagement state mapping matrix, the emotional identification state mapping matrix, and the behavioral participation state mapping matrix, respectively. , as well as These represent the cognitive engagement state mapping bias vector, the emotional identification state mapping bias vector, and the behavioral participation state mapping bias vector, respectively. The cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector are used to characterize the student's cognitive engagement level, emotional acceptance level, and behavioral activity level under the current time window, respectively, and serve as inputs to the state-driven dynamic fusion module for generating the fusion mode control vector.

6. The multimodal ideological and political education assessment system integrating EEG signals and eye movement signals according to claim 1, characterized in that: The state-driven dynamic fusion module includes a state control unit, a weight response generation unit, a weight normalization unit, and a dynamic fusion unit. The state control unit is used to generate a fusion mode control vector through a state control network based on the cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector corresponding to the current time window. The calculation method is as follows: ,in, Indicates the first The fusion mode control vector corresponding to each time window This represents the cognitive input state mapping matrix. Represents the emotional identification state mapping matrix. The behavior-participation state mapping matrix represents the state mapping matrix. This represents the fusion mode control bias vector. , , They represent the first The cognitive engagement state vector, emotional identification state vector, and behavioral participation state vector corresponding to each time window. Represents the hyperbolic tangent activation function; The fusion mode control vector is used to characterize the fusion bias relationship between the EEG mode and the eye-tracking mode under the current time window, and to adjust the generation process of the EEG mode weight response value and the eye-tracking mode weight response value. The weighted response generation unit is used to generate EEG modal weighted response values ​​and eye-tracking modal weighted response values ​​based on the fusion mode control vector, and their calculation methods are as follows: in, Indicates the first The EEG modality weighted response values ​​corresponding to each time window Indicates the first The eye-tracking modal weighted response values ​​corresponding to each time window and These represent the EEG modal weighted response matrix and the eye-tracking modal weighted response matrix, respectively. and These represent the weighted response bias parameters of the EEG modality and the weighted response bias parameters of the eye-tracking modality, respectively. The weight normalization unit is used to perform Softmox normalization processing based on the EEG modal weight response values ​​and eye-tracking modal weight response values ​​to generate EEG dynamic weight coefficients and eye-tracking dynamic weight coefficients, which are calculated as follows: in, Indicates the first The EEG dynamic weighting coefficients corresponding to each time window Indicates the first The eye-tracking dynamic weighting coefficients corresponding to each time window Represents the exponential mapping function; The dynamic weighting coefficients of the EEG modality and the dynamic weighting coefficients of the eye movement modality satisfy the normalization constraint condition: , ,and ; The dynamic fusion unit is used to perform weighted fusion of the unified EEG feature vector and the unified eye-tracking feature vector based on the EEG dynamic weight coefficient and the eye-tracking dynamic weight coefficient to generate a state-driven fusion feature vector. The calculation method is as follows: ,in, Indicates the first The state-driven fusion feature vector corresponds to each time window. Indicates the first The unified EEG feature vector corresponding to each time window Indicates the first The unified eye-tracking feature vector corresponding to each time window, wherein the EEG dynamic weight coefficient and the eye-tracking dynamic weight coefficient change with the changes in the cognitive engagement state vector, the emotional identification state vector and the behavioral participation state vector, are used to make the state-driven fusion feature vector adapt to the differences in information contribution between the EEG modality and the eye-tracking modality under different classroom learning states. The state-driven fusion feature vector serves as the input to the joint evaluation unit in the teaching evaluation module.

7. The multimodal ideological and political education assessment system integrating EEG signals and eye movement signals according to claim 1, characterized in that: The teaching assessment module includes an explicit indicator construction unit, a joint assessment unit, and a dynamic assessment output unit; The explicit indicator construction unit is used to acquire explicit teaching behavior data in the teaching process, and to perform statistical and normalization processing on the explicit teaching behavior data according to the classroom segment, teaching stage or preset statistical period corresponding to the current time window, so as to generate explicit evaluation feature vectors. The explicit teaching behavior data includes student attendance rate, homework completion status, classroom test scores and classroom interaction times. The explicit indicator construction unit constructs an explicit evaluation feature vector based on normalized explicit teaching behavior data, and its calculation method is as follows: ,in, Indicates the first The explicit evaluation feature vector corresponding to each time window. This indicates the attendance rate as an evaluation metric. Indicators representing the evaluation of task completion status Indicators for classroom performance evaluation Indicators representing the number of classroom interactions This represents a vector-level concatenation operation; The joint evaluation unit is used to perform vector-level concatenation of the state-driven fusion feature vector, the learning state vector, and the explicit evaluation feature vector to generate a joint feature vector for teaching evaluation. Its calculation method is as follows: ,in, Indicates the first The joint feature vector of teaching evaluation corresponding to each time window Indicates the first The state-driven fusion feature vector corresponds to each time window. Indicates the first The learning state vector corresponding to each time window Indicates the first The explicit evaluation feature vector corresponding to each time window; The joint evaluation unit, based on the joint feature vector of teaching evaluation, performs a nonlinear mapping on the joint feature vector of teaching evaluation through a teaching evaluation network to generate a teaching effectiveness score for the corresponding time window. The calculation method is as follows: ,in, Indicates the first The teaching effectiveness score corresponds to each time window. This represents the weight matrix corresponding to the teaching evaluation network. Indicates the bias parameter. This represents the Sigmoid activation function; The dynamic evaluation output unit is used to perform time series trend analysis on teaching effectiveness scores within a continuous time window to generate a teaching status trend, which is calculated as follows: ,in, Indicates the first The teaching status trend corresponding to each time window This indicates the trend of teaching status corresponding to the previous time window. This indicates the teaching effectiveness score corresponding to the current time window. Represents the trend smoothing coefficient, and satisfies For the initial time window, the teaching status trend is initialized to the teaching effectiveness score corresponding to the current time window.

Images