Classroom concentration degree and cognitive state evaluation system based on multi-modal data fusion

Abstract

The invention relates to a classroom concentration degree and cognitive state evaluation system based on multi-modal data fusion. The system comprises a data acquisition module which acquires student physiological behaviors and classroom environment signals to form original heterogeneous data; the feature fusion module processes the data to generate a fusion feature vector group, and outputs a standardized dynamic sequence after detecting an eye dissociation and heartbeat abnormality association mode; the state analysis module locates an abnormal fluctuation starting point through a convolutional neural network, determines a potential cognitive decline trajectory through linear regression, and obtains future cognitive state aggravation probability distribution through a long short-term memory network; and the early warning intervention module compares the threshold value, if the threshold value exceeds the standard, the trajectory and the probability are fused to construct a dynamic intervention model, and a personalized intervention strategy is output. The system solves the problems of strong subjectivity, insufficient precision, lagging and the like of traditional evaluation, and provides core technical support for intelligent classroom landing.

Description

Technical Field

[0001] This invention belongs to the field of smart education technology, and in particular relates to a classroom attention and cognitive state assessment system based on multimodal data fusion. Background Technology

[0002] With the deep integration of educational informatization and artificial intelligence technologies, smart education has become a core direction for improving the quality and efficiency of classroom teaching. Student classroom focus and cognitive status, as key indicators reflecting teaching effectiveness and optimizing teaching strategies, are receiving increasing attention from the education and technology sectors for their accurate assessment. Traditional assessments of classroom focus and cognitive status often rely on teacher observation, post-class questionnaires, or single test feedback, which suffer from inherent defects such as strong subjectivity, assessment lag, and incomplete sample coverage, making it difficult to capture real-time changes in students' dynamic learning states. While assessment schemes based on single-modal data are gradually emerging in existing technologies, single-modal data is easily affected by the classroom environment and cannot comprehensively represent the multi-dimensional coupling of cognitive states involving "behavior-physiology-environment," leading to insufficient assessment accuracy and a high misjudgment rate. Although some studies have attempted to introduce multimodal data fusion technology, most schemes suffer from insufficient handling of data heterogeneity, a lack of targeted feature fusion logic, and insufficient adaptation to classroom scenarios, making it difficult to achieve real-time, accurate assessment of students' cognitive states and support for personalized intervention, thus failing to meet the actual needs of current smart classrooms for dynamic learning and emotional cognition. Summary of the Invention

[0003] Therefore, it is necessary to provide a classroom attention and cognitive state assessment system based on multimodal data fusion that can effectively process heterogeneous information, accurately locate cognitive state anomalies, and predict the evolution trend of students' cognitive state, in order to address the above-mentioned technical problems.

[0004] Firstly, this application provides a classroom attention and cognitive state assessment system based on multimodal data fusion, including:

[0005] The data acquisition module is used to collect physiological and behavioral signals of students in the classroom environment and environmental signals of the classroom environment to obtain a raw heterogeneous data set.

[0006] The feature fusion module is used to process the original heterogeneous data set using feature extraction methods to determine the fusion feature vector group. If a correlation pattern between eye movement and abnormal heartbeat is detected in the fusion feature vector group, the heterogeneous characteristics of the fusion feature vector group are uniformly mapped to the standardized analysis framework to obtain a standardized dynamic sequence.

[0007] The state analysis module is used to extract spatiotemporal related features based on standardized dynamic sequences using a convolutional neural network to determine the starting point of abnormal fluctuations; it is also used to determine the potential cognitive decline trajectory of the starting point of abnormal fluctuations using a linear regression method; and it is also used to process the potential cognitive decline trajectory using a long short-term memory network to obtain the probability distribution of the student's cognitive state worsening within a preset time period in the future.

[0008] The early warning and intervention module is used to construct a dynamic intervention model by integrating the potential cognitive decline trajectory with the probability distribution if the probability distribution exceeds a preset threshold, thereby obtaining personalized intervention strategy data.

[0009] In one embodiment, the feature fusion module is further configured to:

[0010] Multimodal preprocessing is performed on the original heterogeneous dataset. Through data cleaning, spatiotemporal alignment, and outlier removal, preprocessed heterogeneous data with a regular structure is obtained.

[0011] Temporal dynamic features and spatial morphological features are extracted from physiological behavioral signals in preprocessed heterogeneous data, and statistical distribution features and intensity variation features are extracted from environmental signals in preprocessed heterogeneous data, forming a multimodal single feature set.

[0012] The attention mechanism is used to perform weighted fusion of multimodal single feature sets to obtain a fused feature vector group.

[0013] A probabilistic graphical model-based association pattern detection algorithm is used to perform feature association analysis on the fused feature vector group to determine whether there is a co-variation association pattern between eye wandering and abnormal heartbeat, and output the association pattern detection results; the co-variation association patterns include cognitive distraction pattern, cognitive fatigue pattern, cognitive overload pattern and emotional anxiety pattern.

[0014] If the correlation pattern detection result indicates the existence of a collaborative change correlation pattern, then the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the preset standardization analysis framework based on the preset feature standardization rules to obtain a standardized dynamic sequence.

[0015] In one embodiment, the state analysis module further includes:

[0016] Feature extraction and localization unit, used for:

[0017] The standardized dynamic sequence is subjected to temporal resampling and dimension normalization to unify the time step and feature dimension of the sequence, thereby generating a normalized temporal feature sequence.

[0018] A multi-scale convolutional neural network model is constructed based on regularized temporal feature sequences. Local short-term spatiotemporal correlation features and global long-term spatiotemporal dependency features are extracted by convolutional kernels of different sizes, resulting in a multi-scale spatiotemporal feature set.

[0019] An adaptive threshold detection algorithm is used to perform first-order difference and second-order derivative analysis on a multi-scale spatiotemporal feature set to capture minute abrupt change signals in feature values ​​and locate the starting point of abnormal fluctuations in students' status.

[0020] Based on the starting point of abnormal fluctuations, the length of the time window is adaptively adjusted according to the intensity of characteristic mutations, and the characteristic sequence segments within the time window are extracted to obtain focused fluctuation analysis segments.

[0021] Trajectory modeling and prediction unit, used for:

[0022] For focused fluctuation analysis segments, linear regression is used to quantify the rise and fall amplitudes, rates of change, and stability of fluctuations, thereby identifying potential cognitive decline trajectories.

[0023] A bidirectional long short-term memory network is used to model the potential cognitive decline trajectory, while capturing the historical dependencies and future evolution trends of the trajectory, and outputting a time-series dependency weight matrix.

[0024] By combining the time-series dependency weight matrix, the probability distribution of the aggravation of students' cognitive state within a future preset time period is obtained through function mapping; the probability distribution includes the aggravation probability of different types of cognitive abnormalities and their corresponding confidence levels.

[0025] In one embodiment, the probability distribution of the student's cognitive state worsening within a predetermined time period is calculated using the following formula:

[0026]

[0027]

[0028]

[0029] in, This represents the probability distribution of an increase in students' cognitive state within a predetermined future time period. Indicates the first Cognitive abnormalities in the first The probability of exacerbation at each time step Indicates the first Cognitive abnormalities in the first Each time step corresponds to a confidence level. The first element in the time series dependency weight matrix represents the... Time step, number Dependency weights corresponding to cognitive abnormalities Indicates the first The type matching weights between cognitive abnormalities and potential cognitive decline trajectories are calculated based on the prior correlation between trajectory features and abnormality types. This indicates the potential cognitive decline trajectory in the 1st... The trend strength factor at each time step is calculated using the first derivative of the trajectory. Indicates the first The consistency coefficient of the time step-dependent weights. , Indicates the minimum value. , for standard deviation This indicates the total number of time steps in the future preset time period. Indicates the number of exception types.

[0030] In one embodiment, if the probability distribution exceeds a preset threshold, a dynamic intervention model is constructed by fusing the potential cognitive decline trajectory with the probability distribution to obtain personalized intervention strategy data, including:

[0031] The core assessment indicators were extracted from the probability distribution of the aggravation of students' cognitive state. The core assessment indicators include the maximum risk probability, the weighted average aggravation probability, the proportion of high-risk time, the number of multiple types of concurrent risks, and the confidence-weighted risk mean.

[0032] If the core assessment indicators exceed the preset cognitive state risk threshold, the potential cognitive decline trajectory and probability distribution are fused at the feature level to generate a real-time alarm signal that includes the anomaly type, risk level, and key nodes of the trajectory.

[0033] The core information of real-time alarm signals is analyzed, and an intervention basis sequence including intervention priority, applicable intervention scenarios, and core intervention objectives is generated by combining classroom scenario adaptation rules. The classroom scenario adaptation rules include the teaching stage and the difficulty of the teaching content.

[0034] By combining the intervention basis sequence with preset intervention rules and students' historical cognitive status data, a dynamic intervention model is constructed. The real-time adjustment plan for students' cognitive status is determined through model iteration and optimization. The real-time adjustment plan includes the direction of adjustment, the intensity of intervention, and the timing of implementation.

[0035] Based on real-time adjustments to the plan, combined with individual student characteristics and historical intervention effect data, personalized intervention strategy data is generated, which includes specific intervention methods, implementation steps, and expected effect evaluation indicators.

[0036] Secondly, this application also provides a method for assessing classroom attention and cognitive state based on multimodal data fusion, the method including:

[0037] The physiological and behavioral signals of students and the environmental signals of the classroom environment were collected to obtain a raw heterogeneous dataset.

[0038] Feature extraction methods are used to process the original heterogeneous dataset and determine the fusion feature vector group.

[0039] If a correlation pattern between eye movement and abnormal heartbeat is detected in the fused feature vector group, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the standardized analysis framework to obtain a standardized dynamic sequence.

[0040] Based on standardized dynamic sequences, spatiotemporal correlation features are extracted using convolutional neural networks to determine the starting point of abnormal fluctuations.

[0041] The potential cognitive decline trajectory was determined by linear regression at the starting point of abnormal fluctuations.

[0042] By using a long short-term memory network to process potential cognitive decline trajectories, the probability distribution of the student's cognitive state worsening within a preset future time period is obtained.

[0043] If the probability distribution exceeds a preset threshold, a dynamic intervention model is constructed by integrating the potential cognitive decline trajectory with the probability distribution to obtain personalized intervention strategy data.

[0044] Secondly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned system.

[0045] Thirdly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned system.

[0046] The aforementioned classroom attention and cognitive state assessment system based on multimodal data fusion, along with computer equipment and storage media, comprises the following modules: A data acquisition module collects physiological behavioral signals from students and environmental signals from the classroom environment, forming a raw heterogeneous data set. A feature fusion module receives this raw heterogeneous data set, processes it using feature extraction methods to generate a fused feature vector group, and then performs correlation pattern detection on the fused feature vector group. If a correlation pattern between wandering eyes and abnormal heartbeat is detected, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to a standardized analysis framework, outputting a standardized dynamic sequence. A state analysis module uses the standardized dynamic sequence as input, extracts spatiotemporal related features through a convolutional neural network, and then determines the starting point of abnormal fluctuations in student state. Based on this starting point, a linear regression method is used to quantify and determine the potential cognitive decline trajectory. A long short-term memory network is then used to perform temporal modeling of the potential cognitive decline trajectory, obtaining a probability distribution of the student's cognitive state worsening within a preset future time period. An early warning and intervention module receives this probability distribution and compares it with a preset threshold. If the probability distribution exceeds the preset threshold, the potential cognitive decline trajectory and the probability distribution are fused to construct a dynamic intervention model, ultimately outputting personalized intervention strategy data. This system effectively addresses the problems of traditional assessment methods, such as strong subjectivity, insufficient accuracy, and significant lag, through multi-module collaboration and multi-technology integration. Firstly, it adopts a multi-modal data acquisition mode, taking into account both student physiological behavior and classroom environment dimensions, which can more comprehensively capture key factors affecting cognitive state compared to single-modal data acquisition. Secondly, it achieves effective integration and noise filtering of heterogeneous data through feature fusion and correlation pattern detection, ensuring the effectiveness of standardized dynamic sequences. Thirdly, by combining convolutional neural networks and long short-term memory networks, it achieves both accurate localization of abnormal fluctuations and temporal prediction of cognitive state, improving the real-time performance and accuracy of assessment. Fourthly, it constructs a dynamic intervention model based on assessment results, generating personalized intervention strategy data, which can specifically support the optimization of teaching strategies and the adjustment of students' cognitive state, providing core technical support for the implementation of smart classrooms. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 The structural block diagram of the classroom attention and cognitive state assessment system based on multimodal data fusion provided in the embodiments of the present invention;

[0049] Figure 2A flowchart illustrating a method for assessing classroom attention and cognitive state based on multimodal data fusion, provided as an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In one embodiment, such as Figure 1 As shown, this application provides a classroom attention and cognitive state assessment system based on multimodal data fusion, which may include the following steps:

[0052] The data acquisition module 101 is used to collect physiological and behavioral signals of students in the classroom environment and environmental signals of the classroom environment to obtain a raw heterogeneous data set.

[0053] Specifically, this module acquires two types of key data by deploying data acquisition devices adapted to the classroom environment: one type is students' physiological behavioral signals, encompassing behavioral and physiological representations that reflect cognitive state, such as changes in eye contact, heart rate, facial expressions, and body movements, which can be obtained through visual acquisition devices (such as high-definition cameras) and wearable physiological sensors; the other type is environmental signals of the classroom environment, including environmental parameters that may affect students' concentration and cognitive state, such as light intensity, noise levels, temperature, and humidity, which are collected through environmental sensors. Because the two types of data differ in their sources, types, formats, and dimensions (e.g., physiological behavioral signals are mostly time-series or image data, while environmental signals are mostly numerical time-series data), the acquired data will be output as a raw, heterogeneous dataset.

[0054] The feature fusion module 102 is used to process the original heterogeneous data set using feature extraction methods to determine the fusion feature vector group. If the correlation pattern between eye movement and abnormal heartbeat is detected in the fusion feature vector group, the heterogeneous characteristics of the fusion feature vector group are uniformly mapped to the standardized analysis framework to obtain a standardized dynamic sequence.

[0055] Furthermore, the feature fusion module receives the raw heterogeneous data set output by the data acquisition module. Its core objective is to complete the feature extraction, fusion, and standardization of this heterogeneous data. First, targeted feature extraction methods are used to process the raw heterogeneous data modally: for physiological behavioral signals, temporal dynamic features (such as temporal changes in heart rate and temporal fluctuations in eye gaze duration) and spatial morphological features (such as spatial posture of limb movements and regional features of facial expressions) are extracted; for environmental signals, statistical distribution features (such as the mean and variance of noise decibels over a certain period) and intensity variation features (such as the instantaneous change in light intensity) are extracted, ultimately integrating them to form a multimodal single feature set. Subsequently, a feature fusion algorithm is used to fuse the multimodal single feature set, eliminating modal differences and enhancing key information to obtain a fused feature vector group. Meanwhile, an association pattern detection algorithm is used to analyze the fused feature vector group to determine whether there is a collaborative change association pattern between eye wandering and abnormal heartbeat. If such an association pattern is detected, it indicates that the student's cognitive state may be abnormal. At this time, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the standardized analysis framework according to the preset feature standardization rules to achieve the unification of data dimensions and scales. Finally, a standardized dynamic sequence is output to provide standardized input for the subsequent state analysis module.

[0056] The state analysis module 103 is used to extract spatiotemporal related features based on a standardized dynamic sequence through a convolutional neural network to determine the starting point of abnormal fluctuations; it is also used to determine the potential cognitive decline trajectory of the starting point of abnormal fluctuations through a linear regression method; and it is also used to process the potential cognitive decline trajectory with a long short-term memory network to obtain the probability distribution of the student's cognitive state intensifying within a future preset time period.

[0057] Schematic illustration: The state analysis module takes the standardized dynamic sequence output by the feature fusion module as input. Through multi-step temporal analysis and modeling, it completes the anomaly localization, trajectory mining, and future trend prediction of students' cognitive states. First, based on the standardized dynamic sequence, the spatiotemporal feature extraction capability of a convolutional neural network (CNN) is used to mine the spatiotemporal related features contained in the sequence. The time dimension features reflect the temporal evolution of the data, while the spatial dimension features reflect the correlation between different feature dimensions. By analyzing these features, minute mutations in feature values ​​are captured, thereby accurately determining the starting point of abnormal fluctuations in students' states. Second, using the starting point of the abnormal fluctuation as a benchmark, feature sequence segments within a preset time window before and after the starting point are extracted. Linear regression is used to quantify these segments. By fitting a trend equation, key indicators such as the magnitude of the fluctuation, the rate of change, and stability are calculated. Combined with the indicator features, the trend of cognitive state changes is fitted to determine the potential cognitive decline trajectory. Third, considering that the potential cognitive decline trajectory has obvious temporal dependence characteristics, a Long Short-Term Memory (LSTM) network is used to perform temporal modeling on the trajectory, fully capturing the historical evolutionary dependence and future development trend of the trajectory, and outputting a temporal sequence dependence weight matrix. Based on this weight matrix, the trajectory evolution characteristics are transformed into probabilistic information through function mapping, and finally the probability distribution of the student's cognitive state aggravation within a future preset time period is obtained. This distribution includes the aggravation probability of different cognitive abnormality types and their corresponding confidence levels.

[0058] The early warning and intervention module 104 is used to construct a dynamic intervention model by integrating the potential cognitive decline trajectory and the probability distribution if the probability distribution exceeds a preset threshold, thereby obtaining personalized intervention strategy data.

[0059] The early warning and intervention module, based on the probability distribution output by the state analysis module, enables early warning of abnormal cognitive states and the generation of personalized intervention strategies. First, the module extracts core assessment indicators from the probability distribution (such as maximum risk probability, weighted average aggravation probability, and high-risk time percentage), compares these indicators with preset cognitive state risk thresholds to determine if the risk of a student's cognitive state worsening meets the early warning criteria. If the core assessment indicators exceed the preset thresholds, it indicates a risk of cognitive deterioration. At this point, the module performs feature-level fusion of the potential cognitive decline trajectory and the probability distribution—integrating the trajectory's evolutionary trend characteristics (such as decline rate and key nodes) with the probability distribution's risk level characteristics (such as risk intensity and confidence level) as input to the dynamic intervention model. The dynamic intervention model, combining a preset intervention rule base, students' historical cognitive state data, and classroom scenario adaptation rules, determines a real-time adjustment plan for the student's cognitive state through iterative model optimization, clarifying the adjustment direction, intervention intensity, and implementation timing. Ultimately, based on the real-time adjustment plan and combined with individual student characteristics (such as learning habits and physiological response patterns) and historical intervention effect data, personalized intervention strategy data is generated, which includes specific intervention methods, implementation steps and expected effect evaluation indicators. This provides data support for teachers to optimize teaching strategies, adjust teaching pace or conduct targeted guidance.

[0060] The aforementioned classroom attention and cognitive state assessment system based on multimodal data fusion comprises the following modules: a data acquisition module collects physiological behavioral signals of students and environmental signals of the classroom environment, forming a raw heterogeneous data set; a feature fusion module receives this raw heterogeneous data set, processes it using feature extraction methods to generate a fused feature vector group, and then performs correlation pattern detection on the fused feature vector group. If a correlation pattern between wandering eyes and abnormal heartbeat is detected, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to a standardized analysis framework, outputting a standardized dynamic sequence; a state analysis module takes the standardized dynamic sequence as input, extracts spatiotemporal related features through a convolutional neural network, and then determines the starting point of abnormal fluctuations in student state. Based on this abnormal fluctuation starting point, a linear regression method is used to quantify and determine the potential cognitive decline trajectory, and then a long short-term memory network is used to perform temporal modeling of the potential cognitive decline trajectory to obtain the probability distribution of the student's cognitive state worsening within a preset time period; an early warning and intervention module receives this probability distribution and compares it with a preset threshold. If the probability distribution exceeds the preset threshold, the potential cognitive decline trajectory and the probability distribution are fused to construct a dynamic intervention model, ultimately outputting personalized intervention strategy data. This system effectively addresses the problems of traditional assessment methods, such as strong subjectivity, insufficient accuracy, and significant lag, through multi-module collaboration and multi-technology integration. Firstly, it adopts a multi-modal data acquisition mode, taking into account both student physiological behavior and classroom environment dimensions, which can more comprehensively capture key factors affecting cognitive state compared to single-modal data acquisition. Secondly, it achieves effective integration and noise filtering of heterogeneous data through feature fusion and correlation pattern detection, ensuring the effectiveness of standardized dynamic sequences. Thirdly, by combining convolutional neural networks and long short-term memory networks, it achieves both accurate localization of abnormal fluctuations and temporal prediction of cognitive state, improving the real-time performance and accuracy of assessment. Fourthly, it constructs a dynamic intervention model based on assessment results, generating personalized intervention strategy data, which can specifically support the optimization of teaching strategies and the adjustment of students' cognitive state, providing core technical support for the implementation of smart classrooms.

[0061] In one embodiment, the feature fusion module can also be used for:

[0062] Step S101: Perform multimodal preprocessing on the original heterogeneous dataset. Through data cleaning, spatiotemporal alignment, and outlier removal, obtain preprocessed heterogeneous data with a regular structure.

[0063] Step S102: Extract temporal dynamic features and spatial morphological features from physiological behavioral signals in preprocessed heterogeneous data, and extract statistical distribution features and intensity change features from environmental signals in preprocessed heterogeneous data to form a multimodal single feature set.

[0064] Step S103: Use the attention mechanism to perform weighted fusion of multimodal single feature sets to obtain a fused feature vector group.

[0065] Step S104: Using a probabilistic graphical model-based association pattern detection algorithm, feature association analysis is performed on the fused feature vector group to determine whether there is a co-current change association pattern between eye wandering and abnormal heartbeat, and the association pattern detection result is output; the co-current change association patterns include cognitive distraction pattern, cognitive fatigue pattern, cognitive overload pattern, and emotional anxiety pattern.

[0066] Step S105: If the correlation pattern detection result is that there is a collaborative change correlation pattern, then the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the preset standardization analysis framework based on the preset feature standardization rules to obtain the standardized dynamic sequence.

[0067] Specifically, the first step is to perform multimodal preprocessing on the acquired raw heterogeneous dataset. Data cleaning removes redundant data, duplicate records, and noise from the acquisition equipment. Spatiotemporal alignment maps physiological and environmental signals from different sources and acquisition timestamps to a unified spatiotemporal dimension. Outlier removal uses statistical methods (3σ criterion) or domain rules to filter out abnormal data exceeding reasonable limits, ultimately resulting in preprocessed heterogeneous data with a well-structured and reliable data quality. Based on this preprocessed heterogeneous data, modal feature extraction is performed: for physiological and behavioral signals, temporal dynamic features reflecting their dynamic changes (such as signal temporal fluctuation trends and key time node feature values) and spatial morphological features characterizing their spatial attributes (such as spatial coordinate distribution of limb movements and morphological parameters of facial feature regions) are extracted. For environmental signals, statistical distribution features reflecting their overall distribution characteristics (such as mean, variance, and quantiles) and intensity change features reflecting their instantaneous changes (such as signal intensity abrupt change amplitude and rate of change) are extracted, and then integrated to form a multimodal single feature set. Subsequently, using an attention mechanism, weights are dynamically assigned based on the correlation between each modal feature and classroom focus and cognitive state. The multimodal single feature sets are then weighted and fused to enhance the contribution of key modal features and suppress redundant information interference, resulting in a fused feature vector group. A probabilistic graphical model-based association pattern detection algorithm is used to perform cross-modal feature association analysis on the fused feature vector group, determining whether there is a collaborative change association pattern between eye movement and abnormal heartbeat. This collaborative change association pattern includes cognitive distraction, cognitive fatigue, cognitive overload, and emotional anxiety patterns. The corresponding association pattern detection results are output. If the detection results indicate the existence of such a collaborative change association pattern, then based on preset feature standardization rules (Min-Max normalization, Z-Score standardization), the heterogeneous characteristics of the fused feature vector group (differences in the dimensions and numerical ranges of different modal features) are uniformly mapped to a preset standardization analysis framework, achieving dimension unification and distribution alignment of the features, ultimately resulting in a standardized dynamic sequence.

[0068] This embodiment effectively solves the problems of heterogeneity, inconsistent data quality, and weak feature correlation in multimodal data, providing high-quality data and feature support for subsequent student cognitive state assessment. Multimodal preprocessing, through cleaning, alignment, and removal operations, significantly improves the reliability and regularity of the original data, avoiding interference from noisy data and outliers in subsequent analysis. Modal feature extraction employs targeted extraction methods based on different attributes of physiological behavioral signals and environmental signals, ensuring the effectiveness and relevance of each modality's features and comprehensively capturing key information affecting students' cognitive states. Attention-based weighted fusion dynamically focuses on core features strongly correlated with cognitive states, improving the information density and discriminability of the fused feature vector group compared to traditional equal-weight fusion. The association pattern detection algorithm based on a probabilistic graphical model accurately identifies collaborative change association patterns between multimodal features, achieving preliminary screening and classification of abnormal cognitive states. Feature standardization eliminates dimensional differences between heterogeneous features, ensuring the stability and accuracy of the entire assessment process and improving the overall accuracy and reliability of the system's assessment of students' cognitive states.

[0069] In one embodiment, the state analysis module may further include:

[0070] Feature extraction and localization unit, used for:

[0071] Step S201: Perform time-series resampling and dimension normalization on the standardized dynamic sequence to unify the time step and feature dimension of the sequence and generate a normalized time-series feature sequence.

[0072] Step S202: Construct a multi-scale convolutional neural network model based on the normalized temporal feature sequence, and extract local short-term spatiotemporal correlation features and global long-term spatiotemporal dependency features through convolutional kernels of different sizes to obtain a multi-scale spatiotemporal feature set.

[0073] Step S203: An adaptive threshold detection algorithm is used to perform first-order difference and second-order derivative analysis on the multi-scale spatiotemporal feature set to capture small mutation signals of feature values ​​and locate the starting point of abnormal fluctuations in student status.

[0074] Step S204: Based on the starting point of the abnormal fluctuation, the length of the time window is adaptively adjusted according to the intensity of the characteristic mutation, and the characteristic sequence segment within the time window is extracted to obtain the focused fluctuation analysis segment.

[0075] Trajectory modeling and prediction unit, used for:

[0076] Step S205: For the focused fluctuation analysis segment, the rise and fall amplitude, rate of change and stability of the fluctuation are quantified by linear regression method to determine the potential cognitive decline trajectory.

[0077] Step S206: A bidirectional long short-term memory network is used to model the potential cognitive decline trajectory, while capturing the historical dependencies and future evolution trends of the trajectory, and outputting a time-series dependency weight matrix.

[0078] Step S207: Combining the time-series dependency weight matrix, the probability distribution of the aggravation of students' cognitive state within a future preset time period is obtained through function mapping; the probability distribution includes the aggravation probability of different types of cognitive abnormalities and their corresponding confidence levels.

[0079] Specifically, the feature extraction and localization unit and the trajectory modeling and prediction unit work together to locate anomalies in students' cognitive states, mine trajectories, and predict trends. The data flow forms a closed loop and is logically progressive: the feature extraction and localization unit takes a standardized dynamic sequence as input, first performs temporal resampling and dimension normalization on it, unifying the time step of the sequence (eliminating temporal misalignment caused by different sampling frequencies) and feature dimensions (ensuring consistency of each feature dimension), generating a normalized temporal feature sequence; based on this normalized temporal feature sequence, a multi-scale convolutional neural network model is constructed. By setting convolutional kernels of different sizes, local short-term spatiotemporal correlation features (small-sized convolutional kernels capture immediate feature correlations) and global long-term spatiotemporal correlation features are extracted respectively. Null-dependent features (large-size convolutional kernels capture cross-time period feature correlations) are integrated to form a multi-scale spatiotemporal feature set. An adaptive threshold detection algorithm is used to perform first-order difference (reflecting the instantaneous change amplitude of features) and second-order derivative (reflecting the abrupt change in the rate of change of features) analysis on the multi-scale spatiotemporal feature set to accurately capture the minute abrupt change signals of feature values, thereby locating the starting point of abnormal fluctuations in student status. Based on the starting point of abnormal fluctuations, the length of the time window is adaptively adjusted according to the feature abrupt change intensity (a shorter time window corresponds to a higher abrupt change intensity, focusing on the core abrupt change region; a longer time window corresponds to a lower abrupt change intensity, covering the complete fluctuation process), and feature sequence segments within the time window are extracted to obtain focused fluctuation analysis segments. After receiving a focused fluctuation analysis segment, the trajectory modeling and prediction unit performs quantitative analysis on it using linear regression. It fits a trend equation to calculate the fluctuation's rise / fall amplitude, rate of change, and stability (characterized by the ratio of standard deviation to mean). Combining the quantitative results, it determines the potential cognitive decline trajectory. A bidirectional long short-term memory network is used to perform temporal modeling of the potential cognitive decline trajectory, simultaneously capturing the trajectory's historical temporal dependencies (the influence of past states on the present) and future evolution trends (the prediction of the future from the current state), outputting a temporal sequence dependency weight matrix. Combining this temporal sequence dependency weight matrix, the trajectory evolution characteristics are transformed into probabilistic information through function mapping, ultimately obtaining the probability distribution of the student's cognitive state worsening within a preset future time period. This probability distribution includes the aggravation probability of different types of cognitive abnormalities and their corresponding confidence levels.

[0080] This embodiment effectively solves the problems of incomplete spatiotemporal feature extraction, inaccurate anomaly localization, insufficient trajectory modeling, and insufficient prediction reliability in cognitive state assessment. The application of multi-scale convolutional neural networks overcomes the limitation of single-scale convolutional kernels in taking into account both local and global spatiotemporal features, ensuring the completeness of feature extraction; the combination of adaptive threshold detection and adaptive time window adjustment improves the accuracy of locating the starting point of abnormal fluctuations and the targeting of focused fluctuation analysis segments, avoiding information redundancy or missing information caused by fixed time windows; the quantitative analysis of linear regression methods makes the determination of potential cognitive decline trajectories based on objective data indicators, reducing subjective judgment errors; compared with traditional long short-term memory networks, bidirectional long short-term memory networks increase the dimension of capturing future evolutionary trends, improving the comprehensiveness and accuracy of temporal modeling; the final output probability distribution includes anomaly type, aggravation probability, and confidence level, which not only clarifies the risk type and intensity but also quantifies the reliability of prediction results, thus improving the real-time performance, accuracy, and interpretability of student cognitive state assessment.

[0081] In one embodiment, the probability distribution of an increase in a student's cognitive state within a predetermined time period can be calculated using the following formula:

[0082]

[0083]

[0084]

[0085] in, This represents the probability distribution of an increase in students' cognitive state within a predetermined future time period. Indicates the first Cognitive abnormalities in the first The probability of exacerbation at each time step Indicates the first Cognitive abnormalities in the first Each time step corresponds to a confidence level. The first element in the time series dependency weight matrix represents the... Time step, number Dependency weights corresponding to cognitive abnormalities Indicates the first The type matching weights between cognitive abnormalities and potential cognitive decline trajectories are calculated based on the prior correlation between trajectory features and abnormality types. This indicates the potential cognitive decline trajectory in the 1st... The trend strength factor at each time step is calculated using the first derivative of the trajectory. Indicates the first The consistency coefficient of the time step-dependent weights. , Indicates the minimum value. , for standard deviation This indicates the total number of time steps in the future preset time period. Indicates the number of exception types.

[0086] This embodiment's probability distribution calculation formula achieves triple collaborative modeling by integrating time-series sequence dependency weights, anomaly type matching weights, and trajectory trend intensity factors, thus leveraging... Capturing the historical temporal dependencies of potential cognitive decline trajectories, through It adapts to the prior correlation between different types of cognitive abnormalities and trajectories, and also utilizes... The dynamic evolution trend of quantified trajectories effectively overcomes the problem of insufficient predictive specificity caused by traditional function mapping relying on only a single dimension of information; at the same time, a consistency coefficient is introduced. The system calculates the corresponding confidence level and simultaneously outputs the amplified probability and credibility, quantifying the reliability of the prediction results and solving the problem that a single probability value cannot reflect the uncertainty of prediction. The final output probability distribution covers the probability and confidence of multiple cognitive abnormality types and multiple time steps, significantly improving the scientific nature of cognitive state prediction and the rationality of subsequent intervention decisions.

[0087] In one embodiment, if the probability distribution exceeds a preset threshold, a dynamic intervention model is constructed by fusing the potential cognitive decline trajectory with the probability distribution to obtain personalized intervention strategy data, which may include the following steps:

[0088] Step S301: Extract the core assessment indicators from the probability distribution of the aggravation of students' cognitive state; the core assessment indicators include the maximum risk probability, the weighted average aggravation probability, the proportion of high-risk time, the number of multiple types of concurrent risks, and the confidence weighted risk mean.

[0089] Step S302: If the core assessment indicators exceed the preset cognitive state risk threshold, feature-level fusion is performed on the potential cognitive decline trajectory and probability distribution to generate a real-time alarm signal containing the anomaly type, risk level, and key nodes of the trajectory.

[0090] Step S303: Analyze the core information of the real-time alarm signal and generate an intervention basis sequence that includes intervention priority, applicable intervention scenarios, and core intervention objectives by combining classroom scenario adaptation rules; classroom scenario adaptation rules include teaching stage and difficulty of teaching content.

[0091] Step S304: Combine the intervention basis sequence with the preset intervention rules and the student's historical cognitive state data to construct a dynamic intervention model. Determine the real-time adjustment plan for the student's cognitive state through model iteration and optimization. The real-time adjustment plan includes the adjustment direction, intervention intensity, and implementation timing.

[0092] Step S305: Based on the real-time adjustment plan, combined with individual student characteristics and historical intervention effect data, generate personalized intervention strategy data that includes specific intervention methods, implementation steps, and expected effect evaluation indicators.

[0093] Specifically, using the probability distribution of students' cognitive state worsening within a preset future time period as input, core assessment indicators are extracted. These core assessment indicators include the maximum risk probability, weighted average aggravation probability, proportion of high-risk times, number of concurrent risks of multiple types, and confidence-weighted risk mean, achieving multi-dimensional quantitative refinement of the probability distribution. The extracted core assessment indicators are compared with preset cognitive state risk thresholds. If any core assessment indicator exceeds the preset threshold, it indicates that the student's cognitive state is at risk of deterioration. At this time, the potential cognitive decline trajectory and probability distribution are fused at the feature level, integrating the evolution characteristics of the trajectory and the risk characteristics of the probability distribution to generate a real-time alarm signal containing anomaly type, risk level, and key nodes of the trajectory. The real-time alarm signal is then processed... The core information is analyzed, and combined with classroom scenario adaptation rules (covering key scenario elements such as teaching stage and difficulty of teaching content), the priority of intervention, applicable classroom scenarios, and core intervention objectives are clarified to generate an intervention basis sequence. This intervention basis sequence is combined with preset intervention rules and students' historical cognitive state data to construct a dynamic intervention model. Through iterative optimization of the model, data bias is eliminated, and a real-time adjustment plan for students' cognitive state, including adjustment direction, intervention intensity, and implementation timing, is determined. Based on the real-time adjustment plan, the intervention details are further refined by combining individual student characteristics (such as learning habits and physiological response patterns) and historical intervention effect data to generate personalized intervention strategy data that includes specific intervention methods, implementation steps, and expected effect evaluation indicators.

[0094] This embodiment effectively addresses the problems of insufficient targeting, poor adaptability, and lack of quantitative support in traditional intervention methods, possessing significant technical advantages: the extraction of multi-core assessment indicators overcomes the one-sidedness of single-indicator assessment, comprehensively assessing cognitive state risks from dimensions such as risk intensity, overall level, duration, complexity, and reliability, thus improving the accuracy of risk identification; feature-level fusion and the generation of real-time alarm signals achieve precise integration and intuitive presentation of risk information, providing clear decision-making guidance for subsequent interventions; the introduction of classroom scenario adaptation rules makes the intervention basis more closely aligned with actual teaching scenarios, avoiding a disconnect between intervention and teaching pace and content difficulty; the dynamic intervention model, combined with preset rules and historical data, improves the scientificity and rationality of adjustment plans through iterative optimization; finally, personalized strategies are generated by combining individual student characteristics and historical intervention effects, achieving dual precision intervention of "scenario adaptation + individual adaptation," ensuring the effectiveness of intervention while providing quantitative support for teaching strategy optimization, and promoting the implementation of personalized teaching in smart classrooms.

[0095] In one embodiment, such as Figure 2 As shown, this application also provides a method for assessing classroom attention and cognitive state based on multimodal data fusion, which may include the following steps:

[0096] Step S401: Collect students' physiological behavior signals and environmental signals of the classroom environment to obtain the original heterogeneous data set.

[0097] Step S402: Use feature extraction methods to process the original heterogeneous data set and determine the fused feature vector group.

[0098] Step S403: If a correlation pattern between eye movement and abnormal heartbeat is detected in the fused feature vector group, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the standardized analysis framework to obtain a standardized dynamic sequence.

[0099] Step S404: Based on the standardized dynamic sequence, extract spatiotemporal related features through a convolutional neural network to determine the starting point of abnormal fluctuations.

[0100] Step S405: Determine the potential cognitive decline trajectory for the starting point of abnormal fluctuations using a linear regression method.

[0101] Step S406: Use a long short-term memory network to process the potential cognitive decline trajectory and obtain the probability distribution of the student's cognitive state worsening within a preset future time period.

[0102] Step S407: If the probability distribution exceeds the preset threshold, then the potential cognitive decline trajectory and the probability distribution are integrated to construct a dynamic intervention model and obtain personalized intervention strategy data.

[0103] The aforementioned method for assessing classroom attention and cognitive state based on multimodal data fusion first collects physiological behavioral signals of students and environmental signals of the classroom environment to form a raw heterogeneous data set. Then, a feature extraction method is used to extract and fuse features from this raw heterogeneous data set in different modalities, eliminating data heterogeneity and enhancing key information to determine a fused feature vector group. The fused feature vector group is then subjected to association pattern detection. If an association pattern between eye movement and abnormal heartbeat is detected, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to a standardized analysis framework according to preset rules, resulting in a standardized dynamic sequence. This standardized dynamic sequence is then used to... Using this as input, a convolutional neural network is used to extract spatiotemporal features to accurately determine the starting point of abnormal fluctuations in students' states. Based on this starting point, a linear regression method is used to quantitatively analyze the trend of feature sequence changes and determine the potential cognitive decline trajectory. A long short-term memory network is used to perform temporal modeling of the potential cognitive decline trajectory to capture the historical dependence and future evolution trend of the trajectory, obtaining the probability distribution of the student's cognitive state worsening within a preset time period. Finally, this probability distribution is compared with a preset threshold. If the probability distribution exceeds the preset threshold, the potential cognitive decline trajectory and the probability distribution are integrated to construct a dynamic intervention model, ultimately outputting personalized intervention strategy data.

[0104] This embodiment of the method effectively addresses the problems of traditional classroom cognitive assessment, such as strong subjectivity, limited data support, and lack of targeted intervention, through the integration of multiple technologies and the collaboration of multiple stages. Multimodal data collection takes into account both students' physiological behavior and classroom environment dimensions, and captures key factors affecting cognitive state more comprehensively than single-modal data, providing a rich data foundation for subsequent analysis. Feature fusion and correlation pattern detection enable effective integration of heterogeneous data and preliminary screening of anomalies, ensuring the effectiveness of standardized dynamic sequences. Combining convolutional neural networks and long short-term memory networks, it achieves both accurate localization of abnormal fluctuations and temporal prediction of cognitive state, improving the real-time performance and accuracy of assessment. Based on probability distribution threshold judgment and dynamic intervention model construction, personalized intervention strategies are generated by combining trajectory and probability information, realizing the seamless connection from state assessment to precise intervention, providing technical support for personalized teaching in smart classrooms, and improving teaching adaptability and the effect of adjusting students' cognitive state.

[0105] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0106] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the classroom attention and cognitive state assessment system based on multimodal data fusion as described above.

[0107] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0108] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0109] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A classroom attention and cognitive state assessment system based on multimodal data fusion, characterized in that, The system includes: The data acquisition module is used to collect physiological and behavioral signals of students and environmental signals of the classroom environment to obtain a raw heterogeneous data set. The feature fusion module is used to process the original heterogeneous data set using feature extraction methods to determine the fusion feature vector group; if a correlation pattern between eye movement and abnormal heartbeat is detected in the fusion feature vector group, the heterogeneous characteristics of the fusion feature vector group are uniformly mapped to a standardized analysis framework to obtain a standardized dynamic sequence. The state analysis module is used to extract spatiotemporal related features based on the standardized dynamic sequence through a convolutional neural network to determine the starting point of abnormal fluctuations; it is also used to determine the potential cognitive decline trajectory of the starting point of abnormal fluctuations through a linear regression method; and it is also used to process the potential cognitive decline trajectory with a long short-term memory network to obtain the probability distribution of the student's cognitive state worsening within a future preset time period. The early warning and intervention module is used to construct a dynamic intervention model by fusing the potential cognitive decline trajectory with the probability distribution if the probability distribution exceeds a preset threshold, thereby obtaining personalized intervention strategy data.

2. The system according to claim 1, characterized in that, The feature fusion module is also used for: The original heterogeneous dataset is subjected to multimodal preprocessing, including data cleaning, spatiotemporal alignment, and outlier removal, to obtain preprocessed heterogeneous data with a regular structure. Temporal dynamic features and spatial morphological features are extracted from the physiological behavioral signals in the preprocessed heterogeneous data, and statistical distribution features and intensity change features are extracted from the environmental signals in the preprocessed heterogeneous data to form a multimodal single feature set; The multimodal single feature set is weighted and fused using an attention mechanism to obtain a fused feature vector group; An association pattern detection algorithm based on a probabilistic graphical model is used to perform feature association analysis on the fused feature vector group to determine whether there is a co-variation association pattern between eye wandering and abnormal heartbeat, and output the association pattern detection results; the co-variation association patterns include cognitive distraction pattern, cognitive fatigue pattern, cognitive overload pattern, and emotional anxiety pattern. If the correlation pattern detection result indicates the existence of the collaborative change correlation pattern, then the heterogeneous characteristics of the fused feature vector group are uniformly mapped to the preset standardization analysis framework based on the preset feature standardization rules to obtain a standardized dynamic sequence.

3. The system according to claim 1, characterized in that, The status analysis module also includes: Feature extraction and localization unit, used for: The standardized dynamic sequence is subjected to temporal resampling and dimension normalization to unify the time step and feature dimension of the sequence, thereby generating a normalized temporal feature sequence. A multi-scale convolutional neural network model is constructed based on the normalized temporal feature sequence. Local short-term spatiotemporal correlation features and global long-term spatiotemporal dependency features are extracted by convolutional kernels of different sizes to obtain a multi-scale spatiotemporal feature set. An adaptive threshold detection algorithm is used to perform first-order difference and second-order derivative analysis on the multi-scale spatiotemporal feature set to capture small abrupt change signals of feature values ​​and locate the starting point of abnormal fluctuations in student status. Based on the starting point of the abnormal fluctuation, the length of the time window is adaptively adjusted according to the intensity of the characteristic mutation, and the characteristic sequence segment within the time window is extracted to obtain the focused fluctuation analysis segment. Trajectory modeling and prediction unit, used for: The focused fluctuation analysis segment is quantified using linear regression to determine the rise and fall amplitude, rate of change, and stability of the fluctuation, thereby identifying the potential cognitive decline trajectory. A bidirectional long short-term memory network is used to model the potential cognitive decline trajectory, while capturing the historical dependencies and future evolution trends of the trajectory, and outputting a time-series dependency weight matrix. By combining the time-series dependent weight matrix, a probability distribution of the aggravation of students' cognitive state within a preset time period is obtained through function mapping; the probability distribution includes the aggravation probability of different types of cognitive abnormalities and their corresponding confidence levels.

4. The system according to claim 3, characterized in that, The probability distribution of the student's cognitive state worsening within the predetermined future time period is calculated using the following formula: in, This represents the probability distribution of an increase in students' cognitive state within a predetermined future time period. Indicates the first Cognitive abnormalities in the first The probability of exacerbation at each time step Indicates the first Cognitive abnormalities in the first Each time step corresponds to a confidence level. The first element in the time series dependency weight matrix represents the... Time step, first Dependency weights corresponding to cognitive abnormalities Indicates the first The type matching weights between cognitive abnormalities and potential cognitive decline trajectories are calculated based on the prior correlation between trajectory features and abnormality types. This indicates the potential cognitive decline trajectory in the first... The trend strength factor at each time step is calculated using the first derivative of the trajectory. Indicates the first The consistency coefficient of the time step-dependent weights. , Indicates the minimum value. , for Standard deviation This indicates the total number of time steps in the future preset time period. Indicates the number of exception types.

5. The system according to claim 1, characterized in that, If the probability distribution exceeds a preset threshold, a dynamic intervention model is constructed by fusing the potential cognitive decline trajectory with the probability distribution to obtain personalized intervention strategy data, including: Extract core assessment indicators from the probability distribution of the aggravation of students' cognitive state; the core assessment indicators include the maximum risk probability, the weighted average aggravation probability, the proportion of high-risk time, the number of multiple types of concurrent risks, and the confidence-weighted risk mean. If the core assessment indicator exceeds the preset cognitive state risk threshold, the potential cognitive decline trajectory and the probability distribution are fused at the feature level to generate a real-time alarm signal containing the anomaly type, risk level, and key nodes of the trajectory. The core information of the real-time alarm signal is analyzed, and an intervention basis sequence including intervention priority, applicable intervention scenario, and core intervention goal is generated by combining classroom scenario adaptation rules; the classroom scenario adaptation rules include teaching stage and teaching content difficulty. By combining the intervention sequence with preset intervention rules and students' historical cognitive state data, a dynamic intervention model is constructed. The real-time adjustment plan for students' cognitive state is determined through iterative optimization of the model. The real-time adjustment plan includes the direction of adjustment, the intensity of intervention, and the timing of implementation. Based on the aforementioned real-time adjustment plan, and combined with individual student characteristics and historical intervention effect data, personalized intervention strategy data is generated, which includes specific intervention methods, implementation steps, and expected effect evaluation indicators.

6. A method for assessing classroom attention and cognitive state based on multimodal data fusion, characterized in that, The method includes: Collect students' physiological and behavioral signals and environmental signals of the classroom environment to obtain a raw heterogeneous dataset. The original heterogeneous data set is processed using feature extraction methods to determine the fused feature vector group; If a correlation pattern between eye movement and abnormal heartbeat is detected in the fused feature vector group, the heterogeneous characteristics of the fused feature vector group are uniformly mapped to a standardized analysis framework to obtain a standardized dynamic sequence. Based on the standardized dynamic sequence, spatiotemporal related features are extracted using a convolutional neural network to determine the starting point of abnormal fluctuations; The potential cognitive decline trajectory was determined by linear regression at the starting point of the abnormal fluctuations. The potential cognitive decline trajectory is processed using a long short-term memory network to obtain the probability distribution of the student's cognitive state worsening within a preset future time period; If the probability distribution exceeds a preset threshold, the potential cognitive decline trajectory and the probability distribution are integrated to construct a dynamic intervention model and obtain personalized intervention strategy data.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the system according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the system according to any one of claims 1 to 5.

Images