Artificial intelligence dynamic voluntary learning behavior analysis method and system based on multi-modal data

Abstract

The application provides an artificial intelligence dynamic voluntary learning behavior analysis method and system based on multi-modal data, comprising: collecting multi-modal behavior sequence data, using a recurrent neural network to process time sequence dependence, and generating a behavior evolution mode; extracting a labeled sample subset according to the behavior evolution mode, generating an expanded sequence through a data enhancement technology, and constructing an enhanced time sequence dataset; training a recurrent neural network model using the enhanced time sequence dataset, optimizing long-term dependence relationship parameters, and generating a time sequence prediction model; extracting a feature vector from the time sequence prediction model, calculating a deviation for an abnormal behavior segment, and generating an abnormal behavior identification set; using a causal inference model to analyze the correlation path between external factors and behavior changes through the abnormal behavior identification set, and generating a causal relationship graph; generating a targeted adjustment sequence according to the causal relationship graph, and obtaining a preliminary intervention scheme sequence; feeding back the preliminary intervention scheme sequence to the time sequence prediction model for simulation verification, and generating a final behavior intervention path.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method and system for analyzing dynamic volunteer learning behavior based on multimodal data using artificial intelligence. Background Technology

[0002] The study of predicting and intervening in student learning behavior is crucial in educational technology. Its core lies in analyzing student behavioral data to gain insights into the dynamic changes in the learning process, thereby improving teaching effectiveness and student learning outcomes. This field is important because it not only helps educators identify students' learning status in a timely manner but also provides data support for personalized education, promoting educational equity and efficiency. However, current research and practice still face many challenges and urgently need to overcome technological bottlenecks to achieve accurate modeling and effective intervention of student behavior. Existing methods often struggle to capture the temporal dynamics of student behavior and the complex causal relationships behind it when processing student behavioral data. Many solutions tend to analyze static data or snapshots of data at a single point in time, ignoring the continuity and evolution of student behavior over time. For example, traditional behavioral analysis models may only assess based on the number of times students submit assignments or the frequency of classroom responses, but fail to delve into how these behaviors change over time and how these changes are influenced by external factors such as the classroom environment and teaching methods. This limitation leads to an insufficient understanding of the deep-seated patterns of student behavior, thus affecting the accuracy of prediction and intervention.

[0003] Furthermore, student behavior during the learning process, such as the transition from low to high engagement, often exhibits both temporal continuity and dynamism. This temporal dependency requires models to capture long-term patterns within behavioral sequences, rather than just short-term fragments. However, existing technologies often suffer from weak model generalization ability when processing long-sequence data due to insufficient data or labeling difficulties. Insufficient data means it's difficult to obtain enough labeled samples to train complex temporal models, while labeling difficulties stem from the complexity and subjectivity of student behavior; for example, states like "daydreaming" or "ineffective interaction" are difficult to define and quantify objectively. Further, the modeling problem of temporal dependency directly relates to another technical challenge: how to accurately identify abnormal behavior and infer its causal relationships. Abnormal behavior, such as students consistently daydreaming or frequently engaging in ineffective interactions in class, not only needs to be detected but also its root causes need to be analyzed—for example, whether the teaching content is unengaging or the student's emotional state affects engagement. Existing technologies often struggle to accurately distinguish between normal and abnormal behavior in dynamically changing temporal data and further infer the causal factors behind these abnormal behaviors. This leads to a lack of targeted intervention strategies, making it difficult to effectively address complex scenarios. Therefore, how to accurately identify abnormal behaviors and their causal relationships by mining the inherent structure of student behavior data, especially when data is limited, has become a key issue in predicting and intervening in student learning behavior in personalized education. This problem is particularly prominent in actual teaching scenarios. For example, teachers may observe a student who is active at the beginning of class but gradually loses interest, exhibiting frequent inattentiveness or ineffective interaction, yet they cannot accurately determine the cause of this change, nor can they formulate effective intervention measures to rekindle the student's enthusiasm. By solving these technical challenges, research needs to focus not only on the surface characteristics of student behavior but also delve into its temporal evolution patterns and causal relationships, thereby providing educators with more accurate prediction and intervention tools.

[0004] In conclusion, classroom management requires a combination of real-time, precision, and personalization. Existing technologies have systemic shortcomings in integrating multimodal data, capturing dynamic behavioral trends, analyzing causal relationships, and generating effective intervention plans, making it difficult to meet the actual needs of educational scenarios for efficient and dynamic management. Summary of the Invention

[0005] On the one hand, this invention provides a method for analyzing dynamic volunteer learning behavior in artificial intelligence based on multimodal data, mainly including:

[0006] S1. Collect multimodal behavioral sequence data, use recurrent neural networks to process temporal dependencies, and generate behavioral evolution patterns; S2. Extract labeled sample subsets based on behavioral evolution patterns, generate extended sequences through data augmentation techniques, and construct an augmented time-series dataset; S3. Train a recurrent neural network model using the augmented time-series dataset, optimize long-term dependency parameters, and generate a time-series prediction model; S4. Extract feature vectors from the time-series prediction model, calculate biases for abnormal behavioral segments, and generate an abnormal behavior identifier set; S5. Analyze the correlation path between external factors and behavioral changes using a causal inference model based on the abnormal behavior identifier set, and generate a causal relationship graph; S6. Generate targeted adjustment sequences based on the causal relationship graph to obtain preliminary intervention plan sequences; S7. Feed the preliminary intervention plan sequences back to the time-series prediction model for simulation verification, and generate the final behavioral intervention path.

[0007] Furthermore, the process of collecting multimodal behavioral sequence data and using a recurrent neural network to process temporal dependencies and generate behavioral evolution patterns includes: S11. Obtaining classroom participation records and interaction frequency changes through sensor devices to form a temporal data set; S12. Inputting the temporal data set into a recurrent neural network, outputting the sequence hidden state, processing temporal dependencies, and generating a preliminary behavioral pattern; S13. Based on the preliminary behavioral pattern, combining educational interaction assessment to optimize the evolution trend and generate a behavioral evolution pattern.

[0008] Furthermore, the step of extracting a subset of labeled samples based on the behavioral evolution pattern, generating an extended sequence through data augmentation techniques, and constructing an augmented time-series dataset includes: S21. Filtering a subset of labeled samples based on the behavioral evolution pattern, comparing the number of subsets with a preset threshold, and determining whether augmentation is needed; S22. If the number of subsets is lower than the preset threshold, generating an extended sequence using sequence interpolation and noise injection; S23. Integrating classroom feedback indicators and interaction frequency adjustments to generate a preliminary augmented sequence; S24. Generating an augmented time-series dataset by integrating the preliminary augmented sequence and the original labeled sample subset.

[0009] Furthermore, the step of training a recurrent neural network model using the enhanced time-series dataset, optimizing long-term dependency parameters, and generating a time-series prediction model includes: S31. Inputting the enhanced time-series dataset into a long short-term memory network variant to process long-term dependencies of behavioral sequences; S32. Iteratively updating parameters using the gradient descent method and adjusting network weights by incorporating classroom feedback indicators; S33. Verifying model convergence by combining sequence mutation expansion and generating a time-series prediction model.

[0010] Furthermore, the step of extracting feature vectors from the time-series prediction model, calculating deviations for abnormal behavior segments, and generating an abnormal behavior identifier set includes: S41. Extracting feature vectors from the hidden layer of the time-series prediction model and calculating Euclidean distance deviations for continuous distraction records; S42. If the deviation exceeds a preset threshold, marking it as an abnormal event and obtaining the abnormal event time-series label; S43. Generating an abnormal behavior identifier set containing timestamps and type labels based on the abnormal event labels.

[0011] Furthermore, the step of analyzing the correlation path between external factors and behavioral changes using a causal inference model through an abnormal behavior identifier set to generate a causal relationship graph includes: S51. Obtaining external factor data through the abnormal behavior identifier set and constructing an initial causal graph using a structural causal model; S52. Calculating the conditional independence between variable nodes using statistical testing methods to generate preliminary correlation paths; S53. If the causal strength of the influence of teaching content exceeds a preset threshold, determining a significant correlation path; S54. Calculating the influence of intervention variables through counterfactual reasoning to generate a complete causal relationship graph.

[0012] Furthermore, the step of generating a targeted adjustment sequence based on the causal relationship map to obtain a preliminary intervention plan sequence includes: S61. Extracting intervention variables based on the causal relationship map and analyzing the weights of teaching factors;

[0013] S62. If the weight of teaching factors is higher than the preset threshold, generate a targeted adjustment sequence; S63. Integrate intervention points by adjusting the sequence to generate a preliminary intervention plan sequence.

[0014] Furthermore, the step of extracting feature vectors from the hidden layer of the time-series prediction model and calculating Euclidean distance deviation for continuous distraction records includes: S411. Calculating feature vector deviation for abnormal behavior segments by combining cosine similarity; S412. Fusing multimodal data sequences to improve the robustness of abnormal behavior detection; S413. Generating an abnormal behavior severity label based on the deviation result.

[0015] On the other hand, this invention provides an artificial intelligence dynamic volunteer learning behavior analysis system based on multimodal data, mainly comprising:

[0016] The behavior sequence acquisition module is used to collect student behavior sequence data, including classroom participation records and changes in interaction frequency, and to process the temporal dependencies using a recurrent neural network to obtain behavior evolution patterns.

[0017] The data augmentation module is used to obtain a subset of labeled samples based on the behavioral evolution pattern. If the number of labeled samples is lower than a preset threshold, an extended sequence is generated through data augmentation technology to determine the augmented time series dataset.

[0018] The time series model training module is used to train a recurrent neural network model using an enhanced time series dataset, optimize parameters for long-term dependencies in behavioral sequences, and obtain a trained time series prediction model.

[0019] The anomaly detection module is used to extract feature vectors from the trained time-series prediction model. For abnormal behavior segments, such as records of continuous inattentiveness, if the deviation of the feature vector exceeds a preset threshold, it is judged as an abnormal event, and an abnormal behavior identifier set is obtained.

[0020] The causal analysis module is used to analyze the correlation path between external factors, such as teaching content, and behavioral changes through a set of abnormal behavior identifiers and a causal inference model, and to determine the causal relationship map.

[0021] The intervention plan generation module is used to obtain intervention points based on the causal relationship graph. If the weight of teaching factors is higher than that of emotional factors in the causal path, a targeted adjustment sequence is generated to obtain a preliminary intervention plan sequence.

[0022] The effect verification module is used to feed the initial intervention plan sequence back into the time series prediction model for simulation verification. It adjusts parameters based on the deviation of the simulation results, judges the intervention effect after optimization, and obtains the final behavioral intervention path.

[0023] This invention discloses an AI-based dynamic volunteer learning behavior analysis method based on multimodal data. Addressing the challenge of accurately capturing and intervening in dynamic changes in student behavior within a classroom environment, this method integrates multimodal data acquisition, recurrent neural network processing, causal inference model analysis, and intervention plan generation to achieve real-time monitoring, anomaly identification, and personalized intervention of student learning behavior. The method first collects student action, speech, and physiological data using sensor devices to form a time-series dataset. A recurrent neural network is then used to capture behavioral change trends and generate behavioral evolution patterns. To address insufficient sample size, data augmentation techniques are employed to expand the dataset, training a long short-term memory network to optimize long-term dependencies and generate an accurate time-series prediction model. Through feature vector deviation calculation and causal inference, abnormal behaviors are identified and their correlation with external factors such as teaching content and teacher style is analyzed, generating a causal relationship graph. Finally, simulation verification generates targeted intervention plans to optimize classroom management. This invention, through the deep integration of multimodal data and AI technology, significantly improves the accuracy and effectiveness of student behavior analysis and intervention, providing intelligent support for personalized education. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating an application scenario of an artificial intelligence dynamic volunteer learning behavior analysis method based on multimodal data, provided by an embodiment of the present invention.

[0025] Figure 2This is a flowchart illustrating the steps of an artificial intelligence dynamic volunteer learning behavior analysis method based on multimodal data, provided in an embodiment of the present invention.

[0026] Figure 3 This is a schematic diagram of the multimodal behavioral data sequence collected in an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of a simplified causal relationship graph generated by an embodiment of the present invention.

[0028] Figure 5 This is a schematic diagram of the process of feeding back the initial intervention plan sequence to the time series prediction model for simulation verification in an embodiment of the present invention.

[0029] Figure 6 This is a schematic diagram of the architecture of an artificial intelligence dynamic volunteer learning behavior analysis system based on multimodal data, provided by an embodiment of the present invention. Detailed Implementation

[0030] To further understand the content of this invention, a detailed description of the invention is provided in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0031] In the classroom environment, the dynamic changes in students' voluntary learning behavior are difficult to accurately capture and effectively intervene in. This core technical problem permeates the dynamic management needs of educational scenarios. Specifically, students' classroom behavior is influenced by a variety of factors, including individual emotional fluctuations, the difficulty of teaching content, teacher teaching style, and changes in the classroom environment. These factors intertwine to form complex behavioral patterns. Traditional single-modal data collection methods are insufficient to comprehensively depict the dynamic evolution of student behavior, leading to insufficient accuracy in identifying behavioral anomalies and consequently affecting the effectiveness of personalized intervention programs. In particular, the temporal characteristics of student behavior exhibit high nonlinearity in long-term dependencies. Existing technologies struggle to effectively capture long-term behavioral patterns when processing small sample data, and insufficient data leads to model training bias and decreased prediction accuracy. Furthermore, causal relationship analysis between behavioral anomalies and external factors (such as teaching content or teacher style) is difficult, lacking a systematic causal inference framework and making it difficult to clarify the triggering mechanisms of abnormal behavior. In addition, intervention programs often lack specificity and are difficult to dynamically adjust according to individual student differences and real-time classroom situations, resulting in poor intervention effects. This problem is particularly prominent in educational settings, because classroom management requires a combination of real-time, precision, and personalization. Existing technologies have systemic shortcomings in integrating multimodal data, capturing dynamic behavioral trends, analyzing causal relationships, and generating effective intervention plans, making it difficult to meet the actual needs of educational settings for efficient dynamic management.

[0032] Example 1

[0033] This embodiment of an artificial intelligence dynamic volunteer learning behavior analysis method based on multimodal data may specifically include:

[0034] This invention provides an artificial intelligence-based dynamic volunteer learning behavior analysis method based on multimodal data. It aims to analyze changes in students' learning behavior in the classroom environment, identify abnormal behaviors, and generate targeted intervention plans by collecting and processing multimodal behavior sequence data, combined with recurrent neural networks and causal inference models. This embodiment is applied to applications such as... Figure 1 The classroom scenario shown includes data collection types (left module) such as video data (e.g., hand-raising frequency, posture changes, participation behavior); audio data (e.g., number of responses, voice intensity, speaking frequency); and physiological data (e.g., heart rate changes, movement status, attention indicators). The time synchronization mechanism (right module) includes a unified timestamp standard, a multi-source data alignment algorithm, and a high-frequency acquisition mechanism (1 time / minute). The overall application environment layout can be a standard 5-row, 4-column student seating configuration, with microphone arrays distributed on the ceiling and a multi-angle camera network covering the area, all data sources pointing to a unified data acquisition center. The following describes the method in detail with specific implementation methods, such as... Figure 2 As shown.

[0035] Step S1 involves collecting multimodal behavioral sequence data and using a recurrent neural network to process temporal dependencies, generating behavioral evolution patterns. Specifically, this involves acquiring student behavioral data in the classroom environment through various sensor devices and using a recurrent neural network to capture the temporal characteristics of the data, generating patterns that reflect trends in student behavior. Multimodal behavioral sequence data includes, but is not limited to, student classroom participation records, changes in interaction frequency, and other relevant behavioral indicators. Sensor devices can be cameras, microphone arrays, or wearable devices such as smart bracelets installed in the classroom to record student movements, speech, and physiological data. During the acquisition process, the data undergoes preprocessing, such as noise reduction and time synchronization, to ensure the accuracy of subsequent analysis. It should be noted that the collected data needs to cover a sufficient time span to reflect the dynamic changes in student behavior, such as classroom records from multiple consecutive days.

[0036] Step S11 involves acquiring classroom participation records and interaction frequency changes using sensor devices to form a time-series data set. Specifically, sensor devices collect sequence data of student behavior in the classroom. These sensor devices can be cameras, microphone arrays, or smart interactive whiteboards installed in the classroom to record student classroom behavior, such as the number of times they speak, the frequency of asking questions, and body language. Classroom participation records include specific student behaviors in the classroom, such as whether they actively answer questions or participate in group discussions. Changes in interaction frequency reflect the changes in the frequency of student interactions with the teacher or other classmates at different times, such as the number of times they speak per minute or their participation in discussions.

[0037] In one embodiment, a camera in the classroom records the frequency of students raising their hands and changes in their seating posture, a microphone array captures the number of times students respond and the intensity of their voices, and a wearable device records heart rate changes. This data is timestamped to form a time-series dataset containing multidimensional features. For example, in a 40-minute class, the number of times students raise their hands and participate in speech is recorded every minute, generating a multidimensional time series reflecting changes in student participation in the class. The acquisition devices must ensure the high frequency and accuracy of the data; for example, the camera must have high resolution to accurately distinguish students' movement characteristics. The generation of the time-series dataset considers various scenarios, such as large classrooms, group discussions, or experimental courses, ensuring the diversity and representativeness of the data.

[0038] It should be noted that generating time-series datasets requires cleaning and formatting the collected raw data. For example, invalid data caused by equipment noise should be removed, and a unified timestamp format should be implemented to ensure data consistency across the time dimension.

[0039] Figure 3This is a schematic diagram of the multimodal behavioral data sequence collected in an embodiment of the present invention. The horizontal axis represents the time of the class, and the vertical axis represents the quantified values ​​of different behaviors or physiological indicators. As shown, the system can simultaneously collect and align multiple time-series data, such as: student A's hand-raising frequency sequence 301, voice intensity sequence 302 (reflecting speaking activity), and heart rate change sequence 303 collected through a wearable device. These parallel, time-aligned sequences together constitute the multimodal input data for analyzing the student's behavioral state, providing rich information dimensions for subsequent time-dependent processing. At time T1, the student's voice intensity and heart rate simultaneously peak, possibly corresponding to a tense question-answering event.

[0040] Step S12 involves inputting the time-series data set into a recurrent neural network (RNN), which outputs the sequence hidden states to process temporal dependencies and generate preliminary behavioral patterns. Specifically, an RNN is a neural network structure suitable for processing time-series data, capable of capturing the dependencies between data over time. The time-series data set serves as input, containing data from each frame of the student's behavioral sequence, such as the number of times a student speaks or the frequency of body movements at a given moment. The RNN processes this data through hidden layers, outputting the sequence hidden states to characterize the evolutionary trend of student behavior over time.

[0041] Recurrent neural networks (RNNs), through their recurrent structure, process the temporal dependencies in input data to generate hidden state sequences that characterize changes in student behavior. Specifically, the input time-series data set contains multiple time steps, each corresponding to a multi-dimensional feature vector, such as hand-raising frequency, voice intensity, and heart rate data. The RNN processes these feature vectors one by one, generating the corresponding hidden state sequences. These hidden state sequences record the trends in student behavior over time; for example, a student's participation may be high in the first half of the class and gradually decrease in the second half. This trend is initially defined as a behavioral pattern, reflecting the student's dynamic performance in the classroom. It should be noted that the structure of the RNN can be adjusted according to the complexity of the data; for example, multiple layers of recurrent units can be used to enhance the ability to capture complex temporal relationships.

[0042] In one possible implementation, recurrent neural networks employ a long short-term memory (LSTM) network structure to better capture long-term dependencies. For example, given a student's behavioral data over a 45-minute class period, the network generates a hidden state vector reflecting behavioral trends by analyzing the frequency of speech and body language over consecutive time periods. The hidden state vector records the student's behavioral patterns at different points in time, such as the trend of transitioning from low engagement at the beginning of class to high engagement.

[0043] Step S13: Based on the preliminary behavioral pattern, and combined with the educational interaction assessment to optimize the evolution trend, a behavioral evolution pattern is generated. The preliminary behavioral pattern is generated only based on time-series data and needs to be further optimized by combining it with professional assessment standards in the field of education. In one possible implementation, educational interaction assessment includes indicators such as teacher feedback on student behavior, completion of classroom tasks, and frequency of interaction among students. For example, teachers may record the quality of a student's participation in group discussions or assess students' attention through classroom quizzes. These assessment data are combined with the preliminary behavioral pattern, and the representation of the behavioral pattern is optimized through weighted fusion. The fusion process can be based on simple linear weighting or a more complex nonlinear mapping method to ensure that the behavioral pattern accurately reflects the student's learning status. For example, in a math class, a student may raise their hand less frequently but score higher on a test, which may indicate that they are more inclined to think independently than to speak up. The optimized behavioral evolution pattern can highlight this personalized behavioral characteristic. This pattern not only reflects the trend of behavioral change but also incorporates the specific semantics of the educational scenario, which is helpful for subsequent abnormal behavior analysis.

[0044] In another embodiment, educational interaction assessment can incorporate student self-assessment data. For example, students submit evaluations of their classroom performance via smart tablets, which, along with teacher scores, serve as a basis for optimization. This approach integrates subjective and objective data, improving the accuracy of behavioral evolution patterns.

[0045] It should be noted that the purpose of generating behavioral evolution patterns is to provide basic data for subsequent steps, reflecting the trends in students' behavioral changes in the classroom. For example, a behavioral evolution pattern can show that a student is relatively passive in the first 10 minutes of class, and then gradually becomes more active, providing a basis for subsequent abnormal behavior detection and intervention program design.

[0046] Step S2 involves extracting a subset of labeled samples based on behavioral evolution patterns and generating extended sequences using data augmentation techniques to construct an augmented time-series dataset. Behavioral evolution patterns provide a dynamic representation of student behavior, but due to the complexity of classroom data and individual differences, labeled samples may not be sufficient to cover all behavioral patterns. Therefore, data augmentation techniques are used to expand the sample size, generating a richer time-series dataset to support subsequent model training. The selection of data augmentation techniques must consider the characteristics of the classroom environment; for example, student behavioral patterns may be influenced by course type or teacher style. The augmented time-series dataset can better represent the diversity of student behavior, thereby improving the model's generalization ability.

[0047] Behavioral evolution patterns provide the foundation for subsequent data processing, requiring the extraction of labeled sample subsets to support model training. Labeled sample subsets refer to data with clear behavioral labels selected from the behavioral evolution patterns, such as labels like "actively engaged" or "distracted." These labels are typically manually annotated by teachers or experts based on classroom observations, or generated using semi-automated tools.

[0048] Step S21: Select a subset of labeled samples based on the behavior evolution pattern, compare the number of subsets with a preset threshold, and determine if augmentation is needed. For example, the behavior evolution pattern contains behavioral sequences from multiple students, each sequence corresponding to a label, such as "actively participating" or "distracted." The selection process extracts a representative subset of samples based on key features of the behavior pattern, such as the fluctuation range or duration of participation frequency. The preset threshold is determined based on class size and data collection period; for example, it requires each behavior category to contain at least 100 labeled sequences. If the number of subsets is below the threshold, the data augmentation process is triggered. For example, in a class of 30 students, if there are fewer than 50 samples in the "distracted" category, data augmentation techniques are needed to supplement the data. The selection process considers the diversity of behavior patterns, ensuring that the subsets can cover different types of classroom behavior.

[0049] Step S22: If the number of subsets is lower than a preset threshold, an extended sequence is generated using sequence interpolation and noise injection. Sequence interpolation generates new time-step data by interpolating the time points of the existing sequence. For example, for a sequence recording the frequency of students raising their hands, if the original data is recorded once per minute, linear interpolation can be used to generate data points once every 30 seconds, increasing the sequence density. Noise injection simulates natural fluctuations in student behavior by adding random perturbations to the original sequence. For example, adding small-amplitude random noise to a speech intensity sequence simulates changes in students' speech caused by environmental interference. It should be noted that the amplitude of noise injection must be controlled within a reasonable range to avoid generating unrealistic sequences. For example, in a discussion session of a Chinese language class, the frequency of students speaking may fluctuate depending on the attractiveness of the discussion topic; noise injection can simulate this fluctuation, thereby generating an extended sequence that is closer to reality.

[0050] Step S23 involves integrating classroom feedback indicators and interaction frequency adjustments to generate an initial enhancement sequence. Classroom feedback indicators include teacher ratings and classroom task completion rates, while interaction frequency adjustments consider the frequency of interaction between students and teachers or classmates. For example, a teacher might record the quality of a student's answers during a class question-and-answer session and adjust the weight of the enhancement sequence based on the student's speaking frequency. The integration process weights the extended sequence to better reflect the actual needs of the classroom scenario. For instance, in a science class, if a student's participation in a group experiment is high, but teacher feedback indicates insufficient focus, the enhancement sequence will correspondingly reduce the weight of that student's participation to reflect their true classroom performance. The initial enhancement sequence retains the temporal characteristics of the original sequence while incorporating semantic information from classroom feedback.

[0051] Step S24 generates an enhanced time-series dataset by integrating the initially enhanced sequences and the original labeled sample subset. The integration process merges the initially enhanced sequences with the original labeled sample subset to form a larger time-series dataset. Specifically, integration can be achieved through a simple set union operation or through weighted fusion to ensure a smooth transition between the enhanced and original sequences. For example, in a dataset containing 100 original sequences and 200 enhanced sequences, the integration process must ensure that the time span and feature distribution of the enhanced sequences are consistent with the original sequences. The enhanced time-series dataset not only increases the sample size but also enhances data diversity, better supporting subsequent model training. For example, in classrooms across different disciplines, the enhanced dataset can cover various behavioral patterns from mathematics to art courses.

[0052] Step S3 involves training a recurrent neural network model using the enhanced time-series dataset, optimizing long-term dependency parameters, and generating a time-series prediction model. The enhanced time-series dataset provides rich sample support for model training. The recurrent neural network, through iterative optimization, captures the long-term dependencies of student behavior, generating a model capable of predicting behavioral trends. The training process must consider the complexity of classroom behavior, such as periodic changes or sudden anomalies in student behavior. The optimized model can accurately predict students' behavioral performance in future classes, providing a foundation for abnormal behavior detection and intervention.

[0053] Step S31 involves inputting the enhanced time-series dataset into a Long Short-Term Memory (LSTM) network variant to process long-term dependencies in the behavioral sequences. The LTM network is a special type of recurrent neural network capable of effectively capturing long-term dependencies in sequences. Specifically, the enhanced time-series dataset contains multiple time series, each corresponding to a student's behavioral record. The LTM network, through its gating mechanism, retains important historical information while ignoring irrelevant short-term fluctuations. For example, in a week-long class record, student engagement may vary depending on the course content or individual state; the LTM network can capture the long-term trends of these changes. During training, the network's input is a multi-dimensional feature vector, such as hand-raising frequency, voice intensity, and heart rate data, and the output is the predicted behavioral sequence. It should be noted that the LTM network variant can be adjusted according to the data scale, such as increasing the number of layers or adjusting the number of units, to adapt to the complexity of different classroom scenarios.

[0054] Specifically, recurrent neural networks take an enhanced time-series dataset as input and output a preliminary set of parameters. The network architecture can be a long short-term memory network or a gated recurrent unit to capture long-term dependencies in behavioral sequences. For example, the network generates hidden state parameters reflecting behavioral trends by analyzing the low frequency of student interaction in the first half of the class and the high frequency of interaction in the second half.

[0055] In one embodiment, for a 40-minute math class, the input data consists of students' speaking frequency and body movement sequences. The network processes this data through multiple layers of recurrent units to generate a preliminary parameter set. The parameter set includes the weights and biases of the hidden layers, representing the temporal characteristics of student behavior.

[0056] Step S32 involves iteratively updating the parameters using gradient descent and adjusting the network weights by incorporating classroom feedback metrics. Gradient descent optimizes the network parameters by minimizing the prediction error. Specifically, during training, the augmented time-series dataset is used as input to calculate the error between the predicted behavior sequence and the actual sequence. Classroom feedback metrics, such as teacher ratings of student focus, are incorporated into the loss function as additional weight adjustment factors. For example, in a history class, students may exhibit lower engagement due to the complexity of the course content; teacher feedback can help the model adjust weights more accurately, highlighting key behavioral features. The iterative update process must ensure model convergence, for example, by setting a learning rate decay strategy to avoid overfitting. Preferably, regularization methods can be introduced during training to further enhance the model's generalization ability.

[0057] Step S33: Verify model convergence by combining sequence variation expansion and generate a time-series prediction model. Sequence variation expansion introduces additional perturbations into the training data to test the model's robustness to different behavioral patterns. For example, in the augmented time-series dataset, the feature values ​​of some sequences are randomly changed to simulate abnormal fluctuations in student behavior. The verification process compares the model's prediction performance on the original data and the mutated data to determine its convergence. If the model's prediction error on the mutated data is below a preset threshold, the model is considered to have converged and can generate a reliable time-series prediction model. For example, in a physical education class, students' behavioral patterns change due to physical exertion, and the model needs to be able to accurately predict these changes. The generated time-series prediction model can capture the dynamic trends of student behavior, providing support for subsequent abnormal behavior detection.

[0058] In one embodiment, the above method was applied to a middle school math class. A camera and microphone array in the classroom recorded the frequency of hand-raising and the number of times students spoke, while wearable devices recorded their heart rate data. The collected data underwent time synchronization and noise reduction to form a time-series dataset containing multi-dimensional features. A recurrent neural network processed this data to generate preliminary behavioral patterns, showing that some students' participation decreased in the latter half of the class. Educational interaction assessment, combined with teacher feedback, optimized the behavioral patterns, highlighting changes in student focus during problem-solving. The data augmentation process, through sequence interpolation and noise injection, expanded the sample size of the "distraction" category, forming an augmented time-series dataset. A long short-term memory network was trained based on this dataset, and the optimized model was able to predict student participation trends in future classes. This implementation effectively captures dynamic changes in student behavior, providing data support for teachers and optimizing classroom management.

[0059] In another embodiment, the method is applied to university discussion classes. Sensor devices include multi-angle cameras and smart bracelets installed in the classroom to record students' discussion participation and physiological data. The time-series data set reflects students' speaking frequency and changes in body movements during group discussions. Behavioral patterns generated by a recurrent neural network show that some students actively participate in the early stages of the discussion but gradually become silent later. Educational interaction assessment, combined with teacher ratings of discussion quality, optimizes behavioral patterns and highlights students' collaborative abilities. Data augmentation simulates sudden disturbances in the discussion by injecting noise, generating an enhanced dataset. After training with a long short-term memory network, the generated time-series prediction model can predict student participation trends under different discussion topics. This implementation is suitable for highly interactive classroom scenarios, helping teachers identify students' collaborative patterns and adjust teaching strategies.

[0060] The above steps, through collecting student behavior sequence data, generating behavior evolution patterns, augmenting the dataset, and training a recurrent neural network model, enable accurate predictions of classroom behavior trends. Subsequent steps will further analyze abnormal behaviors and generate intervention plans based on the trained model.

[0061] Step S4 involves extracting feature vectors from the time-series prediction model, calculating the bias for abnormal behavior segments, and generating an abnormal behavior identifier set. The time-series prediction model, through training, can accurately predict students' classroom behavior trends, and the feature vectors output from its hidden layers contain dynamic information about student behavior. For abnormal behavior segments, such as a student's persistent inattentiveness or sudden drop in participation in class, the bias of the feature vectors is calculated to identify abnormal events and generate a corresponding identifier set. The abnormal behavior identifier set not only records the time of occurrence of the abnormal event but also its type and severity, providing foundational data for subsequent causal analysis.

[0062] Specifically, feature vectors are high-dimensional vectors extracted from the hidden or output layers of a time-series prediction model, containing characteristic information about student behavior over time. For example, a feature vector might include a comprehensive representation of a student's speaking frequency, body language amplitude, and classroom participation within a certain time period. For records of persistent inattentiveness, the model determines whether abnormal behavior exists by comparing the deviation of the feature vector with a preset standard vector.

[0063] Step S41 involves extracting feature vectors from the hidden layer of the time-series prediction model and calculating the Euclidean distance bias for persistent inattentiveness records. Specifically, the feature vector output from the hidden layer of the time-series prediction model is a multi-dimensional vector representing a student's behavioral state at a specific point in time, such as participation, concentration, or interaction frequency. Persistent inattentiveness records typically manifest as a student's lack of hand-raising, speaking, or physical movement over a period of time. The degree of behavioral bias is quantified by calculating the Euclidean distance between the hidden layer feature vector and the feature vector of the normal behavioral pattern. For example, in a 40-minute class, if a student's feature vector shows significantly lower participation in the last 20 minutes compared to the first 20 minutes, the bias is calculated using the Euclidean distance. The bias calculation considers the combined influence of multimodal data, such as combining motion data captured by a camera and heart rate data recorded by a wearable device, to ensure the accuracy of the bias results.

[0064] Step S411: For abnormal behavior segments, calculate the feature vector deviation using cosine similarity. In one embodiment, the Euclidean distance deviation may be affected by differences in the dimensions of the feature vectors; therefore, cosine similarity is introduced as a supplement. Cosine similarity measures the similarity between student behavior and normal patterns by calculating the angle between feature vectors. For example, if a student's behavior vector has a large angle with the reference vector of active participation in the latter half of the class, it indicates that their behavior deviates from the normal pattern. Combining Euclidean distance and cosine similarity, the significance of abnormal behavior is comprehensively judged. For example, in a discussion session of a Chinese language class, if a student does not speak and reduces body movements, both the Euclidean distance and cosine similarity of their feature vector show significant deviations, confirming that they are exhibiting persistent inattentiveness. It should be noted that the introduction of cosine similarity improves the robustness of deviation calculation and is suitable for complex scenarios with multimodal data.

[0065] Step S412 involves fusing multimodal data sequences to improve the robustness of abnormal behavior detection. The multimodal data sequences include motion data recorded by a camera, speech data captured by a microphone array, and physiological data recorded by wearable devices. The fusion process generates a comprehensive behavioral representation by weightedly combining feature vectors from different modalities. For example, the weight of motion data may be adjusted according to the type of class, increasing the weight of motion data in experimental classes and increasing the weight of speech data in theoretical classes. The fused feature vector can more comprehensively reflect student behavior, avoiding misjudgments caused by single-modal data. For example, in a chemistry lab class, students may speak less due to focusing on experimental operations. After fusing motion and physiological data, the model can correctly distinguish between focused operation and inattentiveness, thereby improving detection accuracy.

[0066] Step S413: Based on the deviation results, generate severity labels for abnormal behaviors. The deviation results include a combined value of Euclidean distance and cosine similarity, reflecting the salience of the abnormal behavior. Severity labels are categorized into multiple levels based on the magnitude of the deviation value, such as mild, moderate, and severe. For example, if a student's deviation value exceeds twice a preset threshold, it is marked as severely abnormal; if it is close to the threshold, it is marked as mildly abnormal. Severity labels also incorporate timestamps to record the specific time the abnormality occurred. For example, in a math class, if a student's deviation value in the last 15 minutes shows severe inattentiveness, their severity label would be recorded as "Severe, Timestamp: 25:00-40:00". The generated set of abnormal behavior identifiers contains abnormal event labels for all students, providing detailed data support for subsequent causal analysis.

[0067] Step S42: If the deviation exceeds a preset threshold, it is marked as an abnormal event, and a time-series label for the abnormal event is obtained. The preset threshold is determined based on the classroom type and the behavioral characteristics of the student group. For example, the threshold may be higher in highly interactive discussion classes, while it may be lower in lecture classes that require focused attention. If a student's feature vector deviation exceeds the threshold, their behavior is marked as an abnormal event, and the corresponding timestamp and behavior type are recorded. For example, in a history class, if a student's deviation value exceeds the threshold in the second half of the class, it is marked as an "attention deficit" event, with a timestamp of "20:00-30:00". The generation of time-series labels takes into account the duration and frequency of abnormal events to ensure the comprehensiveness of the labels. It should be noted that the threshold setting can be dynamically adjusted through historical data analysis to adapt to different classroom scenarios.

[0068] Step S43: Based on the anomalous event labels, generate a set of anomalous behavior identifiers containing timestamps and type labels. The anomalous behavior identifier set is a structured dataset containing detailed information on all detected anomalous events. Each anomalous event corresponds to a label, recording its timestamp, type (e.g., "inattentive" or "insufficient participation"), and severity. For example, in a science class, the anomalous behavior identifier set might contain the following records: "Student A, inattentive, severe, timestamp: 15:00-25:00; Student B, insufficient participation, mild, timestamp: 30:00-35:00". The generation process of the set ensures data integrity and traceability, facilitating subsequent analysis of the causes and impacts of anomalous behavior.

[0069] Step S5 involves using a set of abnormal behavior identifiers and a causal inference model to analyze the correlation between external factors and behavioral changes, generating a causal relationship graph. The abnormal behavior identifier set provides information on abnormal student behavior. Combined with external factor data, such as teaching content, teacher style, or classroom environment, a causal inference model is used to analyze the relationship between these factors and abnormal behavior. The generated causal relationship graph, in the form of nodes and edges, represents the causal path between external factors and behavioral changes, providing a basis for developing intervention plans.

[0070] Step S51: Obtain external factor data through the abnormal behavior identifier set and construct an initial causal graph using a structural causal model. External factor data includes the difficulty of the teaching content, the teacher's teaching methods, and class schedule. For example, in a math class, the teaching content might include complex algebra problems, and the teacher might use a rapid explanation method; these factors could lead to student inattention. The structural causal model constructs the initial causal graph by analyzing the association between the abnormal behavior identifier set and the external factor data. The structural causal model constructs the initial causal graph by abstracting these factors and behavioral changes into variable nodes. Variable nodes are connected by directed edges, representing potential causal relationships. The nodes in the graph include abnormal behaviors (such as inattention) and external factors (such as the difficulty of the teaching content), and the edges represent possible causal relationships. For example, the initial causal graph might show an edge from "difficulty of teaching content" to "inattention," indicating that content difficulty might be one of the causes of abnormal behavior. An initial causal graph containing the main influencing factors can be pre-set based on an educational expert knowledge base. It should be noted that the collection of external factor data needs to be synchronized with the behavioral data to ensure time alignment. The difficulty of the teaching content can be quantitatively scored based on the number and complexity of the prerequisite dependencies of the knowledge points in the teaching syllabus; the frequency of teacher interaction can be statistically analyzed by examining the number of turns between teachers and students in classroom recordings.

[0071] Step S52 involves calculating the conditional independence between variable nodes using statistical tests to generate a preliminary association path. Statistical tests identify significant causal relationships by analyzing the correlation and conditional independence between variables. For example, the chi-square test or mutual information method can be used to calculate the strength of the association between the difficulty of teaching content and inattention. If the test results show that the two are still significantly correlated after controlling for other variables (such as class time), a preliminary association path is generated. For instance, in a physics class, statistical tests show a significant correlation between the complexity of the teaching content and students' inattention behavior; the preliminary association path is "complex content → inattention". The generation of the path considers the interaction of multiple external factors to ensure the accuracy of the causal relationship. In one possible implementation, the system analyzes data from a math class and finds that the chi-square test p-value between the difficulty of the teaching content and the frequency of student inattention is less than 0.05, indicating a significant association. The preliminary association path records this relationship, providing a basis for subsequent causal strength analysis.

[0072] For example, Pearson correlation coefficient and conditional mutual information are used as statistical tests to calculate the association strength between nodes of each variable. Paths with association strength below a preset threshold (e.g., 0.2) are removed from the initial causal graph, thereby generating preliminary association paths.

[0073] Step S53: If the causal strength of the influence of the teaching content exceeds a preset threshold, a significant association path is identified. Causal strength is quantified using statistical significance levels or correlation coefficients. For example, if the correlation coefficient between the difficulty of the teaching content and attention loss exceeds 0.8, the causal strength is considered significant, and the path is identified as a significant association path. The preset threshold is adjusted according to the classroom scenario; for example, in higher-grade courses, the threshold may be higher to reflect students' adaptability to complex content. In a chemistry class, if the causal strength shows that rapid switching of teaching content leads to student inattention, the path "content switching frequency → attention loss" is identified as a significant path. The identification of significant paths provides a key direction for the development of subsequent intervention plans.

[0074] In another implementation, causal strength is calculated using path coefficients to reflect the degree of influence between variable nodes. For example, a high path coefficient for the difficulty of teaching content on the frequency of student inattention indicates that it is a major influencing factor. If the path coefficient exceeds a preset threshold (e.g., 0.3), the path is considered a significant association. Extended variables are obtained, including other relevant factors such as the teacher's teaching style or class schedule, and the graph structure is updated after incorporating them into the causal graph.

[0075] For example, in one embodiment, the system found that the path coefficient between the difficulty of the teaching content and the frequency of distraction was 0.4, exceeding the threshold of 0.3, thus confirming a significant association. Extended variables such as "frequency of teacher questions" were added to the causal graph, and the updated graph showed a more complex network of causal relationships.

[0076] Step S54: Calculate the impact of intervention variables through counterfactual reasoning to generate a complete causal relationship map.

[0077] Intervention variables refer to factors that can potentially alter abnormal behavior by adjusting for changes in teaching content, such as reducing the difficulty of teaching materials or increasing classroom interaction. Counterfactual reasoning assesses the effectiveness of an intervention by comparing behavioral changes before and after the intervention. For example, assuming the difficulty of teaching materials is reduced, the system predicts whether the frequency of inattentiveness will significantly decrease. Counterfactual reasoning evaluates the impact of intervention variables by simulating scenarios where "if external factors change, will behavior change?" For example, in a Chinese language class, counterfactual reasoning analyzes whether reducing the difficulty of teaching materials would decrease the probability of students being inattentive. By comparing actual behavioral data with simulated data, the causal effect of intervention variables (such as content difficulty) is determined. A complete causal relationship map integrates all significant association paths, forming a complete causal network from external factors to abnormal behavior. For example, the map might show two paths: "content difficulty → inattention" and "frequency of teacher questioning → insufficient participation," providing a comprehensive basis for subsequent interventions.

[0078] In one embodiment, the system simulated reducing the difficulty of science lesson content and found that the frequency of daydreaming decreased from 0.5 times per minute to 0.2 times per minute, indicating that the intervention was effective. The causal chain records the complete path from "difficulty of teaching content" to "frequency of daydreaming," generating a causal relationship graph containing all significant associations. The number of directional edges is confirmed through statistical testing, and the final graph is output after meeting preset conditions.

[0079] It should be noted that the causal relationship map provides a theoretical basis for the design of subsequent intervention programs, clearly demonstrating the dynamic relationship between external factors and student behavior.

[0080] Figure 4 This is a simplified causal relationship graph generated by an embodiment of the present invention. The graph, in the form of a directed acyclic graph, shows the causal path between external factors and student behavior. In the graph, nodes represent variables, and directed edges represent causal relationships. For example, nodes 401 "Difficulty of teaching content" and 402 "Frequency of teacher questioning" are identified as external influencing factors. Both factors point to the intermediate state node 403 "Student cognitive load" through directed edges, ultimately leading to the observable abnormal behavior node 404 "Student distraction." Simultaneously, "Frequency of teacher questioning" 402 also directly affects behavior node 405 "Insufficient classroom participation." By analyzing such a graph, the key driving factors leading to student distraction can be clearly identified as excessively difficult teaching content and excessively high cognitive load, thus providing a direct basis for generating targeted intervention measures (such as reducing content difficulty or increasing explanation steps).

[0081] Step S6: Generate a targeted adjustment sequence based on the causal relationship map to obtain a preliminary intervention plan sequence.

[0082] Causal mapping provides guidance for intervention, with intervention points being the variable nodes in the map that have the greatest impact on behavioral change. Teaching factors include content difficulty and teaching methods, while emotional factors include students' emotional state and classroom atmosphere. If the weight of teaching factors is higher than that of emotional factors, intervention plans should be designed primarily targeting teaching factors. Furthermore, causal mapping reveals the causal relationship between abnormal behavior and external factors, generating targeted adjustment sequences by analyzing key intervention variables. The initial intervention plan sequence includes specific intervention measures, such as adjusting teaching content or changing classroom interaction methods, aimed at improving students' learning behavior.

[0083] Step S61 involves extracting intervention variables from the causal relationship graph and analyzing the weights of teaching factors. Intervention variables are key external factors extracted from the causal relationship graph, such as the difficulty of the teaching content or the frequency of teacher questioning. The analysis of the weights of teaching factors is determined by quantifying the contribution of each variable to abnormal behavior. For example, in a history lesson, the causal relationship graph shows that the difficulty of the teaching content contributes 60% to attention deficit, while the frequency of teacher questioning contributes 30%. Weight analysis is calculated using weighted averages or regression methods to ensure clear prioritization of intervention variables. For instance, if the difficulty of the content has a higher weight, the content design is adjusted first to reduce the cognitive burden on students.

[0084] Step S62: If the weight of a teaching factor is higher than a preset threshold, a targeted adjustment sequence is generated. The preset threshold is determined based on the classroom scenario; for example, in a course requiring high concentration, the weight threshold might be 50%. If the weight of a teaching factor exceeds the threshold, a corresponding adjustment sequence is generated. For example, in a math class, if the weight of the difficulty of the teaching content is 70%, the adjustment sequence might include "reducing the complexity of example problems" or "increasing step-by-step explanations." The generation of the adjustment sequence takes into account the actual needs of the classroom; for example, in a discussion class, prioritizing increasing the frequency of questioning to improve participation. It should be noted that the adjustment sequence must be specific and operable to ensure that teachers can implement it directly.

[0085] Step S63 involves integrating intervention points through sequence adjustments to generate a preliminary intervention plan sequence. Intervention points refer to specific implementation points within the adjustment sequence, such as at which time of day in class to adjust content or increase interaction. The integration process combines multiple adjustment sequences into a complete intervention plan sequence. For example, in a science class, the intervention plan sequence might include "reducing the difficulty of the content for the first 10 minutes, and adding a group discussion after 20 minutes." The preliminary intervention plan sequence ensures the systematic and coherent nature of the intervention measures, covering multiple aspects of abnormal behavior. For instance, for students with inattentiveness, the plan might combine content adjustments and enhanced interaction to comprehensively improve classroom participation.

[0086] Step S7 involves feeding the preliminary intervention plan sequence back into the time-series prediction model for simulation validation, generating the final behavioral intervention path. The preliminary intervention plan sequence provides adjustment measures for abnormal behavior, but its effectiveness needs to be validated through simulation. The time-series prediction model assesses the feasibility of the plan by simulating behavioral changes after intervention, generating the final behavioral intervention path. The intervention path provides teachers with specific implementation guidelines, optimizing classroom management effectiveness.

[0087] Figure 5This is a schematic diagram illustrating the process of feeding a preliminary intervention plan sequence back to a time-series prediction model for simulation verification in an embodiment of the present invention. The process begins in step 501, obtaining a preliminary intervention plan sequence generated based on a causal graph, such as "reducing the difficulty of example questions at the 20th minute". In step 502, this intervention plan is adjusted as an external variable and input into the trained time-series prediction model 505. The model simultaneously receives a student's historical behavior sequence 503 as its initial state. In step 504, based on the adjusted input, the model simulates and predicts the student's behavior sequence after the intervention. In step 506, the simulated behavior sequence is compared with the expected normal behavior pattern to determine the intervention effect. If the effect is poor (step 507 determines no), the intervention plan is returned and adjusted (step 508); if the effect is significant (step 507 determines yes), the final behavioral intervention path is generated and output in step 509. This closed-loop simulation process ensures the effectiveness of the intervention measures.

[0088] In one embodiment, steps S4 to S7 were applied to a high school physics class. An abnormal behavior identifier set indicated that some students became distracted in the latter half of the class due to the complexity of the teaching content. Euclidean distance and cosine similarity calculations of the feature vectors confirmed the severity of these abnormal events, for example, "Student A, severely distracted, timestamp: 25:00-35:00". Causal inference model analysis showed that the difficulty of the teaching content and the frequency of teacher questioning were the main influencing factors, and the generated causal relationship graph included two paths: "content difficulty → distraction" and "questioning frequency → insufficient participation". Targeted adjustment sequences included "reducing the difficulty of example problems" and "increasing the number of questions," with the initial intervention plan implemented at the 20th and 30th minutes of the class. Time-series prediction model simulations showed that student participation significantly improved after the adjustments, with the final intervention path being "simplified explanation in the first 20 minutes, increased interactive questioning in the last 20 minutes". This implementation effectively reduced student distraction and improved classroom efficiency.

[0089] In another embodiment, the method was applied to university English discussion classes. An anomalous behavior identifier set showed that some students' participation decreased in the later stages of the discussion due to a lack of teacher guidance. Eigenvector bias calculation, combined with multimodal data, confirmed the temporal label of the "insufficient participation" event. Causal relationship mapping showed that the frequency of teacher guidance and the attractiveness of the discussion topic were the main influencing factors. The adjustment sequence included "increasing teacher questions" and "adjusting the discussion topic." The initial intervention sequence introduced a new topic at the 15th minute of the discussion and added guiding questions at the 25th minute. Simulation verification showed that the frequency of student participation increased after the adjustment, and the final intervention path was "introducing new discussion points every 10 minutes and maintaining high-frequency guidance." This implementation method is suitable for highly interactive classroom scenarios, helping teachers optimize discussion effectiveness.

[0090] Example 2

[0091] As attached Figure 6 As shown, this invention provides an artificial intelligence dynamic volunteer learning behavior analysis system based on multimodal data. This system can be implemented on a server or cloud platform and mainly includes:

[0092] The behavior sequence acquisition module is used to collect student behavior sequence data, including classroom participation records and changes in interaction frequency, and to process the temporal dependencies using a recurrent neural network to obtain behavior evolution patterns.

[0093] The data augmentation module is used to obtain a subset of labeled samples based on the behavioral evolution pattern. If the number of labeled samples is lower than a preset threshold, an extended sequence is generated through data augmentation technology to determine the augmented time series dataset.

[0094] The time series model training module is used to train a recurrent neural network model using an enhanced time series dataset, optimize parameters for long-term dependencies in behavioral sequences, and obtain a trained time series prediction model.

[0095] The anomaly detection module is used to extract feature vectors from the trained time-series prediction model. For abnormal behavior segments, such as records of continuous inattentiveness, if the deviation of the feature vector exceeds a preset threshold, it is judged as an abnormal event, and an abnormal behavior identifier set is obtained.

[0096] The causal analysis module is used to analyze the correlation path between external factors, such as teaching content, and behavioral changes through a set of abnormal behavior identifiers and a causal inference model, and to determine the causal relationship map.

[0097] The intervention plan generation module is used to obtain intervention points based on the causal relationship graph. If the weight of teaching factors is higher than that of emotional factors in the causal path, a targeted adjustment sequence is generated to obtain a preliminary intervention plan sequence.

[0098] The effect verification module is used to feed the initial intervention plan sequence back into the time series prediction model for simulation verification. It adjusts parameters based on the deviation of the simulation results, judges the intervention effect after optimization, and obtains the final behavioral intervention path.

[0099] In summary, this invention first captures the temporal dependencies in student behavior sequences using a recurrent neural network to generate behavioral evolution patterns. When samples are insufficient, data augmentation techniques are used to expand the dataset, ensuring the stability of model training. The trained temporal prediction model can extract feature vectors, accurately identify abnormal behavioral segments such as persistent inattentiveness, and analyze the impact of external factors such as teaching content on behavioral changes through a causal inference model, generating a causal relationship graph. Based on this graph, the invention identifies key intervention points, prioritizes adjustments to teaching factors to generate intervention scheme sequences, and optimizes the intervention effect through simulation verification, ultimately forming a precise behavioral intervention path. This invention significantly improves student classroom participation and the targeting and effectiveness of behavioral optimization, providing intelligent support for personalized education and forming a complete method for optimizing student voluntary learning behavior. The method progresses step-by-step from feature extraction to intervention verification, fully considering the complexity of classroom scenarios and providing an operable solution for educational practice.

[0100] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for analyzing dynamic volunteer learning behavior based on multimodal data using artificial intelligence, characterized by: S1. Collect multimodal behavioral sequence data, use recurrent neural networks to process temporal dependencies, and generate behavioral evolution patterns; S2. Extract a subset of labeled samples based on the behavioral evolution pattern, generate extended sequences through data augmentation techniques, and construct an augmented time-series dataset; S3. Train a recurrent neural network model using the enhanced time-series dataset, optimize long-term dependency parameters, and generate a time-series prediction model; S4. Extract feature vectors from the time series prediction model, calculate the deviation for abnormal behavior segments, and generate a set of abnormal behavior identifiers; S5. Using a set of abnormal behavior identifiers, a causal inference model is employed to analyze the correlation path between external factors and behavioral changes, generating a causal relationship map; S6. Generate a targeted adjustment sequence based on the causal relationship map to obtain a preliminary intervention plan sequence; S7. Feed the initial intervention plan sequence back to the time series prediction model for simulation verification, adjust the parameters according to the deviation of the simulation results, and generate the final behavioral intervention path after judging the optimization of the intervention effect. The process of collecting multimodal behavioral sequence data and using recurrent neural networks to process temporal dependencies to generate behavioral evolution patterns includes: S11. Obtain classroom participation records and changes in interaction frequency through sensor devices to form a time-series data set; S12. Input the time series data set into the recurrent neural network, output the sequence hidden state, process the time dependence, and generate a preliminary behavior pattern; the recurrent neural network processes the time dependence in the input data through its recurrent structure, and generates a hidden state sequence that can characterize the changes in student behavior. The hidden state sequence records the changing trend of student behavior in the time dimension. The changing trend is defined as the preliminary behavior pattern, which reflects the dynamic performance of students in the classroom. S13. Based on the initial behavioral patterns, combine educational interaction assessments to optimize the evolution trend and generate behavioral evolution patterns; The process of extracting a subset of labeled samples based on behavioral evolution patterns, generating extended sequences through data augmentation techniques, and constructing an augmented time-series dataset includes: S21. Select a subset of labeled samples based on the behavior evolution pattern, compare the number of subsets with a preset threshold, and determine whether enhancement is needed. S22. If the number of subsets is lower than a preset threshold, use sequence interpolation and noise injection to generate an extended sequence; S23. Integrate classroom feedback indicators and interaction frequency adjustments to generate an initial enhancement sequence; S24. By integrating the initial enhanced sequences and the original labeled sample subset, an enhanced time-series dataset is generated.

2. The method as described in claim 1, characterized in that, The process of training a recurrent neural network model using the enhanced time-series dataset, optimizing long-term dependency parameters, and generating a time-series prediction model includes: S31. Input the enhanced time-series dataset into a variant of the long short-term memory network to process long-term dependencies in behavioral sequences; S32. Update parameters iteratively using the gradient descent method and adjust network weights by incorporating classroom feedback indicators; S33. Combine sequence variation expansion to verify model convergence and generate time series prediction model.

3. The method as described in claim 1, characterized in that, The step of extracting feature vectors from the time-series prediction model, calculating deviations for abnormal behavior segments, and generating an abnormal behavior identifier set includes: S41. Extract feature vectors from the hidden layer of the time series prediction model and calculate the Euclidean distance bias for continuous distraction records; S42. If the deviation exceeds the preset threshold, mark it as an abnormal event and obtain the abnormal event time tag; S43. Generate a set of abnormal behavior identifiers containing timestamps and type labels based on the abnormal event labels.

4. The method as described in claim 1, characterized in that, The process involves analyzing the correlation path between external factors and behavioral changes using a causal inference model based on a set of abnormal behavior identifiers, generating a causal relationship graph, including: S51. Obtain external factor data through the abnormal behavior identifier set, and construct an initial causal graph using a structural causal model; S52. Calculate the conditional independence between variable nodes using statistical testing methods to generate preliminary association paths; S53. If the causal strength of the influence of the teaching content exceeds the preset threshold, determine the significant association path; S54. Calculate the impact of intervention variables through counterfactual reasoning to generate a complete causal relationship map.

5. The method as described in claim 1, characterized in that, The step of generating a targeted adjustment sequence based on the causal relationship map to obtain a preliminary intervention plan sequence includes: S61. Extract intervention variables based on the causal relationship map and analyze the weights of teaching factors; S62. If the weight of teaching factors is higher than the preset threshold, generate a targeted adjustment sequence; S63. By adjusting the sequence of intervention points, a preliminary intervention plan sequence is generated.

6. The method as described in claim 5, characterized in that, The step of extracting feature vectors from the hidden layer of the time-series prediction model and calculating the Euclidean distance bias for continuous attention loss records includes: S411. For abnormal behavior segments, calculate the feature vector deviation by combining cosine similarity; S412. Integrate multimodal data sequences to improve the robustness of abnormal behavior detection; S413. Based on the deviation results, generate a label indicating the severity of the abnormal behavior.

7. An AI-based dynamic volunteer learning behavior analysis system based on multimodal data, used to implement the method described in any one of claims 1-6, characterized in that, The system includes: The behavior sequence acquisition module is used to collect student behavior sequence data, including classroom participation records and changes in interaction frequency. It then uses a recurrent neural network to process temporal dependencies and obtain behavior evolution patterns, specifically including: S11. Obtain classroom participation records and changes in interaction frequency through sensor devices to form a time-series data set; S12. Input the time series data set into the recurrent neural network, output the sequence hidden state, process the time dependence, and generate a preliminary behavior pattern; the recurrent neural network processes the time dependence in the input data through its recurrent structure, and generates a hidden state sequence that can characterize the changes in student behavior. The hidden state sequence records the changing trend of student behavior in the time dimension. The changing trend is defined as the preliminary behavior pattern, which reflects the dynamic performance of students in the classroom. S13. Based on the initial behavioral patterns, combine educational interaction assessments to optimize the evolution trend and generate behavioral evolution patterns; The data augmentation module is used to obtain a subset of labeled samples based on behavioral evolution patterns. If the number of labeled samples is lower than a preset threshold, an extended sequence is generated using data augmentation techniques to determine the augmented time-series dataset. Specifically, it includes: S21. Select a subset of labeled samples based on the behavior evolution pattern, compare the number of subsets with a preset threshold, and determine whether enhancement is needed. S22. If the number of subsets is lower than a preset threshold, use sequence interpolation and noise injection to generate an extended sequence; S23. Integrate classroom feedback indicators and interaction frequency adjustments to generate an initial enhancement sequence; S24. By integrating the initially enhanced sequences and the original labeled sample subset, an enhanced time-series dataset is generated; The time series model training module is used to train a recurrent neural network model using an enhanced time series dataset, optimize parameters for long-term dependencies in behavioral sequences, and obtain a trained time series prediction model. The anomaly detection module is used to extract feature vectors from the trained time-series prediction model. For abnormal behavior segments, such as records of continuous inattentiveness, if the deviation of the feature vector exceeds a preset threshold, it is judged as an abnormal event, and an abnormal behavior identifier set is obtained. The causal analysis module is used to analyze the correlation path between external factors, such as teaching content, and behavioral changes through a set of abnormal behavior identifiers and a causal inference model, and to determine the causal relationship map. The intervention plan generation module is used to obtain intervention points based on the causal relationship graph. If the weight of teaching factors is higher than that of emotional factors in the causal path, a targeted adjustment sequence is generated to obtain a preliminary intervention plan sequence. The effect verification module is used to feed the initial intervention plan sequence back into the time series prediction model for simulation verification. It adjusts parameters based on the deviation of the simulation results, judges the intervention effect after optimization, and obtains the final behavioral intervention path.

Images