Student behavior and teaching strategy guidance method based on deep learning multi-dimensional data

Abstract

The invention relates to the technical field of education, in particular to a student behavior and teaching strategy guidance method based on deep learning multi-dimensional data. The method comprises the following steps: acquiring a time sequence behavior sequence, static characteristics and environment context data from multi-source heterogeneous data; generating a unified feature vector through data cleaning, feature extraction and multi-modal fusion; dynamically fusing the features by using a multi-modal deep learning model based on an attention mechanism and outputting a behavior pattern recognition result; and adopting a deep reinforcement learning algorithm to generate a personalized teaching strategy by taking an identification result as a state space. According to the method, efficient integration and deep analysis of multi-dimensional data are realized, a complex behavior mode can be accurately identified, a teaching strategy can be dynamically optimized, and the accuracy and adaptability of personalized teaching are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of education, more particularly, the present application relates to a student behavior and teaching strategy guidance method based on deep learning multi-dimensional data. BACKGROUND

[0002] With the continuous deepening of education informatization process, multi-dimensional collection and analysis of student behavior data has become an important basis for realizing personalized teaching. Traditional teaching evaluation methods often rely on single-dimensional test scores or subjective observation, which is difficult to fully reflect the learning state and development needs of students. The present application integrates multi-source heterogeneous data and uses advanced computing methods to provide a scientific basis for the accurate formulation of teaching strategies, which helps to improve the quality and efficiency of education and teaching.

[0003] The prior art has obvious deficiencies in student behavior analysis and teaching strategy guidance. First, the data collection dimension is relatively single, most systems can only process structured data such as academic performance or basic attendance records, and lack effective integration of unstructured data such as classroom interaction, physiological state, and environmental factors. Second, the data analysis method mostly uses traditional statistical models or shallow machine learning algorithms, which are difficult to capture the complex correlation and time-dependent relationship between multi-modal data. For example, some systems use simple regression analysis or decision tree models, which cannot handle non-linear patterns in high-dimensional feature space, resulting in limited behavior recognition accuracy. In addition, the existing method usually uses a static analysis framework, which lacks the ability to adapt to the dynamic changes in data distribution. When the student behavior pattern evolves over time, the model performance will decrease. In terms of strategy generation, most technologies rely on pre-defined rule bases or expert experience, which are difficult to dynamically adjust teaching interventions according to real-time behavior data, and often ignore the synergistic effects of cognitive state and emotional state. Finally, the privacy protection mechanism is not perfect, and there is a risk of sensitive information leakage in the data collection and processing process, which restricts the actual application range of the technology.

[0004] Therefore, in view of the above problems, a student behavior and teaching strategy guidance method based on deep learning multi-dimensional data is proposed, which solves the following core problems: how to realize efficient fusion and deep analysis of multi-dimensional student behavior data, how to accurately identify complex behavior patterns and their evolution rules, and how to dynamically generate personalized teaching strategies based on analysis results, while ensuring the safety and adaptability of the data processing process. SUMMARY

[0005] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide a student behavior and teaching strategy guidance method based on deep learning multi-dimensional data to solve the problems raised in the background art.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions: The application discloses a student behavior and teaching strategy guidance method based on deep learning multidimensional data. S1, continuously collecting multidimensional student behavior data from multiple heterogeneous data sources, wherein the multidimensional student behavior data comprises time sequence behavior sequence data, static feature data and environmental context data, and the collection frequency ranges from 1 to 60 times per minute; S2, performing a preprocessing operation on the collected multidimensional student behavior data, wherein the preprocessing operation comprises data cleaning, feature extraction and multimodal fusion, and a unified format multimodal feature vector is generated; S3, inputting the preprocessed multimodal feature vector into a multimodal deep learning model based on an attention mechanism, wherein the model dynamically weights the importance of different modal features through a cross-modal attention layer, and outputs a student behavior pattern recognition result containing a cognitive state index and an emotional state index; S4, generating an individualized teaching strategy guidance based on the student behavior pattern recognition result, wherein a deep reinforcement learning algorithm is used, the student behavior pattern recognition result is used as a state space, a teaching strategy action is used as an action space, and a policy gradient method is used to optimize policy network parameters.

[0007] Preferably, in step S1, the time sequence behavior sequence data comprises a classroom interaction frequency change sequence, a learning activity duration sequence and a physiological signal time sequence, the static feature data comprises feature information, historical academic performance and learning preference data, and the environmental context data comprises classroom temperature, illumination intensity and social network interaction density, wherein the physiological signal time sequence is collected by a wearable device, and the sampling frequency is 10-100 Hz.

[0008] Preferably, in step S2, the data cleaning comprises missing value processing and abnormal value filtering, wherein the missing value processing adopts a multiple imputation algorithm, and the abnormal value filtering is based on the 3σ principle, the feature extraction comprises time domain feature extraction and frequency domain feature extraction, the time domain features comprise mean and variance, the frequency domain features are obtained through fast Fourier transform, and the multimodal fusion adopts a feature level fusion manner to splice feature vectors of different modalities into a unified multimodal feature vector.

[0009] Preferably, in step S3, the multimodal deep learning model comprises a feature encoding layer, a cross-modal attention layer and an output layer, the feature encoding layer adopts a combined structure of a convolutional neural network and a gated recurrent unit, the convolution kernel size of the convolutional neural network is set to 3x3, the hidden layer dimension of the gated recurrent unit ranges from 128 to 512, the cross-modal attention layer realizes feature recalibration by calculating attention weights between different modal features, and the output layer adopts a softmax activation function to generate a behavior pattern classification result.

[0010] Preferably, the calculation process of the cross-modal attention layer comprises: firstly, calculating a query vector, a key vector and a value vector, wherein the query vector is from the feature encoding output of one modality, and the key vector and the value vector are from the feature encoding output of another modality; then calculating attention weights through a scaled dot-product attention mechanism, wherein a scaling factor in the scaled dot-product attention mechanism is set as the square root of the feature dimension; and finally, obtaining a cross-modal feature representation by weighted summation of the attention weights and the value vector.

[0011] Preferably, in step S4, the deep reinforcement learning algorithm adopts a proximal policy optimization algorithm, the proximal policy optimization algorithm comprises an actor network and a critic network, the actor network outputs a probability distribution of a teaching strategy action, the critic network evaluates a state value function, a clipping parameter epsilon set to 0.1-0.3 during policy updating, and a learning rate set to 0.0001-0.001, and the teaching strategy action comprises learning content adjustment, teaching rhythm control and intervention measure triggering.

[0012] Preferably, the actor network and the critic network adopt a multi-layer perceptron structure, the number of hidden layers is 3-5 layers, the number of neurons in each layer ranges from 256 to 1024, the activation function adopts a ReLU function, the network parameter initialization adopts an Xavier initialization method, the policy updating adopts a gradient ascent algorithm, and the sample batch size for each update is set to 32-128.

[0013] Preferably, the method further comprises step S5: presenting the individualized teaching strategy guidance in the form of an interactive dashboard through a visualization interface module, the interactive dashboard comprises real-time behavior monitoring charts, strategy execution effect comparison charts and historical trend analysis charts, supports data export and parameter adjustment functions, and the data update frequency is 1-10 times / second.

[0014] Preferably, the method further comprises a model updating module for periodically updating the multi-modal deep learning model, the update cycle is 7-30 days, the update process adopts an incremental learning mode, the model parameters are fine-tuned in combination with newly collected data, an elastic weight consolidation algorithm is used to prevent catastrophic forgetting, and a regularization coefficient lambda is set to 0.5-1.5.

[0015] Preferably, the method further comprises a privacy protection module, differential privacy technology is used in the data collection and transmission process, sensitive information is protected by adding Laplace noise, a privacy budget epsilon is set to 0.1-1.0, a federated learning framework is used in the model training process, and original data is retained locally and only model gradient update parameters are uploaded.

[0016] Technical effects and advantages of the present application: Compared with the prior art, the present application continuously collects time series behavior sequence data, static feature data and environmental context data from multiple heterogeneous data sources, and generates a unified format multi-modal feature vector by using a preprocessing process including data cleaning, feature extraction and multi-modal fusion. This process effectively retains the key information in the original data by using multiple imputation algorithms to handle missing values, filtering outliers based on the 3σ principle, and combining time domain and frequency domain feature extraction methods. This approach overcomes the limitations of traditional methods with single data dimension and insufficient feature expression capability, enabling more comprehensive and accurate student behavior representation, laying a solid foundation for subsequent in-depth analysis.

[0017] Compared with the prior art, the present application constructs a multi-modal deep learning model based on attention mechanism, uses a combination of feature encoding layer, cross-modal attention layer and output layer to dynamically weight the importance of different modal features. Specifically, convolutional neural networks and gated recurrent units are used to extract spatial local features and temporal dependencies, respectively. The cross-modal attention layer uses a scaled dot-product attention mechanism to recalibrate the features, and finally outputs the recognition results containing cognitive state and emotional indicators. This design solves the problem of traditional models that cannot effectively handle complex interactions between multi-modal data, improving the accuracy and robustness of behavior pattern recognition, and enhancing the model's adaptive judgment ability of feature importance.

[0018] Compared with the prior art, the present application uses a deep reinforcement learning algorithm, takes the student behavior pattern recognition result as the state space and the teaching strategy action as the action space, and realizes the iterative optimization of the strategy network based on the proximal policy optimization framework. The actor network outputs the policy probability distribution, the critic network evaluates the state value, and the parameters are updated through the gradient ascent algorithm, and the trust region constraint is combined to ensure the stability of the training. This method breaks through the limitations of traditional rule-based or static strategies, and can dynamically adjust the teaching content, pace and intervention measures according to real-time behavior data, realizing the continuous optimization and personalized adaptation of teaching strategies, effectively improving the adaptability and pertinence of the learning process. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 The system flowchart of the present application.

[0020] Fig. 2 The multi-modal deep learning model structure diagram of the present application.

[0021] Fig. 3 The reinforcement learning strategy generation mechanism diagram of the present application.

[0022] Fig. 4 The system auxiliary function and architecture diagram of the present application. DETAILED DESCRIPTION

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1 As attached Figs. 1 to 4 The student behavior and teaching strategy guidance method based on deep learning multi-dimensional data, as shown, is implemented as follows: This solution provides a student behavior and teaching strategy guidance method based on deep learning multi-dimensional data. This method constructs a complete data processing and analysis system. The core architecture of the system includes three key components: a multi-source data acquisition and preprocessing component responsible for acquiring raw information from heterogeneous data sources and standardizing it; a multimodal deep learning analysis component extracting deep behavioral features through a neural network model incorporating attention mechanisms; and an adaptive strategy generation component dynamically optimizing the teaching guidance scheme using reinforcement learning algorithms. These three components form a tightly integrated processing pipeline, achieving end-to-end transformation from raw data to teaching strategies.

[0025] In the basic infrastructure architecture, data acquisition is achieved through various sensing devices distributed throughout the teaching environment. Intel RealSense D455 depth cameras installed at the front and back of classrooms capture student behavior video at a resolution of 1280×800 and a frame rate of 30fps, accurately capturing students' body movements and facial expressions. Empatica E4 smart bracelets worn by students acquire PPG and EDA signals at a frequency of 4Hz, while simultaneously acquiring triaxial acceleration data at a frequency of 32Hz. The online learning platform obtains students' homework completion status and resource browsing history in real time through a RESTful API interface. This multi-source data is transmitted to the central processing server via the campus LAN, with the MQTT protocol ensuring real-time data transmission.

[0026] The data preprocessing stage is deployed on a server cluster equipped with the Apache Spark distributed computing framework. The raw data first passes through a quality inspection module to detect abnormal timestamps and missing data segments. For video data, the OpenPose library is used to extract the coordinates of human keypoints, converting each frame into a two-dimensional coordinate sequence of 18 keypoints. Physiological signal data undergoes bandpass filtering to remove baseline drift and high-frequency noise; PPG signals are passed through a 0.4-5Hz Butterworth filter, and EDA signals are passed through a 0.01-0.5Hz filter. In the feature extraction stage, time-domain, frequency-domain, and nonlinear features are calculated from the preprocessed signals, including a total of 125-dimensional feature vectors such as mean, standard deviation, sample entropy, and power spectral density.

[0027] The multi-modal deep learning model adopts a modular design and is implemented using the PyTorch 1.9 framework. The input layer of the model receives feature vectors from different modalities and is preliminarily processed by independent feature encoding sub-networks. The visual feature encoding uses a convolutional neural network with a ResNet34 architecture, the physiological feature encoding uses a bidirectional GRU network, and the learning behavior feature is encoded using a fully connected network. The model training adopts a phased strategy, first pre-training the modal encoder, and then jointly training the entire network. The Adam optimizer is used during training, with an initial learning rate of 0.001, and a cosine annealing strategy is used to adjust the learning rate.

[0028] The strategy generation system is based on the Proximal Policy Optimization (PPO) algorithm and consists of two core components: the actor network and the critic network. The actor network receives the behavior pattern recognition results as input and outputs a 15-dimensional action probability distribution corresponding to different teaching strategy choices. The critic network evaluates the value function of the current state to provide a benchmark for policy updating. The system updates the strategy every 24 hours, sampling 256 experience trajectories from the experience replay buffer and calculating the advantage function using Generalized Advantage Estimation (GAE).

[0029] Furthermore, in the refinement of the data collection process, eye movement trajectory data is collected using the Tobii ProFusion eye tracker with a sampling frequency of 60Hz, which accurately records gaze point coordinates and pupil diameter changes. The device is installed about 65cm in front of the classroom, ensuring that the visual angle covers the entire teaching area. Speech activity data is collected by deploying a ReSpeaker microphone array on the desk, with a sampling rate of 16kHz and 16-bit quantization. The opensmile toolkit is used to extract speech features, calculating fundamental frequency, energy, and spectral features every 2 seconds.

[0030] The collection of environmental context data is achieved through a sensor network distributed in the classroom. The temperature sensor DS18B20 measures in the range of -10℃ to 85℃ with an accuracy of ±0.5℃. The light sensor BH1750 measures in the range of 0-65535lx with a resolution of 1lx. The carbon dioxide concentration is monitored using the SGP30 sensor, measuring in the range of 400-60000ppm. These environmental parameters are recorded every 10 seconds, aligned with the behavior data timestamp, providing data support for analyzing the impact of environmental factors on learning effectiveness.

[0031] The collection of learning preference data is achieved by a standardized learning style questionnaire, which is designed based on the Felder-Silverman learning style model, containing 11 questions in four dimensions of information processing, perception, input and understanding. Each question uses a Likert 5-point scale, from "completely disagree" to "completely agree". The questionnaire data is collected through a web interface and associated with sensor data through student ID to build a complete student profile.

[0032] Further, in the in-depth optimization of data preprocessing, the data smoothing process uses the Savitzky-Golay filter, which preserves the overall shape characteristics of the signal by polynomial fitting within a sliding window. For a window length of , the filter output is calculated by the following formula: where is the original signal, is the convolution coefficient derived from the least squares polynomial fitting. In this implementation, the window length is set to 11 and the polynomial order is 3. This configuration can smooth the noise while preserving important peaks of behavioral characteristics.

[0033] The missing value processing uses a multiple imputation algorithm based on k-nearest neighbors. For samples containing missing values, the algorithm finds the k most similar complete samples in the feature space, and generates multiple imputation values by weighted averaging the values of these similar samples. The specific calculation is: where the weight is determined by the inverse distance between samples, . This method is particularly suitable for physiological signal data and can maintain the statistical properties of data distribution.

[0034] Nonlinear feature extraction uses the multiscale entropy algorithm, which analyzes the complexity of time series at different scales to reveal the dynamic characteristics of the system. For a given time series , first construct a coarse-grained sequence: Then calculate the sample entropy at each scale . Multiscale entropy features can effectively distinguish different physiological states, such as stress level and cognitive load.

[0035] Further, in the detailed construction of the multi-modal deep learning model, the feature normalization layer uses the Layer Normalization method, which standardizes all feature dimensions of each sample. The calculation formula is: where and are the mean and standard deviation of all features of the layer respectively, is a small constant added for numerical stability. Compared with Batch Normalization, Layer Normalization is more suitable for handling variable-length sequences and is not sensitive to batch size.

[0036] The cross-modal attention layer adopts a multi-head attention mechanism. The calculation process of each attention head is as follows: where , , represent the query, key, and value matrices respectively, is the dimension of the key vector. The output of multi-head attention is the concatenation of the outputs of each attention head: This design allows the model to jointly focus on information from different positions in different representation subspaces, enhancing the model's ability to capture complex relationships between multiple modalities.

[0037] The update mechanism of the Gated Recurrent Unit (GRU) controls information flow through the reset gate and the update gate. The specific calculation is as follows: where controls the proportion of historical information retained, controls the influence of historical information on the current candidate state. This gating mechanism enables the model to effectively learn long-term dependencies, making it particularly suitable for processing time series data of student behavior.

[0038] Furthermore, in the optimization implementation of the cross-modal attention mechanism, the application of residual connection solves the problem of gradient vanishing in deep network training. The output calculation of the residual block is as follows: where represents the output of the attention layer or the feedforward network layer. This design ensures that information can be effectively propagated in deep networks, allowing the model to be extended to more layers.

[0039] The causal mask in attention calculation ensures that the model can only access current and previous time step information when processing time series data, preventing future information leakage. The definition of the mask matrix is as follows: The mask matrix is added to the attention weight matrix before softmax calculation: This mechanism ensures the practicability of the model in online prediction scenarios.

[0040] The layer normalization operation adopts the RMSNorm variant, which has lower computational complexity than the standard LayerNorm. The calculation formula is: where is a learnable gain parameter. RMSNorm improves computational efficiency while maintaining performance, making it particularly suitable for processing high-dimensional features.

[0041] Furthermore, in the perfect design of deep reinforcement learning systems, the Proximal Policy Optimization (PPO) algorithm limits the magnitude of policy updates by clipping the probability ratio. The objective function is defined as: where represents the probability ratio, is the advantage function estimate, is a hyperparameter set to 0.2. This design ensures training stability while improving sample efficiency.

[0042] Generalized Advantage Estimation (GAE) is used to more accurately estimate the advantage function, with the calculation formula: where is a discount factor set to 0.99, is a parameter that balances bias and variance set to 0.95. GAE reduces the variance of advantage estimation by combining multi-step return information.

[0043] The experience replay mechanism uses the Prioritized Experience Replay (PER) strategy, and the sampling probability of each experience is based on its TD error: where controls the priority, is the TD error of the experience. This mechanism enables the model to learn more from experiences with higher learning value, improving training efficiency.

[0044] Further, in the fine-tuning of network structure, the actor-critic network adopts a dual-branch architecture with a shared encoder. The shared encoder consists of 3 fully connected layers with 256 neurons each, using ReLU activation functions. The output layer of the actor network uses a softmax activation function to generate a probability distribution over actions: where is the output of the actor network corresponding to action . The critic network outputs a single scalar value representing the expected cumulative reward for state .

[0045] Batch normalization layers use different calculations during training and inference. During training, the statistics of the current batch are used: During inference, moving average statistics are used: This design ensures consistency in behavior during training and inference.

[0046] Gradient clipping stabilizes the training process by limiting the gradient norm, as follows: where threshold is set to 1.0. This mechanism prevents gradient explosion, which is particularly important when training deep neural networks.

[0047] Further, in the complete implementation of the visualization interface module, the front-end architecture uses the Vue.js 3.0 combined API design, combined with TypeScript to provide type safety. The visualization component uses the ECharts 5.0 library, supporting efficient rendering of real-time data streams. The behavior monitoring dashboard includes a heat map component, using the D3.js library to draw the position distribution and movement trajectory of students in the classroom, with an update frequency of 2 times per second.

[0048] The strategy execution effect analysis module uses a comparative analysis method to compare the current policy effect with historical benchmarks. Effectiveness evaluation indicators include learning efficiency improvement rate, knowledge mastery consolidation, and student engagement. Each dimension is visualized through a radar chart, supporting multi-time scale comparative analysis, including day-on-day, week-on-week, and month-on-month comparisons.

[0049] The data export service is implemented as a RESTful API, supporting filtering by time range, data type, and granularity. CSV format exports contain complete raw data and derived features, while JSON format exports contain structured event sequences and metadata. Export tasks are processed asynchronously through a Celery distributed task queue, supporting batch export of large volumes of data.

[0050] The parameter configuration interface uses responsive design and can be used normally on desktop and mobile terminals. Configuration items are organized in a tree structure, including data acquisition parameters, model parameters, and strategy parameters. Each parameter provides detailed tooltips and value range explanations to reduce the user's threshold.

[0051] Furthermore, in the advanced features of the model update module, the incremental learning algorithm uses the Elastic Weight Consolidation (EWC) method, which protects important parameters by adding a regularization term to the loss function: where is the Fisher information matrix diagonal element, measuring the importance of parameter to the old task, is the regularization coefficient set to 0.8. This method effectively alleviates the catastrophic forgetting problem.

[0052] The implementation of the cosine annealing learning rate scheduler is based on the following formula: where , , is the number of iterations for half a cycle. This scheduling method helps the model jump out of local optimal solution and improves the convergence quality.

[0053] The model performance monitoring system tracks multiple indicators, including accuracy, F1 score, AUC value, and inference delay. When detecting model performance degradation, the system automatically triggers the rollback mechanism to restore the last stable version and starts the diagnosis process to analyze the cause of performance degradation.

[0054] Furthermore, in the comprehensive deployment of the privacy protection module, the implementation of the differential privacy mechanism is based on the Laplace mechanism, with the noise addition formula: where the sensitivity is automatically calculated according to the query type, and the privacy budget is configured hierarchically, set to 0.1 for sensitive data and 0.5 for ordinary data. This hierarchical protection ensures privacy while optimizing data utility.

[0055] The federated learning framework employs the FedAvg algorithm, with each round of training comprising the following steps: first, randomly select 10% of the clients to participate in training; then distribute the global model to each client; each client trains for 3 epochs using local data; finally, aggregate the model updates: where is the data volume of client , is the total data volume. This distributed training mode ensures that the original data does not leave the local.

[0056] The secure aggregation protocol employs the Paillier homomorphic encryption system, supporting model aggregation in ciphertext state. The encryption process is: where is the plaintext message, is a random number, is the public key. This encryption method allows model update calculations without decryption, providing end-to-end privacy protection.

[0057] To fully demonstrate the system workflow, the following details are combined with the specific application scenario of the junior high school mathematics wisdom classroom. This scenario is deployed in a class environment with 45 people, with a teaching cycle of one month, aiming to improve the effectiveness of function concept teaching through data analysis.

[0058] In the system initialization phase, data collection parameters are first configured. The visual sensor is set to detect the entire classroom, focusing on the whiteboard area and student seating area. The smart bracelet performs individual calibration and records baseline physiological data. The learning platform interface configures the course structure and knowledge point map, establishing semantic associations for teaching content.

[0059] In the daily operation phase, the system collects approximately 2.3GB of raw data each day. The visual data processing flow first uses the YOLOv5 model to detect human bounding boxes, then extracts skeletal key points through the AlphaPose model. For each detected student, 13 behavior indicators are calculated, including gaze direction change rate, head posture angle, and hand gesture activity frequency. These indicators are aggregated in 5-second windows to generate daily behavior sequences for 1440 time points.

[0060] Physiological signal processing employed a hierarchical analysis approach. First, peak detection was performed on the PPG signal, and time-domain indices (SDNN, RMSSD) and frequency-domain indices (LF, HF) of heart rate variability (HRV) were calculated. The EDA signal underwent phasic component extraction, and the amplitude of skin conductance responses triggered by specific events was statistically analyzed. Accelerometer data was classified into six states—resting, writing, raising hands, etc.—using an activity recognition model, with the state distribution output every 30 seconds.

[0061] Environmental and behavioral data are spatiotemporally aligned. Parameters such as temperature and illumination are unified to the same timestamp through interpolation methods to establish a correlation model between environmental factors and learning behavior. When the light intensity is detected to be below 300 lx, the system automatically records environmental factors that may affect learning performance.

[0062] The inference process of a multimodal deep learning model includes three steps: feature encoding, attention fusion, and pattern recognition. Taking data from a student's second class in the morning as an example, the visual feature encoder outputs a 256-dimensional feature vector, the physiological feature encoder outputs a 128-dimensional feature vector, and the learning behavior encoder outputs a 64-dimensional feature vector. These features enter the cross-modal attention layer to calculate a weighted multimodal representation.

[0063] The calculation process for attention weights is as follows: Query matrix From visual features, key matrix Sum matrix Based on physiological characteristics. First, calculate the attention score: in Then apply the softmax function and causal mask: Finally, the weighted feature representation is obtained: This process is performed in parallel in each attention head, and the outputs of all heads are finally spliced ​​together.

[0064] The input to the pattern recognition layer is a 512-dimensional fusion feature, which is mapped to three core indicators through a three-layer fully connected network: attention index (0-100), knowledge mastery (0-1), and emotional state score (-1 to 1). These indicators are updated every 5 minutes, forming the trajectory of the student's state evolution during the course.

[0065] The reinforcement learning policy generator is based on the current state vector. Select teaching actions. The actor network outputs a probability distribution of 15 actions, with the first 5 actions corresponding to content adjustments (e.g., simplifying examples, adding exercises), the middle 5 actions corresponding to interaction methods (e.g., individual guidance, group discussion), and the last 5 actions corresponding to progress control (e.g., acceleration, pause, repetition). The critic network evaluates the state value Provide a benchmark for policy updates.

[0066] During policy execution, the system prioritizes the action with the highest probability, but when that action is repeated with recent actions, it considers suboptimal actions to maintain policy diversity. For example, when detecting similar patterns of decreased focus among multiple students, the system may trigger a whole-class interactive activity instead of individual guidance.

[0067] Effect evaluation compares the state changes before and after policy execution. The focus index is calculated based on gaze persistence and saccade speed from eye movement data, knowledge mastery is evaluated through correct rate and response time of real-time tests, and emotional state is comprehensively judged through facial expression recognition and voice emotion analysis. These evaluation results are fed back to the reinforcement learning system to update the value function and policy network.

[0068] In a month of application period, the system made 62 policy adjustments and generated 285 personalized learning paths. The final evaluation showed that the average score of function concept understanding test and learning interest survey score of the experimental class increased, proving that the system improved the teaching effect.

[0069] The whole system realizes a complete closed loop from data collection to policy optimization through the organic combination of multi-modal data fusion, deep learning analysis and reinforcement learning decision-making. The close cooperation between various modules ensures that the system can accurately understand the learning state of students and adjust the teaching strategy in a timely manner, providing reliable technical support for personalized education.

[0070] Finally, it should be pointed out that in the description of the present application, it should be pointed out that unless otherwise specified and limited, the terms "installation", "connection", "connection" should be broadly understood, which can be mechanical connection or electrical connection, or the communication between two elements, or direct connection, "up", "down", "left", "right" and the like are only used to indicate the relative positional relationship, when the absolute position of the described object changes, the relative positional relationship may change; Secondly: the disclosure of the present application embodiment only involves the structure involved in the disclosure of the present application embodiment, other structures can refer to the usual design, under the condition of no conflict, the same embodiment and different embodiments of the present application can be combined with each other; Finally: the above only for the preferred embodiments of the present application, and not for limiting the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application, should be included in the scope of protection of the present application.

Claims

1. A method for guiding student behavior and teaching strategies based on multi-dimensional data from deep learning, characterized in that: The method includes the following steps: S1. Continuously collect multi-dimensional student behavior data from multiple heterogeneous data sources. The multi-dimensional student behavior data includes time-series behavior sequence data, static feature data, and environmental context data. The collection frequency range is 1-60 times / minute. S2. Perform preprocessing operations on the collected multi-dimensional student behavior data. The preprocessing operations include data cleaning, feature extraction, and multimodal fusion to generate multimodal feature vectors in a unified format. S3. Input the preprocessed multimodal feature vector into a multimodal deep learning model based on an attention mechanism. The model dynamically weights the importance of different modal features through a cross-modal attention layer and outputs student behavior pattern recognition results that include cognitive state indicators and emotional state indicators. S4. Based on the student behavior pattern recognition results, a deep reinforcement learning algorithm is used to generate personalized teaching strategy guidance. The deep reinforcement learning algorithm uses the student behavior pattern recognition results as the state space and the teaching strategy actions as the action space, and optimizes the policy network parameters through the policy gradient method.

2. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, In step S1, the time-series behavioral sequence data includes a classroom interaction frequency change sequence, a learning activity duration sequence, and a physiological signal time series. The static feature data includes feature information, historical academic performance, and learning preference data. The environmental context data includes classroom temperature, light intensity, and social network interaction density. The physiological signal time series is collected through a wearable device with a sampling frequency of 10-100Hz.

3. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, In step S2, the data cleaning includes missing value processing and outlier filtering. Missing value processing uses a multiple interpolation algorithm, and outlier filtering is based on the 3σ principle. The feature extraction includes time-domain feature extraction and frequency-domain feature extraction. The time-domain features include mean and variance, and the frequency-domain features are obtained through fast Fourier transform. The multimodal fusion adopts a feature-level fusion method, which concatenates the feature vectors of different modalities into a unified multimodal feature vector.

4. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, In step S3, the multimodal deep learning model includes a feature encoding layer, a cross-modal attention layer, and an output layer. The feature encoding layer adopts a combination structure of convolutional neural network and gated recurrent unit. The convolutional kernel size of the convolutional neural network is set to 3×3, and the hidden layer dimension of the gated recurrent unit is in the range of 128-512. The cross-modal attention layer realizes feature recalibration by calculating the attention weights between features of different modalities. The output layer uses the softmax activation function to generate behavior pattern classification results.

5. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 4, characterized in that, The calculation process of the cross-modal attention layer includes: first, calculating the query vector, key vector, and value vector, where the query vector comes from the feature encoding output of one modality, and the key vector and value vector come from the feature encoding output of another modality; then, calculating the attention weights through a scaled dot product attention mechanism; finally, weighted summing of the attention weights and the value vectors to obtain the cross-modal feature representation, where the scaling factor in the scaled dot product attention mechanism is set to the square root of the feature dimension.

6. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, In step S4, the deep reinforcement learning algorithm employs a proximal policy optimization algorithm, which includes an actor network and a critic network. The actor network outputs the probability distribution of teaching strategy actions, and the critic network evaluates the state value function. The pruning parameter ε during policy update is set to 0.1-0.3, and the learning rate is set to 0.0001-0.

001. The teaching strategy actions include adjusting learning content, controlling teaching pace, and triggering intervention measures.

7. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 6, characterized in that, The actor network and critic network adopt a multilayer perceptron structure with 3-5 hidden layers and 256-1024 neurons per layer. The activation function is ReLU, the network parameters are initialized using the Xavier initialization method, the policy update uses the gradient ascent algorithm, and the sample batch size for each update is set to 32-128.

8. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, The method further includes step S5: presenting personalized teaching strategy guidance in the form of an interactive dashboard through a visualization interface module. The interactive dashboard includes real-time behavior monitoring charts, strategy execution effect comparison charts, and historical trend analysis charts, and supports data export and parameter adjustment functions. The data update frequency is 1-10 times / second.

9. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, The method also includes a model update module for periodically updating the multimodal deep learning model. The update cycle is 7-30 days. The update process adopts an incremental learning approach, which combines newly collected data to fine-tune the model parameters. At the same time, an elastic weight consolidation algorithm is used to prevent catastrophic forgetting. The regularization coefficient λ is set to 0.5-1.

5.

10. The method for guiding student behavior and teaching strategies based on deep learning multi-dimensional data according to claim 1, characterized in that, The method also includes a privacy protection module, which employs differential privacy technology during data acquisition and transmission, protects sensitive information by adding Laplacian noise, sets the privacy budget ε to 0.1-1.0, and uses a federated learning framework during model training, keeping the original data locally and uploading only the model gradient update parameters.

Images