Intelligent teaching system based on reinforcement learning

Abstract

The invention provides an intelligent teaching system based on reinforcement learning, which comprises an enhanced LSTM model, a real-time feedback mechanism and a multi-modal data fusion module, and is characterized in that the enhanced LSTM model is used for improving capture of student learning behaviors and personalized teaching recommendation, and comprises an LSTM model and an attention mechanism; the real-time feedback mechanism is used for providing real-time dynamic teaching feedback for learning behaviors of students, the real-time feedback mechanism comprises a high-level strategy and a bottom-level strategy, and the multi-modal data fusion module is used for fusing text, voice and image modal data according to the real-time feedback and providing rich and natural learning experience. By introducing the enhanced LSTM model, the real-time feedback mechanism and the multi-modal data fusion module, the system is more personalized in personalized teaching and timely masters feedback novelty of students, teaching content is conveniently adjusted in real time, richer and more natural learning experience is provided through the multi-modal data fusion module, and the teaching experience is improved. And finally, the learning effect and satisfaction of students are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and relates to an intelligent teaching system and method based on reinforcement learning. Background Art

[0002] Although the existing intelligent teacher system can simulate some teacher behaviors, provide personalized teaching and feedback to students, help teachers reduce their burden, and improve teaching quality and efficiency. However, there are still deficiencies in personalized teaching and instant feedback, and it is unable to fully meet students' personalized learning needs. At present, the intelligent teacher system uses predefined rules and logic to provide personalized teaching for different students' learning behaviors, but due to the simplicity of such systems, they lack flexibility and adaptability; using machine learning algorithms, by analyzing students' learning data, dynamically adjusting teaching content and progress, such systems can provide more flexible personalized teaching, but have high requirements for data volume and quality; using deep learning models (such as RNN, LSTM, Transformer) to analyze students' learning behaviors, provide highly personalized teaching and feedback, but such systems are more complex to implement. Summary of the invention

[0003] The purpose of the present invention is to provide an intelligent teaching system based on reinforcement learning in view of the problems existing in the prior art.

[0004] To this end, the present invention adopts the following technical solutions: An intelligent teaching system based on reinforcement learning, characterized by comprising An enhanced LSTM model, wherein the enhanced LSTM model is used to improve the capture of student learning behaviors and personalized teaching recommendations, and the enhanced LSTM model includes an LSTM model and an attention mechanism; A real-time feedback mechanism, which is used to provide real-time dynamic teaching feedback on students' learning behaviors. The real-time feedback mechanism includes high-level strategies and low-level strategies; A multimodal data fusion module is used to fuse text, voice and image modality data according to real-time feedback to provide a rich and natural learning experience.

[0005] Furthermore, the algorithm model of the LSTM model is: in, f t is the activation vector of the forget gate, w f is the weight matrix of the forget gate, b f is the bias term of the forget gate, i t is the activation vector of the input gate,w i is the weight matrix of the input gate, b is the bias term of the input gate, o t is the activation vector of the output gate, w o is the weight matrix of the output gate, b o is the bias term of the output gate, c t is the current memory unit state, h t is the hidden state vector, a t is the attention weight vector, is the hidden state vector after the attention mechanism, tanh for tanh Activation function, σ for sigmoid The activation function is w c is the weight matrix associated with the unit state update, b c is the bias term related to the unit state update, and the mathematical model formula of the attention mechanism is: in: f ( Wx i +b ) is the sigmoid activation function, x i To input information i The characteristic vector of W、b are the weight parameter matrix and bias for adjusting feature weights, a i To input information i The corresponding weights of the features are Z Weighted sum of attention mechanism features.

[0006] Furthermore, the algorithm model of the high-level strategy is as follows: The algorithm model of the underlying strategy is as follows: in, π H ( s t ) is a high-level strategy, π L ( s t , at ) is the low-level strategy, Q H ( s t , a t ) for high-level Q Function value, Q L ( s t , a t ) is the lower layer Q Function value, s t is the time step t status, a t is the time step t Actions, r t is the time step t Rewards, is the discount factor.

[0007] Furthermore, the core algorithm model of the multimodal data fusion module is as follows: in X test Input feature vector for text, X avdio is the speech input feature vector, X vdio is the video input feature vector, h test For text hidden state, h avdio The voice is hidden. h vdio To hide the video, LSTM test To process text data LSTM network, LSTM avdio To process voice data LSTM Network, 3 D - CNN is a 3D convolutional neural network for processing video data. h fusion is the hidden state vector after multimodal feature fusion, w fusion is the weight matrix of the fusion layer, b fusion is the bias term of the fusion layer, softmax To generate the probability distribution of the final prediction results, y is the final output.

[0008] The beneficial effects of the present invention are: By introducing the enhanced LSTM model, real-time feedback mechanism and multimodal data fusion module, the system can make personalized teaching more personalized and timely grasp new types of student feedback, which is convenient for immediate adjustment of teaching content. Through the multimodal data fusion module, a richer and more natural learning experience is provided, which ultimately improves students' learning effect and satisfaction. DETAILED DESCRIPTION

[0009] The present invention is described in detail below in conjunction with embodiments: An intelligent teaching system based on reinforcement learning, including reinforcement LSTM model, real-time feedback mechanism and multimodal data fusion module, specifically, enhancing LSTM The model is used to improve the capture of students' learning behavior and personalized teaching recommendations, and enhance LSTM The model is in the original LSTM The attention mechanism is introduced on the basis of the model. The attention mechanism can better capture key learning behavior information, enhance the expressiveness of the LSTM model, dynamically adjust the teaching content and progress, improve the personalization and flexibility of teaching, achieve more personalized and effective teaching, and improve students' learning effects and satisfaction.

[0010] LSTM The core of the model is LSTM Memory unit, LSTM The memory unit includes a forget gate, an input gate, and an output gate, and the above three gates are composed of sigmoid Function and 1 dot product unit, LSTM The algorithm model of the model is: in, f t is the activation vector of the forget gate, w f is the weight matrix of the forget gate, b f is the bias term of the forget gate, i t is the activation vector of the input gate, w i is the weight matrix of the input gate, b is the bias term of the input gate, o t is the activation vector of the output gate, w o is the weight matrix of the output gate, b o is the bias term of the output gate, c t is the current memory unit state,h t is the hidden state vector, a t is the attention weight vector, is the hidden state vector after the attention mechanism, tanh for tanh Activation function, σ for sigmoid Activation function, the mathematical model formula of the attention mechanism is: in: f ( Wx i +b ) is the sigmoid activation function, x i To input information i The characteristic vector of W、b are the weight parameter matrix and bias for adjusting feature weights, a i To input information i The corresponding weights of the features are Z The weighted sum of the attention mechanism features (i.e., from the input information i To enter input information l weighted sum of ).

[0011] The enhanced LSTM model introduces the above attention mechanism LSTM In the model, the attention distribution formula is α i = softmax ( s ( x i , q )), where α 1 + α 2 +…… α n =1, where x i is the amount of input information, q is the query vector, α i For the i Input variables x i The corresponding attention distribution weight, s ( x i , q ) is the i The scoring function of the input variables is usually derived from the additive model, the dot product model and the bilinear model. The three models are: in, W and U are the learned network parameters, d is the dimension of the input information.

[0012] After obtaining the attention distribution, the input information is summarized using the weighted summation method to obtain the attention value: The attention value is the enhanced LSTM model.

[0013] The real-time feedback mechanism is used to provide real-time dynamic teaching feedback on students' learning behaviors. According to the learning behaviors of students during teaching, the teaching methods are adjusted in real time. The real-time feedback mechanism includes high-level strategies and low-level strategies. Hierarchical reinforcement learning can handle complex decision-making problems and adapt to students' changing learning status. High-level strategies manage the scheduling of the overall teaching module, and low-level strategies optimize specific teaching behaviors. It can optimize strategies more effectively to provide real-time and dynamic teaching feedback, improve the immediacy and effectiveness of teaching. Specifically, The algorithm model of the high-level strategy is as follows: The high-level strategy first sets one or more task sub-goals based on the overall teaching task, and considers the better decision direction in the current situation when setting the goal.

[0014] Then, according to the current situation and the set task sub-goals, a specific decision is made. The decision selects a sub-task to execute, or switches between multiple sub-tasks. During the decision-making process, the high-level strategy uses its own decision function (such as value function, policy network, etc.) to evaluate the expected benefits of different decisions and select the decision with the greatest benefit.

[0015] Finally, after the high-level strategy makes a decision, it generates feedback signals based on the actual performance of the low-level strategy. These feedback signals are based on the low-level strategy's completion of the subtask. The high-level strategy will use these feedback signals to update its own policy function so that it can set sub-goals and make decisions more accurately in the future.

[0016] The algorithm model of the high-level strategy is as follows: in, π H ( s t ) is a high-level strategy, Q H ( s t , a t ) for high-level QFunction value, s t is the time step t status, a t is the time step t action, then a t+1 is the time step t+ 1 action.

[0017] The bottom-level strategy must first receive the subtasks or options set by the high-level strategy as instructions.

[0018] Then, based on the instructions received, the underlying strategy selects a specific action to execute. During the action selection process, the underlying strategy will use its own strategy functions (such as action selection network, controller, etc.) to evaluate the effects of different actions and select the optimal action to execute.

[0019] Finally, the bottom-level strategy passes the feedback obtained after executing the action to the high-level strategy. These feedback signals are important bases for the high-level strategy to update its own strategy and set new subtasks. The bottom-level strategy updates its own strategy function based on the feedback signals obtained, and continuously learns and adjusts, so that the strategy gradually approaches the global optimal solution.

[0020] The algorithm of the underlying strategy is as follows: in, π L ( s t , a t ) is the low-level strategy, Q L ( s t , a t ) is the lower layer Q Function value, r t is the time step t Rewards, is the discount factor.

[0021] The multimodal data fusion module is used to combine text, voice and image modal data based on real-time feedback using multimodal deep learning methods to provide a rich and natural learning experience. This is because data in different modalities can provide richer learning information and improve the system's understanding and responsiveness. Multimodal fusion can enhance the interactivity and naturalness of the system, provide a richer and natural learning experience, and enhance students' learning interest and effectiveness.

[0022] The core algorithm model of the multimodal data fusion module includes LSTM layer, 3D-CNN layer and fusion layer. Specifically, The LSTM layer (hetert, haudio) is used to process data. In the LSTM layer, hetert is used to process text data. Text data usually appears in the form of a sequence, and meaningful features can be effectively extracted from the sequence. In the LSTM layer, haudio is used to process audio data. Audio is also a kind of sequence data. The sound wave signal that changes with time can be converted into a series of numerical representations. Haudio can capture the time dynamic characteristics of these sequences, thereby extracting useful information from the audio. The expression of this layer is: in, X test Input feature vector for text, X avdio is the speech input feature vector, h test For text hidden state, h avdio The voice is hidden. LSTM test To process text data LSTM network, LSTM avdio To process voice data LSTM network.

[0023] The 3D-CNN layer (hvideo) is used to process video data. Video data is essentially a temporal sequence of images. 3D-CNN can perform convolution operations in both spatial and temporal dimensions to capture complex features such as motion and scene changes in the video. The expression of this layer is: Among them, 3 D - CNN A 3D convolutional neural network for processing video data.

[0024] The fusion layer (hfusion) is used to fuse feature vectors from different modalities (text, audio, video). During the fusion process, the model needs to learn how to effectively integrate information from different sources in order to better complete downstream tasks (such as classification, regression, etc.). The expression of this layer is: in, h fusion is the hidden state vector after multimodal feature fusion, w fusion is the weight matrix of the fusion layer, bfusion is the bias term of the fusion layer, softmax To generate the probability distribution of the final prediction result.

[0025] The final output y is a probability distribution vector, each element of which corresponds to the predicted probability of a category. The final output is used for a variety of tasks, such as sentiment analysis (determining the type of sentiment expressed by text, audio or video), event classification (identifying the type of event in a video), etc. The Softmax function is used to convert the feature vector output by the fusion layer into a probability distribution, which represents the predicted probability of each category.

Claims

1. An intelligent teaching system based on reinforcement learning, characterized in that: include An enhanced LSTM model, wherein the enhanced LSTM model is used to improve the capture of student learning behaviors and personalized teaching recommendations, and the enhanced LSTM model includes an LSTM model and an attention mechanism; A real-time feedback mechanism, which is used to provide real-time dynamic teaching feedback on students' learning behaviors. The real-time feedback mechanism includes high-level strategies and low-level strategies; A multimodal data fusion module is used to fuse text, voice and image modality data according to real-time feedback to provide a rich and natural learning experience.

2. The intelligent teaching system based on reinforcement learning according to claim 1, characterized in that: The algorithm model of the LSTM model is: in, f t is the activation vector of the forget gate, w f is the weight matrix of the forget gate, b f is the bias term of the forget gate, i t is the activation vector of the input gate, w i is the weight matrix of the input gate, b is the bias term of the input gate, o t is the activation vector of the output gate, w o is the weight matrix of the output gate, b o is the bias term of the output gate, c t is the current memory unit state, h t is the hidden state vector, a t is the attention weight vector, is the hidden state vector after the attention mechanism, tanh for tanh Activation function, σ for sigmoid The activation function is w c is the weight matrix associated with the unit state update, b c is the bias term related to the unit state update, and the mathematical model formula of the attention mechanism is: in: f ( Wx i + b )for sigmoid Activation function, x i To input information i The characteristic vector of W, b are the weight parameter matrix and bias for adjusting feature weights, a i To input information i The corresponding weights of the features are Z Weighted sum of attention mechanism features.

3. The intelligent teaching system based on reinforcement learning according to claim 1, characterized in that: The algorithm model of the high-level strategy is as follows: The algorithm model of the underlying strategy is as follows: in, π H ( s t ) is a high-level strategy, π L ( s t , a t ) is the low-level strategy, Q H ( s t , a t ) for high-level Q Function value, Q L ( s t , a t ) is the lower layer Q Function value, s t is the time step t status, a t is the time step t Actions, r t is the time step t Rewards, is the discount factor.

4. The intelligent teaching system based on reinforcement learning according to claim 1, characterized in that: The core algorithm model of the multimodal data fusion module is as follows: in X test Input feature vector for text, X avdio is the speech input feature vector, X vdio is the video input feature vector, h test For text hidden state, h avdio The voice is hidden. h vdio To hide the video, LSTM test To process text data LSTM network, LSTM avdio To process voice data LSTM Network, 3 D - CNN is a 3D convolutional neural network for processing video data. h fusion is the hidden state vector after multimodal feature fusion, w fusion is the weight matrix of the fusion layer, b fusion is the bias term of the fusion layer, softmax To generate the probability distribution of the final prediction results, y is the final output.

Images