An online training education information teaching method based on big data

Abstract

The present invention relates to the field of online education technology and discloses an online training education informationization teaching method based on big data, comprising the following steps: Step S1: collecting data from an online teaching platform, a real-time assessment system, an emotion monitoring device, and user interaction logs, and performing cleaning, standardization, noise filtering, and feature extraction on the data; Step S2: constructing a student state vector based on the data from Step S1 to implement dimensionality reduction processing; and Step S3: establishing a state-action model. A personalized teaching recommendation system based on big data is adopted. By collecting student learning data in real time and utilizing big data technology to analyze and optimize learning paths, the system achieves the technical effect of accurately matching student needs. Compared to the technical solutions in traditional education platforms that cannot respond to student status in a timely manner, the present invention can adjust recommended content in real time, solving the problem of inaccurate allocation of personalized teaching resources.

Description

Technical Field

[0001] The present invention relates to the field of online education technology, and specifically to an online training education informationization teaching method based on big data. Background Art

[0002] The rapid development of online education, especially the rise of big data-based teaching systems, has presented numerous challenges for traditional online education platforms. These platforms often fail to fully leverage students' real-time data for personalized recommendations, and the delivery of instructional content suffers from inaccurate and delayed delivery. Traditional teaching methods often rely on fixed course designs and instructional processes, making it difficult to adapt them to students' learning progress, emotional state, and feedback. Consequently, the system is unable to tailor instructional resources to students' specific needs, which in turn impacts learning outcomes.

[0003] Existing online education platforms, especially traditional ones, often rely on static teaching plans. These methods are unable to flexibly respond to students' diverse needs, making it difficult to accurately match teaching content with students' actual situations, resulting in wasted resources and reduced learning outcomes.

[0004] Many existing teaching platforms lack effective feedback mechanisms and are unable to respond to students' needs in a timely manner. Student feedback cannot directly influence the adjustment of teaching content, resulting in problems encountered by students during the learning process not being resolved in a timely manner, making the recommendation of personalized teaching resources out of sync with students' actual needs.

[0005] Most traditional education platforms lack intelligent mechanisms for adjusting teaching strategies. While data on students' learning status and behavior can be collected and analyzed to a certain extent, dynamically adjusting teaching strategies based on this data remains a challenge in practical applications. Many platforms still rely on simple rule-based systems and static models when processing data, failing to make sufficiently intelligent recommendations and dynamic optimization, effectively improving teaching quality and student learning outcomes.

[0006] Therefore, those skilled in the art provide an online training education information-based teaching method based on big data to solve the above-mentioned problems. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the present invention provides an online training education information teaching method based on big data to solve the problems raised in the above background technology.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: an online training education informationization teaching method based on big data, comprising the following steps:

[0009] Step S1: Collect data from the online teaching platform, real-time evaluation system, emotion monitoring equipment, and user interaction logs, and perform data cleaning, standardization, noise filtering, and feature extraction;

[0010] Step S2: constructing a student state vector based on the data in step S1 to implement dimensionality reduction processing;

[0011] Step S3: Establish a state-action model, define the video resources, topic resources, and discussion modules corresponding to the recommended actions, and design a reward function to calculate the reward score;

[0012] Step S4: The teaching recommendation problem is abstracted into a Markov decision process, which consists of a state set, an action set, a state transition probability function, and an immediate reward function. The Bellman optimality principle is used to solve the optimal state value function.

[0013] Step S5: Using a deep Q-network to implement reinforcement learning training on the Markov decision process in step S4, so as to obtain the optimal recommendation strategy by iteratively updating the state-action-value function;

[0014] Step S6: Construct a dynamic game model to describe the interaction between the system and students. The system rewards and student rewards are expressed in the form of cumulative discount rewards and meet the Nash equilibrium conditions.

[0015] Step S7: Integrate steps S1 to S6 to form a closed-loop feedback system, collect feedback data in real time, and adjust various parameters to verify system performance.

[0016] Preferably, the data collected in step S1 include student login data, page click data, video viewing time data and question answering accuracy data in the online teaching platform, periodic performance data in the real-time evaluation system, emotion scoring data in the emotion monitoring device, and discussion record data in the user interaction log.

[0017] Preferably, the step S2 further comprises:

[0018] Step 2.1: Obtain the original feature data of each student from step S1 to form a state vector X, where each row corresponds to a student and the columns are the learning progress component, assessment score component, emotion score component, and interaction frequency component.

[0019] Step 2.2: Normalize each feature in the state vector X according to the following formula to obtain the normalized data matrix x′: (ij) =(x (ij) -μ j ) / σ j ,

[0020] Among them, x (ij) is the original value of the i-th student on the j-th feature, μ jis the mean of the jth characteristic among all students, σ j is the standard deviation of the jth characteristic among all students,

[0021] i = 1, 2, ..., n and j = 1, 2, ..., d, where n is the total number of students and d is the dimension of the initial state vector.

[0022] Preferably, the step S2 further comprises:

[0023] Step 2.3, use the normalized data matrix x′ to calculate the covariance matrix S. The calculation formula is:

[0024]

[0025] in, is the transpose of x′, n is the total number of students;

[0026] Step 2.4, perform eigendecomposition on the covariance matrix S and solve the eigenvalues ​​λ1,λ2,…,λ d and the corresponding eigenvectors v1,v2,…,v d ,satisfy:

[0027] S·v j =λ j ·v j , j=1,2,...,d,

[0028] Among them, λ j is the j-th eigenvalue of the covariance matrix S, v j is the j-th eigenvector of the covariance matrix S;

[0029] Step 2.4, select the eigenvectors corresponding to the first k largest eigenvalues ​​according to the eigenvalue size to form the projection matrix W;

[0030] Step 2.5: Use the projection matrix W to reduce the dimension of the normalized data matrix x′ to obtain the reduced-dimensional state vector matrix Z. The calculation formula is: Z = x′·W,

[0031] Among them, Z is the state vector matrix after dimensionality reduction.

[0032] Preferably, the step S3 further comprises:

[0033] Step 3.1, establish a state-action model, and define the state vector according to the student state vector constructed in step S2 To describe students' learning progress, assessment scores, emotional state and interactive behavior; define the recommended action set U, which contains multiple recommended actions;

[0034] Step 3.2: Design a reward function, which includes learning progress rewards, knowledge mastery rewards, and emotional motivation rewards. The specific calculation formula is:

[0035] r (p) (ξ q ,v q )=f (p) (ξ q ,v q ),

[0036] r (k) (ξ q ,v q )=f (k) (ξ q ,v q ),

[0037] r (e) (ξ q ,v q )=f (e) (ξ q ,v q ),

[0038] Among them, r (p) (ξ q ,v q ) is the learning progress reward, r (k) (ξ q ,v q ) is the knowledge mastery reward, r (e) (ξ q ,v q ) is the emotional incentive reward, ξ q is the student's state vector, v q is the recommended action, f (p) 、f (k) and f (e) A function for calculating various rewards;

[0039] Step 3.3: Calculate the cumulative reward. To evaluate the long-term effect of the recommended action, use the discount factor δ to reduce and accumulate the immediate rewards to obtain the overall reward. The overall reward function is:

[0040]

[0041] Among them, ω p is the learning progress reward, ω k Is the knowledge mastery reward, ω e is the weight coefficient of the emotional incentive reward, δ is the discount factor, T is the number of time steps, and J(π) is the total reward;

[0042] Step 3.4: The system determines the optimal recommended action v based on the cumulative reward J(π) qto maximize the reward value.

[0043] Preferably, the step S4 further comprises:

[0044] Step 4.1: Model the teaching recommendation problem as a Markov decision process, defining the state set S, the action set A, the state transition probability function P, and the immediate reward function R;

[0045] The state set S represents all states of students;

[0046] The action set A contains all the actions recommended by the system for students;

[0047] The state transition probability function P(s′|s,a) represents the probability of transitioning to state s′ after taking action a in state s;

[0048] The immediate reward function R(s,a) represents the immediate reward obtained after taking action a in state s.

[0049] Preferably, the step S4 further comprises:

[0050] Step 4.2, use Bellman's optimality principle to solve the optimal state value function V * (s), which represents the maximum cumulative reward for taking the optimal strategy in state s. The recursive form of the Bellman equation is:

[0051]

[0052] Among them, V * (s) is the optimal value function of state s, a is the currently available action,

[0053] R(s,a) is the immediate reward after taking action a in state s, s′ is the next state,

[0054] P(s′|s,a) is the probability of transitioning to state s′ after taking action a in state s.

[0055] γ is the discount factor;

[0056] Step 4.3, update the Bellman equation by iteration until the optimal state value function V * (s) converges. During each iterative update, the value of the next state is calculated based on the combination of the current state s and action a, and finally the optimal strategy and optimal state value function are obtained.

[0057] Preferably, the step S5 further comprises:

[0058] Step 5.1: After obtaining the optimal state value function through the Markov decision process in step S4, the deep Q network is used to approximate the optimal value function. The Q network is defined as a neural network whose input is the state vector s q , the output is each action a q The corresponding Q value Q(s q ,a q );

[0059] The Q value represents the q Next, perform action a q Expected cumulative rewards after

[0060] Step 5.2: During the training process, the Q-value function is updated based on the current Q-network by interacting with the environment. During each update, the current Q-value is updated by calculating the actual reward and the Q-value of the next state. The Bellman equation is used to update the Q-value. The formula is:

[0061]

[0062] Among them, Q(s q ,a q ) is the current state s q Next, perform action a q Q value,

[0063] α is the learning rate, r q is the current state s q Next, perform action a q After receiving the instant reward,

[0064] γ is the discount factor, is the next state s q+1 The maximum Q value under .

[0065] Preferably, the step S5 further comprises:

[0066] Step 5.3, to increase the stability of the training process, the deep Q network usually uses the target network to calculate the target Q value. The parameters of the target network θ - It is regularly copied from the parameters θ of the main Q network, and the update formula is:

[0067] θ - ←θ, the update of the target network can avoid large fluctuations during training;

[0068] In step 5.4, the deep Q network adjusts the network parameters θ through multiple training cycles to reduce the error between the Q value prediction and the actual reward, so that the Q network can accurately predict the Q value of each state-action pair and finally obtain the optimal strategy.

[0069] Preferably, step S6 further includes:

[0070] Step 6.1: Model the interaction between the system and students as a dynamic game model to describe the interaction process between the system and students. The dynamic game model includes the system's rewards and the students' rewards.

[0071] Step 6.2, define the reward functions of the system and the student. The reward functions of the system and the student are expressed by the following formula:

[0072] System Rewards s : Measures the effectiveness of the system's recommended actions;

[0073] Student Rewards st : reflects students’ feedback on recommended actions;

[0074] Step 6.3, Nash equilibrium condition. To ensure the stability of the strategies in the game, the strategies of the system and the student must satisfy the Nash equilibrium condition. Given the strategy of the other party, neither party has the motivation to unilaterally change its own strategy. The Nash equilibrium condition is expressed as:

[0075]

[0076] Among them, r s is the reward function of the system, r st is the student’s reward function, a s is the recommended action of the system, a st is the student's reaction action, E[r s (a s ,a st )] is the expected value of the system reward, E[r st (a s ,a st )] is the expected value of the student’s reward.

[0077] Step 6.4: In the dynamic game, the system and the student optimize their respective behaviors through dynamic game strategies based on their own reward functions. The system's strategy π s Strategies for students st All of them can be optimized through the iterative game process until the Nash equilibrium is reached.

[0078] The present invention provides an online training education information teaching method based on big data.

[0079] Beneficial effects:

[0080] 1. The present invention adopts a personalized teaching recommendation system based on big data. By collecting student learning data in real time and using big data technology to analyze and optimize learning paths, it achieves the technical effect of accurately matching student needs. Compared with the technical solutions in traditional education platforms that cannot respond to student status in a timely manner, the present invention can adjust the recommended content in real time and solve the problem of inaccurate configuration of personalized teaching resources.

[0081] 2. The present invention utilizes deep reinforcement learning and multi-objective optimization to dynamically adjust teaching strategies, ensuring that learning resources during the teaching process can be automatically optimized based on students' status and feedback, thereby improving learning outcomes. Compared with the static teaching methods in the prior art, the present invention addresses the inability of fixed teaching strategies to meet the diverse needs of students, thereby greatly improving the adaptability and flexibility of teaching.

[0082] 3. The present invention optimizes the interaction between the system and students through the Markov decision process and dynamic game model, ensuring a high degree of fit between recommended actions and student behavior. Unlike the technical solution of a single evaluation system in traditional teaching platforms, the present invention effectively improves students' participation and emotional motivation through the optimization of reward functions and game strategies, solving the problems of delayed student feedback and insufficient interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0084] To help those skilled in the art understand the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the described embodiments are only partial embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0085] The present invention is described in detail below with reference to the accompanying drawings:

[0086] Example:

[0087] Please see the attached Figure 1 The embodiment of the present invention provides an online training education information-based teaching method based on big data, comprising the following steps:

[0088] Step S1: Collect data from the online teaching platform, real-time assessment system, emotion monitoring device, and user interaction logs, and perform data cleaning, standardization, noise filtering, and feature extraction on the data. The data collected in step S1 includes student login data, page click data, video viewing time data, and answer accuracy data from the online teaching platform, interim performance data from the real-time assessment system, emotion rating data from the emotion monitoring device, and discussion record data from the user interaction log;

[0089] Step S2: constructing a student state vector based on the data in step S1 to implement dimensionality reduction processing;

[0090] Step S3: Establish a state-action model, define the video resources, topic resources, and discussion modules corresponding to the recommended actions, and design a reward function to calculate the reward score;

[0091] Step S4: The teaching recommendation problem is abstracted into a Markov decision process, which consists of a state set, an action set, a state transition probability function, and an immediate reward function. The Bellman optimality principle is used to solve the optimal state value function.

[0092] Step S5: Using a deep Q-network to implement reinforcement learning training on the Markov decision process in step S4, so as to obtain the optimal recommendation strategy by iteratively updating the state-action-value function;

[0093] Step S6: Construct a dynamic game model to describe the interaction between the system and students. The system rewards and student rewards are expressed in the form of cumulative discount rewards and meet the Nash equilibrium conditions.

[0094] Step S7: Integrate steps S1 to S6 to form a closed-loop feedback system, collect feedback data in real time, and adjust various parameters to verify system performance.

[0095] Step S1: Data Collection and Preprocessing: This removes noise and redundant information, ensuring that subsequent analysis accurately reflects students' actual learning outcomes. Compared to traditional systems that rely on a single data source, this method integrates multi-source data to provide a comprehensive profile of student behavior, laying the foundation for subsequent personalized recommendations.

[0096] Step S2: Constructing and reducing the student state vector can help the system efficiently process large amounts of student data, improving computing performance while ensuring the responsiveness and accuracy of the recommendation system. Compared to traditional methods that directly process high-dimensional data, this method can balance performance and effectiveness.

[0097] Step S3: Designing a state-action model and reward function can motivate students to actively participate and improve learning outcomes. Furthermore, dynamically adjusting recommended actions can address students' individual needs and overcome the rigidity of fixed teaching strategies in traditional education platforms.

[0098] Step S4: The benefits of applying the Markov decision process and Bellman optimality principle are to achieve dynamic optimization of the student learning process. Compared with traditional static teaching models, this method can dynamically adjust the recommendation strategy based on the student's immediate feedback and learning status, significantly improving the adaptability and flexibility of teaching.

[0099] Step S5: Deep Q-Network reinforcement learning training allows the recommendation system to gradually adapt to students' personalized learning needs and maximize their learning benefits. Compared to traditional recommendation methods based on rules and simple models, Deep Q-Network can intelligently learn and make decisions based on large amounts of data to provide accurate teaching resource recommendations.

[0100] Step S6: Dynamic Game Model and Nash Equilibrium Optimization Benefits: The system effectively promotes interaction between the system and students, enhancing feedback and engagement during the learning process. Unlike traditional one-way evaluation systems, this invention adjusts teaching strategies based on student feedback and behavior, enhancing the interactivity and motivational effects of teaching, and helping students maintain high learning interest and emotional motivation.

[0101] Step S7: The integrated closed-loop feedback system allows the system to update its strategies based on student changes, improving the system's long-term effectiveness. Compared to traditional education platforms that lack dynamic feedback, this approach ensures that recommended content is consistently aligned with student needs.

[0102] Step S2 further comprises:

[0103] Step 2.1: Obtain the original feature data of each student from step S1 to form a state vector X, where each row corresponds to a student and the columns are the learning progress component, assessment score component, emotion score component, and interaction frequency component.

[0104] Step 2.2: Normalize each feature in the state vector X according to the following formula to obtain the normalized data matrix x′: (ij) =(x (ij) -μ j ) / σ j ,

[0105] Among them, x (ij) is the original value of the i-th student on the j-th feature, μ j is the mean of the jth characteristic among all students, σ j is the standard deviation of the jth characteristic among all students,

[0106] i = 1, 2, ..., n and j = 1, 2, ..., d, where n is the total number of students and d is the dimension of the initial state vector;

[0107] Step 2.3, use the normalized data matrix x′ to calculate the covariance matrix S. The calculation formula is:

[0108]

[0109] in, is the transpose of x′, n is the total number of students;

[0110] Step 2.4, perform eigendecomposition on the covariance matrix S and solve the eigenvalues ​​λ1,λ2,…,λ d and the corresponding eigenvectors v1,v2,…,v d ,satisfy:

[0111] S·v j =λ j ·v j , j=1,2,...,d,

[0112] Among them, λ j is the j-th eigenvalue of the covariance matrix S, v j is the j-th eigenvector of the covariance matrix S;

[0113] Step 2.4, select the eigenvectors corresponding to the first k largest eigenvalues ​​according to the eigenvalue size to form the projection matrix W;

[0114] Step 2.5: Use the projection matrix W to reduce the dimension of the normalized data matrix x′ to obtain the reduced-dimensional state vector matrix Z. The calculation formula is: Z = x′·W,

[0115] Among them, Z is the state vector matrix after dimensionality reduction.

[0116] The advantage of step 2.1 is that the data is comprehensive, which avoids missing key information in subsequent processing.

[0117] The benefit of step 2.2 is to improve data consistency and reduce dimensionality effects.

[0118] Step 2.3 helps reveal the intrinsic structure of the data, facilitates subsequent dimensionality reduction processing, and enhances data expression capabilities.

[0119] The advantage of step 2.4 is that it highlights key information and reduces noise interference.

[0120] The advantage of step 2.5 is that it reduces the data dimension, retains the main information, reduces the amount of calculation, and facilitates subsequent efficient analysis.

[0121] Step S3 further comprises:

[0122] Step 3.1, establish a state-action model, and define the state vector according to the student state vector constructed in step S2 To describe students' learning progress, assessment scores, emotional state and interactive behavior; define the recommended action set U, which contains multiple recommended actions;

[0123] Step 3.2: Design a reward function, which includes learning progress rewards, knowledge mastery rewards, and emotional motivation rewards. The specific calculation formula is:

[0124] r (p) (ξ q ,v q )=f (p) (ξ q ,v q ),

[0125] r (k) (ξ q ,v q )=f (k) (ξ q ,v q ),

[0126] r (e) (ξ q ,v q )=f (e) (ξ q ,v q ),

[0127] Among them, r (p) (ξ q ,v q ) is the learning progress reward, r (k) (ξ q ,v q ) is the knowledge mastery reward, r (e) (ξ q ,v q ) is the emotional incentive reward, ξ q is the student's state vector, v q is the recommended action, f (p) 、f (k) and f (e) A function for calculating various rewards;

[0128] Step 3.3: Calculate the cumulative reward. To evaluate the long-term effect of the recommended action, use the discount factor δ to reduce and accumulate the immediate rewards to obtain the overall reward. The overall reward function is:

[0129]

[0130] Among them, ω p is the learning progress reward, ω k Is the knowledge mastery reward, ω eis the weight coefficient of the emotional incentive reward, δ is the discount factor, T is the number of time steps, and J(π) is the total reward;

[0131] Step 3.4: The system determines the optimal recommended action v based on the cumulative reward J(π) q to maximize the reward value.

[0132] Step 3.1: Benefits of Establishing a State-Action Model Compared to the single-dimensional learning progress tracking in traditional teaching platforms, this invention can comprehensively consider multiple aspects of information, fully capture the student's learning status, and provide an accurate basis for the subsequent recommendation system.

[0133] Step 3.2: Benefits of Designing a Reward Function: Designing rewards for learning progress, knowledge mastery, and emotional motivation allows for targeted recommendation strategies and effectively boosts learning motivation. Compared to traditional reward mechanisms, this step fully considers multi-dimensional student feedback, ensuring the adaptability of the recommendation strategy.

[0134] Step 3.3: Calculating Cumulative Rewards: By applying a discount factor to the immediate rewards, calculating the cumulative reward balances the importance of current and future rewards. Compared to traditional systems that prioritize short-term gains, this step helps optimize long-term learning outcomes, addressing the issue of insufficient short-term incentives.

[0135] Step 3.4: Determine the benefits of the optimal recommendation action. This ensures that students receive the resource recommendations that will best enhance their learning outcomes. Compared to traditional fixed recommendation strategies, this step allows recommendations to be dynamically optimized based on students' immediate needs and long-term development goals, improving learning outcomes.

[0136] Step S4 further comprises:

[0137] Step 4.1: Model the teaching recommendation problem as a Markov decision process, defining the state set S, the action set A, the state transition probability function P, and the immediate reward function R;

[0138] The state set S represents all states of students;

[0139] The action set A contains all the actions recommended by the system for students;

[0140] The state transition probability function P(s′|s,a) represents the probability of transitioning to state s′ after taking action a in state s;

[0141] The immediate reward function R(s,a) represents the immediate reward obtained after taking action a in state s;

[0142] Step 4.2, use Bellman's optimality principle to solve the optimal state value function V *(s), which represents the maximum cumulative reward for taking the optimal strategy in state s. The recursive form of the Bellman equation is:

[0143]

[0144] Among them, V * (s) is the optimal value function of state s, a is the currently available action,

[0145] R(s,a) is the immediate reward after taking action a in state s, s′ is the next state,

[0146] P(s′|s,a) is the probability of transitioning to state s′ after taking action a in state s.

[0147] γ is the discount factor;

[0148] Step 4.3, update the Bellman equation by iteration until the optimal state value function V * (s) converges. During each iterative update, the value of the next state is calculated based on the combination of the current state s and action a, and finally the optimal strategy and optimal state value function are obtained.

[0149] Step 4.1: The benefit of constructing a Markov decision process model is that it accurately reproduces the dynamic interaction between student status and system behavior. It comprehensively captures changes in student status, avoiding the limitations of single-point data decision-making. Compared to traditional static models, its description is more timely and adaptable.

[0150] Step 4.2: Using the Bellman Optimality Principle to solve the optimal state value function takes into account both immediate and future returns, ensuring long-term decision-making. This helps the system evaluate the combined effects of each action and improve overall recommendation quality.

[0151] Step 4.3: Iterate and update until the optimal state value function converges. The benefit of the iterative process is that it continuously corrects errors, ensuring a stable and accurate optimal strategy. The dynamic update mechanism effectively adapts to changes in student status and responds promptly. Compared to fixed strategies, iterative solutions are more flexible and suitable for practical teaching situations.

[0152] Step S5 further comprises:

[0153] Step 5.1: After obtaining the optimal state value function through the Markov decision process in step S4, the deep Q network is used to approximate the optimal value function. The Q network is defined as a neural network whose input is the state vector s q , the output is each action a q The corresponding Q value Q(s q ,a q );

[0154] The Q value represents the q Next, perform action a q Expected cumulative rewards after

[0155] Step 5.2: During the training process, the Q-value function is updated based on the current Q-network by interacting with the environment. During each update, the current Q-value is updated by calculating the actual reward and the Q-value of the next state. The Bellman equation is used to update the Q-value. The formula is:

[0156]

[0157] Among them, Q(s q ,a q ) is the current state s q Next, perform action a q Q value,

[0158] α is the learning rate, r q is the current state s q Next, perform action a q After receiving the instant reward,

[0159] γ is the discount factor, is the next state s q+1 The maximum Q value under

[0160] Step 5.3, to increase the stability of the training process, the deep Q network usually uses the target network to calculate the target Q value. The parameters of the target network θ - It is regularly copied from the parameters θ of the main Q network, and the update formula is:

[0161] θ - ←θ, the update of the target network can avoid large fluctuations during training;

[0162] In step 5.4, the deep Q network adjusts the network parameters θ through multiple training cycles to reduce the error between the Q value prediction and the actual reward, so that the Q network can accurately predict the Q value of each state-action pair and finally obtain the optimal strategy.

[0163] Step 5.1: The advantage of using a deep Q-network to approximate the optimal state-value function is that it can capture the complex nonlinear relationship between state and action, accurately predict the long-term benefits of each action, and is more flexible than traditional methods and can adapt to changing environments.

[0164] Step 5.2: The benefits of updating the Q-value function based on environmental interaction are that it can correct prediction errors in real time, quickly adapt to feedback, effectively balance current and future benefits, and converge faster than a fixed update strategy.

[0165] Step 5.3: Using the target network to improve training stability has the benefits of smoothing the training process, reducing sharp fluctuations in parameters, avoiding shocks caused by unstable estimation, and improving the robustness of the overall training.

[0166] Step 5.4: Multi-cycle training and tuning to obtain the benefits of the optimal strategy. Continuous optimization gradually reduces the prediction error and ultimately outputs the optimal recommendation strategy. The dynamic adjustment capability far exceeds that of traditional fixed strategies.

[0167] Step S6 further comprises:

[0168] Step 6.1: Model the interaction between the system and students as a dynamic game model to describe the interaction process between the system and students. The dynamic game model includes the system's rewards and the students' rewards.

[0169] Step 6.2, define the reward functions of the system and the student. The reward functions of the system and the student are expressed by the following formula:

[0170] System Rewards s : Measures the effectiveness of the system's recommended actions;

[0171] Student Rewards st : reflects students’ feedback on recommended actions;

[0172] Step 6.3, Nash equilibrium condition. To ensure the stability of the strategies in the game, the strategies of the system and the student must satisfy the Nash equilibrium condition. Given the strategy of the other party, neither party has the motivation to unilaterally change its own strategy. The Nash equilibrium condition is expressed as:

[0173]

[0174] Among them, r s is the reward function of the system, r st is the student’s reward function, a s is the recommended action of the system, a st is the student's reaction action, E[r s (a s ,a st )] is the expected value of the system reward, E[r st (a s ,a st )] is the expected value of the student’s reward.

[0175] Step 6.4: In the dynamic game, the system and the student optimize their respective behaviors through dynamic game strategies based on their own reward functions. The system's strategy π s Strategies for students st All of them can be optimized through the iterative game process until the Nash equilibrium is reached.

[0176] Step S6 greatly improves the quality of interaction between the system and students by introducing a dynamic game model and Nash equilibrium conditions. Through dynamic adjustment and game strategy optimization, the system can dynamically adjust the recommended content based on students' feedback and behavior, ensuring the stability and effectiveness of the recommendation strategy. This process enables the recommendation system to flexibly respond to complex and changing student needs and provide accurate and personalized learning resource recommendations. Compared with traditional fixed strategy teaching methods, the game optimization strategy of the present invention performs outstandingly in improving student participation and interactivity, thereby improving teaching effectiveness and system adaptability.

[0177] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A big data-based online training education information-based teaching method, characterized by comprising the following steps: Step S1: Collect data from the online teaching platform, real-time evaluation system, emotion monitoring equipment, and user interaction logs, and perform data cleaning, standardization, noise filtering, and feature extraction; Step S2: constructing a student state vector based on the data in step S1 to implement dimensionality reduction processing; Step S3: Establish a state-action model, define the video resources, topic resources, and discussion modules corresponding to the recommended actions, and design a reward function to calculate the reward score; The step S3 further comprises: Step 3.1, establish a state-action model, and define the state vector according to the student state vector constructed in step S2 To describe students' learning progress, assessment scores, emotional state and interactive behavior; define the recommended action set U, which contains multiple recommended actions; Step 3.2: Design a reward function, which includes learning progress rewards, knowledge mastery rewards, and emotional motivation rewards. The specific calculation formula is: r (p) (x) q ,v q )=f (p) (x) q ,v q ), r (k) (x) q ,v q )=f (k) (x) q ,v q ), r (e) (x) q ,v q )=f (e) (x) q ,v q ), Among them, r (p) (ξ q ,v q ) is the learning progress reward, r (k) (ξ q ,v q ) is the knowledge mastery reward, r (e) (ξ q ,v q ) is the emotional incentive reward, ξ q is the student's state vector, v q is the recommended action, f (p) 、f (k) and f (e) A function for calculating various rewards; Step 3.3: Calculate the cumulative reward. To evaluate the long-term effect of the recommended action, use the discount factor δ to reduce and accumulate the immediate rewards to obtain the overall reward. The overall reward function is: Among them, ω p is the learning progress reward, ω k Is the knowledge mastery reward, ω e is the weight coefficient of the emotional incentive reward, δ is the discount factor, T is the number of time steps, and J(π) is the total reward; Step 3.4: The system determines the optimal recommended action v based on the cumulative reward J(π) q To maximize the reward value; Step S4: The teaching recommendation problem is abstracted into a Markov decision process, which consists of a state set, an action set, a state transition probability function, and an immediate reward function. The Bellman optimality principle is used to solve the optimal state value function. The step S4 further comprises: Step 4.1: Model the teaching recommendation problem as a Markov decision process, defining the state set S, the action set A, the state transition probability function P, and the immediate reward function R; The state set S represents all states of students; The action set A contains all the actions recommended by the system for students; The state transition probability function P(s′|s,a) represents the probability of transitioning to state s′ after taking action a in state s; The immediate reward function R(s,a) represents the immediate reward obtained after taking action a in state s; Step S5: Using a deep Q-network to implement reinforcement learning training on the Markov decision process in step S4, the state-action-value function is iteratively updated to obtain the optimal recommendation strategy. The step S5 further comprises: Step 5.1: Use the deep Q network to implement reinforcement learning training on the Markov decision process in step S4 to iteratively update the state-action-value function to obtain the optimal recommendation strategy. Define the Q network as a neural network with the state vector s as input. q , the output is each action a q The corresponding Q value Q(s q ,a q ); The Q value represents the q Next, perform action a q Expected cumulative rewards after Step 5.2: During the training process, the Q-value function is updated based on the current Q-network by interacting with the environment. During each update, the current Q-value is updated by calculating the actual reward and the Q-value of the next state. The Bellman equation is used to update the Q-value. The formula is: Among them, Q(s q ,a q ) is the current state s q Next, perform action a q Q value, α is the learning rate, r q is the current state s q Next, perform action a q After receiving the instant reward, γ is the discount factor, is the next state s q+1 The maximum Q value under Step S6: Construct a dynamic game model to describe the interaction between the system and students. The system rewards and student rewards are expressed in the form of cumulative discount rewards and meet the Nash equilibrium conditions. Step S7: Integrate steps S1 to S6 to form a closed-loop feedback system, collect feedback data in real time, and adjust various parameters to verify system performance.

2. The method for online training education informationization based on big data according to claim 1 is characterized in that the data collected in step S1 includes student login data, page click data, video viewing time data and answer accuracy data in the online teaching platform, periodic performance data in the real-time evaluation system, emotion scoring data in the emotion monitoring device, and discussion record data in the user interaction log.

3. The method for online training education informationization based on big data according to claim 1, wherein step S2 further comprises: Step 2.1: Obtain the original feature data of each student from step S1 to form a state vector X, where each row corresponds to a student and the columns are the learning progress component, assessment score component, emotion score component, and interaction frequency component. Step 2.2: Normalize each feature in the state vector X according to the following formula to obtain the normalized data matrix x′: (ij) =(x (ij) -μ j ) / σ j , Among them, x (ij) is the original value of the i-th student on the j-th feature, μ j is the mean of the jth characteristic among all students, σ j is the standard deviation of the jth characteristic among all students, i = 1, 2, ..., n and j = 1, 2, ..., d, where n is the total number of students and d is the dimension of the initial state vector.

4. The online training education informationization teaching method based on big data according to claim 1 is characterized in that: The step S2 further comprises: Step 2.3, use the normalized data matrix x′ to calculate the covariance matrix S. The calculation formula is: S=(1 / (n-1))·(x′) T ·x′, Where (x′) T is the transpose of x′, n is the total number of students; Step 2.4, perform eigendecomposition on the covariance matrix S and solve the eigenvalues ​​λ1,λ2,…,λ d and the corresponding eigenvectors v1,v2,…,v d ,satisfy: S·v j =λ j ·v j ,j=1,2,...,d, Among them, λ j is the j-th eigenvalue of the covariance matrix S, v j is the j-th eigenvector of the covariance matrix S; Step 2.4, select the eigenvectors corresponding to the first k largest eigenvalues ​​according to the eigenvalue size to form the projection matrix W; Step 2.5: Use the projection matrix W to reduce the dimension of the normalized data matrix x′ to obtain the reduced-dimensional state vector matrix Z. The calculation formula is: Z = x′·W, Among them, Z is the state vector matrix after dimensionality reduction.

5. The online training education informationization teaching method based on big data according to claim 1 is characterized in that: The step S4 further comprises: Step 4.2, use Bellman's optimality principle to solve the optimal state value function V * (s), which represents the maximum cumulative reward for taking the optimal strategy in state s. The recursive form of the Bellman equation is: Among them, V * (s) is the optimal value function of state s, a is the currently available action, R(s,a) is the immediate reward after taking action a in state s, s′ is the next state, P(s′|s,a) is the probability of transitioning to state s′ after taking action a in state s; γ is the discount factor; Step 4.3, update the Bellman equation by iteration until the optimal state value function V * (s) converges. During each iterative update, the value of the next state is calculated based on the combination of the current state s and action a, and finally the optimal strategy and optimal state value function are obtained.

6. The online training education informationization teaching method based on big data according to claim 1 is characterized in that: The step S5 further comprises: Step 5.3, to increase the stability of the training process, the deep Q network usually uses the target network to calculate the target Q value. The parameters of the target network θ - It is regularly copied from the parameters θ of the main Q network, and the update formula is: θ - ←θ, the update of the target network can avoid large fluctuations during training; In step 5.4, the deep Q network adjusts the network parameters θ through multiple training cycles to reduce the error between the Q value prediction and the actual reward, so that the Q network can accurately predict the Q value of each state-action pair and finally obtain the optimal strategy.

7. The online training education informationization teaching method based on big data according to claim 1 is characterized in that: The step S6 further comprises: Step 6.1: Model the interaction between the system and students as a dynamic game model to describe the interaction process between the system and students. The dynamic game model includes the system's rewards and the students' rewards. Step 6.2, define the reward functions of the system and the student. The reward functions of the system and the student are expressed by the following formula: System Rewards s : Measures the effectiveness of the system's recommended actions; Student Rewards st : reflects students’ feedback on recommended actions; Step 6.3, Nash equilibrium condition. To ensure the stability of the strategies in the game, the strategies of the system and the student must satisfy the Nash equilibrium condition. Given the strategy of the other party, neither party has the motivation to unilaterally change its own strategy. The Nash equilibrium condition is expressed as: Among them, r s is the reward function of the system, r st is the student’s reward function, a s is the recommended action of the system, a st is the student's reaction action, E[r s (a s ,a st )] is the expected value of the system reward, E[r st (a s ,a st )] is the expected value of the student’s reward; Step 6.4: In the dynamic game, the system and the student optimize their respective behaviors through dynamic game strategies based on their own reward functions. The system's strategy π s Strategies for students st All of them can be optimized through the iterative game process until the Nash equilibrium is reached.

Images