A personalized learning path recommendation method based on deep reinforcement learning

Abstract

This invention proposes a personalized learning path recommendation method based on deep reinforcement learning. The method includes: acquiring an educational dataset and constructing a knowledge graph with triples as basic units based on the dataset; acquiring the user's learning sequence and inputting it into a question-answering model to obtain the probability of the user correctly answering the next question; filtering learning paths based on the knowledge graph and the probability of the user correctly answering the next question to obtain the optimal learning path; and recommending the optimal learning path to the user. This invention effectively eliminates the interference of browsing order by deeply analyzing the learner's actual performance in various tasks and inversely deducing the true logical preconditions between knowledge points, thus constructing a reliable underlying knowledge network. During recommendation calculation, the system adopts a pruning-then-filtering mechanism, compressing the massive computational dimensions by several orders of magnitude, ensuring recommendation quality, significantly reducing computational costs, and achieving extremely fast real-time response.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a personalized learning path recommendation method based on deep reinforcement learning. Background Technology

[0002] With the deep integration of information technology and education, online education systems have provided learners with abundant learning resources. However, the booming development of online education has also given rise to significant problems such as "knowledge overload" and "knowledge disorientation," greatly limiting learners' learning gains and efficiency. Faced with diverse learning needs and massive learning resources, how to quickly achieve learning goals, reduce learning costs, and rationally allocate learning resources have become major issues restricting individual and societal development. However, the traditional "one-size-fits-all" education model can no longer meet people's needs for knowledge acquisition; a more efficient and scientific personalized education model is needed to help learners achieve their learning goals with minimal learning costs. Against this backdrop, how to efficiently identify learner characteristics, efficiently organize and allocate learning resources, and plan personalized learning paths for each learner has become a problem that needs to be solved in research on personalized, precise educational resource matching mechanisms. However, current personalized learning recommendation methods have failed to comprehensively and systematically meet learners' needs.

[0003] In the field of personalized learning path recommendation, existing recommendation systems typically establish connections between knowledge points solely based on the order in which learners browse materials. This association, built purely on click order, often involves numerous manually predetermined processes rather than reflecting the true sequential dependencies of the knowledge itself. This leads to inherent biases in the system's underlying logic, causing subsequent recommendations to deviate from objective learning patterns. Traditional recommendation algorithms, when faced with massive amounts of learning resources, evaluate all resources simultaneously at the same level. As the resource library grows, the amount of data the computer needs to process increases dramatically. This not only makes model training extremely difficult and consumes enormous computational resources but also severely slows down the system's feedback speed during actual operation. Existing personalized path recommendation methods are rigid and ignore the learner's physiological state. Most current technologies focus only on improving predicted learning performance, completely disregarding the fatigue learners experience after continuous high-intensity learning. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention proposes a personalized learning path recommendation method based on deep reinforcement learning. The method includes: acquiring an educational dataset; constructing a knowledge graph based on triples as basic units using the educational dataset; acquiring the user's learning sequence; inputting the learning sequence into a question-answering model to obtain the probability of the user correctly answering the next question; filtering learning paths based on the knowledge graph and the probability of the user correctly answering the next question to obtain the optimal learning path; and recommending the optimal learning path to the user.

[0005] The beneficial effects of this invention are:

[0006] This invention, through in-depth analysis of learners' actual performance in various tasks, reverse-engineers the true logical prerequisites between knowledge points, thereby effectively eliminating the interference of browsing order and constructing a reliable underlying knowledge network. In recommendation calculation, the system employs a pruning-then-filtering mechanism, compressing the massive computational dimensions by several orders of magnitude, ensuring recommendation quality while significantly reducing computational costs and achieving extremely fast real-time response. Furthermore, the model introduces a dynamic fatigue perception mechanism, tracking learners' continuous learning progress in real time and automatically increasing the recommendation ratio of easy learning content when fatigue is detected. This dynamic fine-tuning, combining work and rest, not only effectively protects learners' enthusiasm but also improves long-term comprehensive learning outcomes. Attached Figure Description

[0007] Figure 1 This is a diagram illustrating the overall framework structure of the present invention;

[0008] Figure 2 This is a flowchart illustrating the personalized learning path recommendation process of the present invention.

[0009] Figure 3 This is a diagram of the DQN network structure of the present invention;

[0010] Figure 4 This is a functional structure diagram of the DQN of the present invention;

[0011] Figure 5 This is a structural diagram of the PPO algorithm of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] A personalized learning path recommendation method based on deep reinforcement learning includes: acquiring an educational dataset and constructing a knowledge graph with triples as basic units based on the educational dataset; acquiring the user's learning sequence and inputting the learning sequence into a question-answering model to obtain the probability that the user will correctly answer the next question; filtering learning paths based on the knowledge graph and the probability that the user will correctly answer the next question to obtain the optimal learning path; and recommending the optimal learning path to the user.

[0014] This invention is divided into three stages: The first stage is the design of an educational knowledge graph framework. This stage constructs a multimodal knowledge graph based on an educational dataset and defines entity types, relation types, and their related attributes. This information will be used for subsequent path finding and path evaluation calculations. The second stage is the research on learner profile modeling. Addressing the needs of learning path recommendation, it comprehensively analyzes learners' historical learning process data and online interaction behavior data to construct learner profiles, providing personalized decision-making basis for subsequent path planning, resource selection, and dynamic recommendation. The third stage is the implementation of a personalized learning path recommendation algorithm based on deep reinforcement learning. This algorithm performs real-time path planning for target students; it uses policy updates for optimization and prunes the action space using value-based methods before action selection. The overall architecture of the personalized learning path recommendation algorithm is shown below. Figure 1 As shown.

[0015] In the first phase of the educational knowledge graph framework design process, the knowledge graph is first constructed with triples as the basic building blocks, represented as follows: .

[0016] Represents a set of nodes, containing knowledge point nodes. and resource nodes Knowledge Points Composed of abstract knowledge concepts, these represent knowledge units within a subject's system. Resource Nodes Composed of specific learning materials, these resources are divided into two main subcategories based on their functional attributes: assessment resources. Non-evaluational resources Evaluation resources Resources that provide quantitative feedback on students' knowledge acquisition include after-class exercises, unit tests, and periodic exams. The interaction results of these resources (such as correct / incorrect answers, scores) are key indicators for assessing students' real-time knowledge mastery; non-assessment resources... This refers to materials used for knowledge input and transmission, including instructional videos, textbook texts, and demonstration animations. These elements focus on the explanation and presentation of knowledge and are the primary source for students to acquire new knowledge and trigger cognitive transfer.

[0017] Represents a set of relations, containing containment edges that characterize resource ownership. Pre-trimming of edges representing logical order Similar edges that represent overlapping content The specific process for edge construction and weight calculation is as follows:

[0018] Step 1: Include relation edges The construction.

[0019] Inclusion relationships describe resource nodes With knowledge point nodes The relationship between learning resources and knowledge points is established. This relationship is constructed based on a predefined mapping matrix M, which is a Boolean matrix describing the association between learning resources and knowledge points. The M matrix is ​​defined as follows:

[0020]

[0021] Based on the definition of the M-matrix, the association between resource nodes and corresponding knowledge point nodes is mapped to containment edges in the graph. Each non-zero term represents a mapping... Each corresponds to an edge in the graph that connects a learning resource to a knowledge point, and its set is... The mathematical expression is as follows:

[0022]

[0023] Step 2: Repair the relationship edges first. Data mining and construction.

[0024] This invention employs association rule technology to deduce prerequisite dependencies between knowledge points from learners' actual correct and incorrect answer data, thereby constructing a directed prerequisite graph. Let U be the set of all learners. For exercise a in the assessment resources, the set of learners who answered the question correctly is defined as... The set of learners who answered the question incorrectly is: .

[0025] First, we calculate the consistency of the questions. We define the consistency between two questions, a and b, as the total number of learners who answer both questions correctly or incorrectly at the same time.

[0026]

[0027] To filter out occasional random answer noise, a consistency threshold is set. When If the number of questions is less than the threshold, the two questions are deemed to lack statistical significance and are removed from subsequent calculations.

[0028] In the initial screening of question pairs, the confidence level of a causal relationship between them is further calculated. If a large number of learners who answer question a incorrectly will also answer question b incorrectly, it indicates that the basic concept contained in question a is a prerequisite for question b. The confidence level of the rule that answering a incorrectly will necessarily answer b incorrectly is shown in Formula 4. According to the logical equivalence of the contrapositive, if learners who answer question a correctly will also answer question b correctly, it is equivalent to deducing that b is a prerequisite for a. Its confidence level is shown in Formula 5. At the same time, a minimum confidence threshold is set, and only association rules with a confidence level higher than this threshold are retained.

[0029]

[0030]

[0031] Next, we establish the relevance weights between the knowledge points. For example, if question a tests knowledge point... Question b tests the knowledge point. This maps the confidence level of the correlation between questions to the knowledge point level.

[0032]

[0033] Finally, the relationship of repairing first and repairing later was determined. If the strong association discovered is... This indicates a lack of understanding of the knowledge points. Unable to grasp Therefore, establishing the origin from the map in the atlas point to Directed edges. If the discovered strong correlation is Based on the logical equivalence principle of the contrapositive proposition, the key points are: This is the more fundamental cognitive foundation. Therefore, establishing from... point to The directed edges in the opposite direction. Since multiple sets of questions may cover the same knowledge point pairs, multiple redundant edges in the same direction are calculated. The edges in the same direction are deduplicated, and only the value with the highest relevance weight is retained as the directed edge. The final priority weight.

[0034] Step 3, Similarity Relationship Edges The construction.

[0035] Similarity edges are undirected edges primarily used to characterize the degree of overlap in content examination among different learning resource nodes in a knowledge graph. This study employs knowledge coverage based on a mapping matrix to calculate the similarity between resources. According to the pre-resource-knowledge point mapping matrix M, any learning resource node in the knowledge graph can be extracted and represented as a feature vector. This vector is the corresponding row vector in matrix M, and its dimension is equal to the total number of knowledge point nodes in the knowledge graph. The elements in the vector take only 0 or 1 values, indicating whether the learning resource contains the corresponding knowledge point.

[0036] This invention introduces cosine similarity to quantify learning resources. and The mathematical formula for calculating the degree of overlap in the coverage of knowledge points is as follows:

[0037]

[0038] in, The inner product of two resource feature vectors represents the number of knowledge points they jointly cover; the denominator is the product of the magnitudes of the two vectors, used to normalize the vector lengths. The calculated similarity value... The closer the value is to 1, the more consistent the knowledge points covered by the two learning resources; if it equals 0, it means that the two are completely independent in terms of the content being tested. To ensure the connection quality of the educational knowledge graph and effectively filter out redundant edges with weak content connections, the system sets a similarity threshold. Resources are only considered valid if the similarity exceeds the set threshold. and They have similarities. At this point, connection nodes are created in the graph. and undirected edge The calculated similarity values As the weight of that edge.

[0039] In this embodiment, the second phase of learner profiling modeling aims to track students' knowledge mastery status. The specific implementation process is as follows:

[0040] Step 1: Define the activity learning sequence.

[0041] In real-world online learning environments, learners' interactive behaviors extend beyond answering practice questions; they also include watching instructional videos and reading texts to acquire knowledge. To accurately characterize this hybrid behavioral pattern, we first formally define multi-activity learning sequences and design corresponding embedding layers to vectorize discrete interactive features.

[0042] The complete learning process of learner u is defined as a sequence of time-step tuples containing activity type identifiers:

[0043]

[0044] in, As a two-way indicator variable, when At that time, the student representative conducted an assessment activity, and the interactive content was represented as follows: ,in A unique number for the question or test. For the performance in answering questions. Regarding the practice questions, Take the binary result These represent incorrect and correct answers, respectively; for unit tests, Take the normalized continuous score (map the percentage score to) (Interval). When At that time, the student representative conducted an assessment activity, and the interactive content was represented as follows: ,in For specific learning resources.

[0045] Step 2, Embedding Layer: This layer aims to integrate the aforementioned discrete learning activities. The embeddings are mapped to high-dimensional continuous vectors, serving as inputs for subsequently capturing dynamic changes in learners' knowledge mastery. Due to differences in cognitive load and modal information between evaluative and non-evaluative resources, independent embedding matrices are constructed for each. The learning matrix is ​​defined for the embeddings of evaluative learning resources. ,Will One assessment learning resource to 3D space. The learning matrix is ​​defined for embeddings of non-evaluative learning resources. ,Will Learning resources covering different modalities such as video and text are mapped to 3D space.

[0046] Step 3, Aggregation Layer: Similar edges from the previously constructed educational knowledge graph are used as local aggregation paths in the graph neural network. The specific propagation formula is as follows:

[0047]

[0048]

[0049] in, The question itself represents the initial content characteristics used to express which knowledge points are being tested; and These represent the evaluative question features and non-evaluative handout features output after aggregation by the graph neural network, respectively. Matrix and They are respectively responsible for the feature space transformation between isomorphic question nodes and isomorphic lecture note nodes. Simultaneously, the matrix... and Responsible for cross-modal feature propagation. , and These correspond to the transformation weight matrix and bias vector of this propagation layer, respectively. In the allocation of aggregation weights, two types of neighbor sets are extracted for the current node, including the evaluation learning resource neighbor set. and non-evaluational learning resource neighbor sets Weighting factors This represents the aggregation weight after normalization of similar edges in the educational knowledge graph. The higher the overlap of learning resources among neighboring nodes, the greater the weight of their associated semantic and difficulty features in the feature reconstruction of the current node.

[0050] Step 4: Dynamic Knowledge State Transition Layer. First, define two matrices: the static key matrix. This represents K knowledge point features. Each row represents the latent semantic feature vector of a knowledge point, storing the attributes of the knowledge point itself. During the student's learning process, whenever a student encounters a new learning resource, the system calculates the features of that resource and... The similarity of each line in the matrix is ​​used to accurately pinpoint which knowledge points are affected by the current interaction; dynamic value matrix. This represents the learner's mastery of K knowledge points at time t, with each row representing a... A dimensional vector records a learner's mastery of a particular knowledge point and is continuously updated as the student's learning activities progress. When a learner engages in a new interaction, the model first needs to determine the weight of that interaction's impact on each knowledge point.

[0051]

[0052] in, This represents the specific weight of the current interactive activity's impact on the i-th knowledge point; and It is a feature mapping matrix used to project enhanced features of different modalities into the same semantic space as the knowledge point features. The function performs exponential normalization on the calculated similarity scores of all knowledge points to ensure that the sum of the weights of all concepts is 1.

[0053] After receiving the learning activity at time t, the knowledge state matrix is ​​processed through a "transfer-erase-add" mechanism. Dynamic adjustments are made. Before dynamically adjusting the memory matrix, the model must clearly define the learner's specific performance in the current activity to determine the extent of knowledge forgetting or gain. First, the system uses activity type indicator variables... Extracting the current joint interaction features :

[0054]

[0055] In the formula, if the current activity is an evaluation activity, the model will enhance the question features. Characteristics of answering questions Perform splicing; if it is a non-evaluative activity, only use the handout features. Subsequently, using a fully connected network and a non-linear activation function, erase and add vectors are generated for the current interaction state:

[0056]

[0057]

[0058] Among them, the Sigmoid activation function Ensure that the erasure ratio is strictly controlled within the range of (0, 1); Functions allow the generation of knowledge increments that include fluctuations in both positive and negative directions.

[0059] Switching between sensing and erasing operations:

[0060]

[0061] in, This represents students switching from learning from non-assessment resources to learning from assessment resources. : These represent the transition states of continuous practice, continuous learning, and practice-to-learning, respectively. Transition weight matrix ( ): Used to capture the differences in the impact of various learning activity transitions on knowledge state, ensuring that the model can understand different cognitive transfer effects. Erasure vector Used to remove redundant or erroneous cognitive information from old knowledge states. For the previous moment (moment) The dynamic value matrix records the students' previous knowledge mastery status.

[0062] Adding incremental knowledge:

[0063]

[0064] Among them, adding vectors This is used to capture the incremental increase in new knowledge acquired by students at time t. A vector will be added. Multiply by the attention weight of that knowledge point It is directly superimposed onto the intermediate state that was just erased. This completes a full evolution of the knowledge state.

[0065] Step 5, Prediction Layer: The prediction process is designed to predict learners' performance when faced with specific learning resources based on the current knowledge profile. It calculates knowledge association weights based on the current joint interaction features to determine the next time step. The correlation strength vector between N knowledge points Then use indicator variables Determine the activity type at time t. Based on the transformation logic of "question to question" or "video to question," perform spatial transformation on the profile using a transition matrix, and extract the overall mastery level for the new question by weighting. :

[0066]

[0067] The degree of mastery of reading Original embedding of the question Concatenation. First, a summary vector is generated through a fully connected layer. Finally, the sigmoid activation function is used to output the probability that the student correctly answers the next question. :

[0068]

[0069]

[0070] In this embodiment, to achieve accurate and logically coherent path recommendation in a complex resource space, a recommendation framework combining knowledge graphs and dual reinforcement learning algorithms is proposed in the third stage. This framework uses a three-step selection mechanism to progressively compress and optimize the original action space, ultimately producing the optimal learning path. The overall path recommendation flowchart is as follows: Figure 2 As shown.

[0071] This invention defines the learning path recommendation process as a quintuple. Each element is defined as follows:

[0072] (1) State space: states It is a complete characterization of the learner's current cognitive level, including the degree of mastery of each knowledge point in the knowledge graph within the implicit semantic space.

[0073] (2) Action Space: Action The learning resources pushed to students by the system consist of both assessment-based and non-assessment-based resources in the knowledge graph.

[0074] (3) State transition function (Transition Probability) When learners and resources A transition is triggered when an interaction occurs, moving the user to the next state. The probability of is denoted as .

[0075] 4) Reward Function: The reward design follows the "goal-oriented" principle, guiding the agent to find the shortest and most efficient path. At each time step t, the agent takes an action. The overall reward obtained later The comprehensive reward function designed in this invention for:

[0076]

[0077] in, The total knowledge gain represents the sum of the improvement in students' mastery of all knowledge points after learning the resource. This is achieved by comparing the learner profile matrix before and after the update. and The inputs are fed into the prediction layer to obtain the difference in the mastery probability of each knowledge point. This is a positive incentive that guides the system to recommend resources that can significantly fill knowledge gaps. As a dynamic fatigue factor, this value gradually increases from 1 as learning progresses, amplifying the effect of the step count penalty and reflecting the physiological fatigue accumulation process of students. It represents the total number of interaction steps, which indicates the total number of resource recommendations a student has experienced from the start of learning to the current moment. The more steps, the higher the cognitive load for the student. This represents the fatigue growth coefficient. By introducing a dynamic fatigue factor and mapping it to knowledge gain, a comprehensive assessment of the quality and efficiency of the learning path is achieved. The base step penalty prevents the system from recommending a large number of irrelevant resources in pursuit of minimal gains, and deducts the basic time / effort cost required for each resource recommended. Compared to traditional single-objective recommendation algorithms, this function effectively suppresses ineffective and redundant recommendations, minimizing cognitive load while ensuring learners achieve mastery goals and generating learning paths that better align with human physiological learning patterns.

[0078] (5) Discount factor (Discount Factor): Range of values It is used to measure the weight of future rewards in current decisions.

[0079] The first step is logical pruning based on the knowledge graph. The recommendation process first performs physical constraint filtering at the knowledge point level. Using the knowledge graph constructed in the first stage, it retrieves the prerequisite relationships between all knowledge points and calls up the student mastery matrix in real time. Only when the mastery of all direct prerequisite knowledge points for a certain knowledge point reaches a preset threshold is that knowledge point and all its associated learning resources activated and added to the candidate pool. This stage ensures, through "hard constraints," that the recommendation path strictly follows the pedagogical principle of gradual progression, filtering out "advanced" resources that students' current cognitive level cannot handle.

[0080] The second step is a preliminary value screening based on Deep Q-Network (DQN). Within the pool of resource candidates that meet the logical admission criteria, this invention utilizes DQN to perform further spatial compression. The DQN agent takes the learner's current knowledge profile state as input and fits the action value function through a deep neural network. Its network structure diagram is as follows Figure 3 As shown, the Q value represents the expected cumulative reward a learner can obtain throughout the entire learning sequence after recommending a specific resource in the current profile state. Then, based on the calculated Q value, all candidate actions are sorted in descending order, and value pruning is performed, eliminating actions with low Q values ​​(i.e., low long-term cost-effectiveness), retaining only the top-K resources with high value potential to construct a selected action set. This stage achieves a shift from "logically sound" to "high cost-effectiveness," significantly reducing the search dimensionality in the subsequent PPO strategy selection stage, accelerating model convergence while ensuring path efficiency.

[0081] The update of DQN follows the Bellman equation:

[0082]

[0083] The key variables are defined as follows: It is an action-value function, representing the state of knowledge mastery that a student is in at time t. At that time, take action The expected cumulative benefit that can be obtained by recommending a specific resource. For recommended resources Afterwards, students immediately receive comprehensive feedback rewards. The discount factor is a constant between 0 and 1 used to adjust the weight of "immediate benefits" and "long-term benefits". Predicting the optimal value for the future represents the period after students have completed learning the current resources and moved on to the next stage. Then, estimate the maximum Q value that can be obtained among all possible actions to be taken in the future.

[0084] To ensure the model's stability and convergence speed in complex learning environments, the following experience replay optimization technique is adopted: An experience replay pool is set up to cache historical interaction sequences. During the training phase, a batch of data is randomly selected for training. The target network is constructed using the following parameters: Target network It is used to calculate the target value and prevent parameter oscillations caused by frequent updates. The functional structure diagram of DQN is as follows: Figure 4 As shown. The Q value of the target network is calculated as follows:

[0085]

[0086] Loss function and parameter update: Optimize the current Q network parameters using MSE (Mean Square Error). :

[0087]

[0088] The third step is policy selection based on Proximal Policy Optimization (PPO). This is an improved algorithm based on the Actor-Critic architecture. In the Actor-Critic algorithm, the Actor decides the action based on the environment state, while the Critic evaluates the potential reward value that action may obtain. They are represented by different neural networks: the Actor is a policy-based reinforcement learning model, and the Critic is a value-based reinforcement learning model. Although the Actor-Critic algorithm can learn autonomously in the environment, its convergence often takes a long time because its training and sample efficiency are usually low. Learning simple policies may require thousands of samples, while learning complex policies requires even more. The PPO algorithm effectively solves this problem by limiting the update range of the actor network's policy through a pruning function. After double pruning, the PPO algorithm is used for the final decision selection. The PPO agent learns the probability distribution within the action space. Because candidate actions have undergone both logical and value-based pruning, PPO can focus on capturing micro-level learning preferences and path smoothness. The structure diagram of the PPO algorithm is shown below. Figure 5 As shown.

[0089] Actor-Critic Collaborative Architecture:

[0090] (1) Actor (Policy Network) Based on the current knowledge profile of the learner. Using this as input, calculate the execution probability distribution for each resource among the high-value selected actions after DQN filtering.

[0091] (2) Critic (Value Network) ): The benchmark value for assessing the current state of the portrait. It provides feedback to the Actor. It evaluates the superiority of the current action relative to the average level by calculating a superiority function.

[0092] Core loss function and optimization:

[0093] To ensure the stability of the recommendation path and avoid drastic jumps in recommended content due to minor fluctuations in user profiles, PPO uses a pruning objective function to optimize the Actor network.

[0094]

[0095] in, It is the empirical expectation, which represents the average value of the learning samples in a batch. (Dominance function) is derived from The calculation shows that, among which It is a comprehensive reward that includes knowledge gain and fatigue penalty. The probability ratio between the old and new strategies reflects the extent of the update to the recommendation logic. The function limits the update range to To prevent system crashes caused by overly abrupt policy updates, the generated learning paths should have a smooth transition.

[0096] Evaluation benchmarks for Critic networks:

[0097] The Critic network continuously corrects its assessment accuracy of profile value by minimizing the TD Error. Its loss function is expressed as:

[0098]

[0099] in, It is a timing difference error. It is the total value actually observed. It is the estimated value of the current state; the Critic network is based on the student's current profile. The estimated total reward points that can be obtained from now until the end of the study period. It is the estimated value of the next state when the student completes the current step and enters the new state. The network predicts its future total score. This is a discount factor used to adjust the system's emphasis on long-term gains, encouraging the algorithm to ensure immediate gains while also achieving long-term knowledge acquisition goals.

[0100] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A personalized learning path recommendation method based on deep reinforcement learning, characterized in that, include: Acquire educational datasets and construct a knowledge graph based on triples as the basic unit. The system acquires the user's learning sequence and inputs it into the question-answering model to obtain the probability that the user will correctly answer the next question. Based on the knowledge graph and the probability that the user will correctly answer the next question, the system filters the learning paths to obtain the optimal learning path. The optimal learning path is then recommended to the user.

2. The personalized learning path recommendation method based on deep reinforcement learning according to claim 1, characterized in that, The knowledge graph is constructed as follows: acquire knowledge nodes and resource nodes; construct the attribution relationship between knowledge nodes and resource nodes based on a predefined mapping matrix; use association rule technology to deduce the prerequisite dependency relationship between knowledge points from learners' real answer data, and then construct a directed prerequisite graph; calculate question consistency, set a consistency threshold, and when the question consistency is less than the consistency threshold, it is determined that the two questions lack statistical significance and the data is deleted. Calculate the confidence level of causal relationships in the filtered question pairs; Set a minimum confidence threshold, and filter question pairs that exceed the threshold based on the minimum confidence threshold; map the correlation confidence between questions to the knowledge point level to obtain the correlation weight between knowledge points; Strong associations are obtained by associating the edges of the pre-prepared graph based on correlation weights; Based on strong association, cosine similarity is used to quantify learning resources. and The overlap in the coverage of knowledge points; based on the overlap, edges are constructed to obtain the knowledge graph.

3. The personalized learning path recommendation method based on deep reinforcement learning according to claim 2, characterized in that, Using cosine similarity to quantify learning resources and The degree of overlap in the coverage of knowledge points is: ； in, This represents the inner product of two resource feature vectors.

4. The personalized learning path recommendation method based on deep reinforcement learning according to claim 1, characterized in that, The question-answering model includes: an embedding layer, an aggregation layer, a dynamic knowledge state transition layer, and a prediction layer.

5. The personalized learning path recommendation method based on deep reinforcement learning according to claim 4, characterized in that, The question-answering model processes the input data as follows: defining a time-step tuple sequence containing activity type identifiers; inputting the time-step tuple sequence into the embedding layer to obtain a high-dimensional continuous vector; aggregating the high-dimensional continuous vector based on similar edges in the knowledge graph as local aggregation paths in the graph neural network; defining a static key matrix and a dynamic value matrix; constructing a knowledge state matrix based on the aggregated vectors; correcting the knowledge state matrix using the static key matrix and the dynamic value matrix; performing transformation perception and erasure operations on the corrected knowledge state matrix, and adding knowledge increments; inputting the features with added knowledge increments into the prediction layer to obtain the probability that the user correctly answers the next question.

6. The personalized learning path recommendation method based on deep reinforcement learning according to claim 5, characterized in that, Aggregating high-dimensional continuous vectors includes: high-dimensional continuous vectors comprising question embeddings and difficulty embeddings; fusing question embeddings and difficulty embeddings to obtain an enhanced representation, and aggregating similar and dissimilar neighbors based on similar edges in the knowledge graph to obtain an aggregated representation of evaluative resources; for non-evaluative resources, aggregating similar and dissimilar neighbors based on similar edges in the knowledge graph to obtain an aggregated representation of non-evaluative resources; wherein, similar neighbor aggregation is used to propagate local structural information between learning resources of the same type, and cross-type neighbor aggregation is used to propagate the association information between evaluative and non-evaluative resources, and the aggregation weight is obtained by normalizing the similar edge weights.

7. The personalized learning path recommendation method based on deep reinforcement learning according to claim 5, characterized in that, The modification of the knowledge state matrix includes: defining a static key matrix and a dynamic value matrix; calculating the influence weight of the current interaction activity on each knowledge point based on its aggregated representation; constructing joint interaction features according to the activity type, and generating erasure vectors and addition vectors from the joint interaction features; selecting the corresponding state transition matrix according to the switching relationship between the current activity type and the activity type at the previous time step, and performing transformation perception processing on the dynamic value matrix at the previous time step; using the erasure vector to perform an erasure operation on the transformed knowledge state; and writing the addition vector into the corresponding knowledge point according to the influence weight to complete the knowledge increment addition and obtain the updated knowledge state matrix.

8. The personalized learning path recommendation method based on deep reinforcement learning according to claim 5, characterized in that, Including the input of features with added knowledge increments into the prediction layer: ； ； in, For the summary vector, For activation function, The weight matrix for generating the summary vector, To grasp the degree, For splicing operations, The original title was embedded. The bias vector for generating the summary vector. The weight matrix for the output probability of the prediction layer. This is the bias vector for the output probability of the prediction layer.

9. The personalized learning path recommendation method based on deep reinforcement learning according to claim 1, characterized in that, The learning path selection process includes: defining the state space, action space, state transition function, reward function, and discount factor; performing knowledge graph-based logical pruning, which involves retrieving prerequisite relationships between knowledge points and calculating knowledge mastery; when the mastery of all direct prerequisite knowledge points of a knowledge point reaches a preset threshold, the knowledge point and its associated learning resources are added to the candidate pool; performing DQN-based value coarse screening: using the current knowledge profile state as input, a deep neural network is used to fit the action value function to calculate the Q value, and the candidate actions are sorted in descending order based on the calculated Q value, retaining the top K resources with high value potential to construct a selected action set; and performing PPO-based policy selection: in the selected action set, the policy network and value network of the PPO agent are used to calculate the execution probability distribution of each resource, and the optimal learning path is output.

Images