Knowledge point intelligent navigation method for learner competency model

Through low-cost deep neural network model and intelligent navigation method, the problem of personalized learning platforms relying on historical behaviors and biases in the existing technology is solved, and the learner's fair and personalized course recommendations and shortest paths to achieve target competency are achieved. .

CN118966331AActive Publication Date: 2024-11-15SHAANXI NORMAL UNIV
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202410975746.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-11-15
Estimated Expiration
2044-07-19

AI Technical Summary

Technical Problem

The existing personalized learning platforms rely on historical behaviors and biases, which leads to unfair and accurate recommendations, making it difficult for learners to acquire updated knowledge and effectively achieve learning goals.

Method used

Using a low-cost deep neural network model, based on the learner's initial competency, goal competency and learning interest, we recommend learning paths that conform to educational laws to learners through intelligent navigation methods, and continuously optimize recommendation strategies through reinforcement learning mechanisms to ensure learners Be able to choose courses to achieve the goal competency.

Benefits of technology

A fair and personalized course recommendation for each learner is achieved, reducing the number and time of courses required for learning, ensuring that learners can achieve the goal competency at the shortest path, and avoiding the influence of historical data dependence and bias.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966331A_ABST
    Figure CN118966331A_ABST
Patent Text Reader

Abstract

The invention provides a knowledge point intelligent navigation method for a learner competency model, and the method comprises the steps: obtaining the initial competency information of a learner and a target knowledge point to be learned for achieving the target competency from the knowledge point level based on a low-cost deep neural network model, recommending a knowledge point which can be learned at present for the learner, and enabling the learner to know the target competency. The learner enters a course containing the knowledge point to learn, tests are carried out before and after learning, an award obtained after the current action is executed is calculated according to an award function, and after learning is finished, a next knowledge point is recommended for the learner according to the obtained award, and the operation is repeated continuously; the method is advantaged in that the target knowledge point is acquired until the learner learns the target knowledge point, the target competency is reached, a course path to be learned by the learner to reach the target competency is acquired, through integration and analysis of teaching data, a more accurate and personalized learning path is provided for the student, and improvement of learning efficiency and effect of the student is facilitated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of computer technology, and in particular relates to a knowledge point intelligent navigation method oriented to a learner competency model. Background Art

[0002] "Building a networked, digital, personalized, and lifelong education system" is an important condition for building a learning society. With the support of many new technologies such as mobile Internet, big data, artificial intelligence, and cloud computing, education models, teaching methods, and learning methods are undergoing profound changes, prompting my country's education to move towards wisdom and intelligence. Among them, the concepts of "paying attention to individual differences among learners" and "providing appropriate education for each learner" are gradually gaining consensus in society. Personalized education and personalized learning have become the main ways to solve the current contradiction between supply and demand of social education. In the new media environment represented by self-media, learning resources are extremely rich. How to use intelligent technology to provide learners with customized learning paths is a hot topic in the current field of personalized learning research.

[0003] The existing personalized course recommendations are based on the learner's previous learning level and learning goals. The platform will customize the learning path so that the learner can learn effectively. Some personalized learning platforms also use machine learning algorithms to analyze the learner's learning behavior and ability, etc., to recommend learning courses. However, this method also has some disadvantages. For example, the algorithm tends to recommend content that the learner was interested in in the past. Over-reliance on some historical behaviors of learners for recommendation will result in learners not being able to learn newer knowledge. In addition, the algorithm recommendation may be affected by some biases in the system, causing some deviations in the analysis of some learners, affecting the accuracy and fairness of the recommendation system. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a knowledge point intelligent navigation method for learners' competency models. Based on the differences in learners' learning abilities, knowledge backgrounds, learning interests, and goals, intelligent technology is used to customize a learning path for learners that conforms to educational laws and can achieve learning goals, while realizing learners' learning status detection.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a knowledge point intelligent navigation method for learner competency model, based on a low-cost deep neural network model, obtains a learner's initial competency information and the target knowledge point to be learned to achieve the target competency from the knowledge point level, recommends a knowledge point that can be learned currently to the learner, and the learner enters the course containing the knowledge point to learn, takes a test before and after learning, calculates the reward obtained by executing the current action according to the reward function, and after the learning is completed, recommends the next knowledge point to the learner according to the reward obtained, and repeats continuously; until the learner learns the target knowledge point and achieves the target competency, and obtains the course path that the learner needs to learn to achieve the target competency;

[0006] The low-cost deep neural network model transforms the entire process into a state s t 、Action a t , Reward t The Markov decision process composed of elements, by exploring the Markov decision process, learn how to generate the course route of the course learned to achieve the target competency. In the learning process, the current state is used as the input of the intelligent agent, the intelligent agent outputs an action, and the reward for doing this action in the current state is used to stimulate the intelligent agent to continue learning;

[0007] Among them, the knowledge points that can be learned currently are obtained from the candidate knowledge point set, and the candidate knowledge point set is obtained through the knowledge point navigation method.

[0008] Furthermore, the specific method of training a deep neural network model with low learning cost includes the following steps:

[0009] S1, randomly initializes the parameters of the deep neural network model with low learning cost;

[0010] S2, initialize the experience pool H;

[0011] S3, initialize a new course collection It is used to store the courses that a learner has studied and has improved his competence. Initially empty;

[0012] S4, randomly select a learner l i And obtain its target knowledge points

[0013] S5, initialize knowledge point learning record And obtain the initial state s0;

[0014] S6, obtaining a candidate knowledge point set through a knowledge point navigation method;

[0015] S7, select a knowledge point from the candidate knowledge point set as the current action Then enter the course that contains this knowledge point To study;

[0016] S8, will complete action a t The new competence generated is compared with the initial competence to obtain the corresponding reward under the current state Update the knowledge point learning record L and obtain the next step status s t+1 ;

[0017] S9, the course selected at this time Put in In the collection;

[0018] S10, (s t ,a t ,r t ,s t+1 ) into the experience pool H.

[0019] S11, using state s t+1 Replaces t ;

[0020] S12, randomly collect a sample (s i ,a i ,r i ,s i+1 );

[0021] S13, calculate the target Q value:

[0022] S14, calculation error Used to train model parameters;

[0023] S15, if d i -t i =0, the learner reaches the target competency; if d i <t i , then return to step S7 and execute in sequence until the learner reaches the target competency.

[0024] Furthermore, when selecting a knowledge point from the candidate knowledge point set as the current action, the Epsilon-Greedy algorithm is used for exploration.

[0025] Furthermore, obtaining a candidate knowledge point set through a knowledge point navigation method includes:

[0026] According to the user's knowledge point learning record and knowledge structure diagram, a set of candidate knowledge points suitable for learning is screened out. The knowledge point navigation method can reduce unnecessary exploration of the model in the environment.

[0027] The nodes of the known knowledge point structure graph G correspond to the knowledge points, and the edges correspond to the precedence relationship between the knowledge points. It is a directed acyclic graph. The set of learned knowledge points K a , the set of knowledge points that have not been learned yet K e , target knowledge point set K t , the steps to determine the candidate knowledge point set K are:

[0028] S21, all the learned knowledge points K a And knowledge point K a The edge of is deleted from the knowledge point structure graph G;

[0029] S22, the knowledge points K that have not been learned yet e Put it into the candidate knowledge point set K and delete K from the knowledge point structure graph G e and its edges;

[0030] S23, traverse all nodes in the knowledge point structure graph G, if the knowledge point set K that has not been learned e and the target knowledge point set K t If there is no path between them, delete the unlearned knowledge point in the knowledge point structure graph G;

[0031] S24, traverse all nodes in the knowledge point structure graph G, and store all nodes with in-degree 0 into the candidate knowledge point set K.

[0032] S25, output the candidate knowledge point set K.

[0033] Furthermore, the loss function of the deep neural network model training process is as follows:

[0034]

[0035] where x i is the target Q value used by the DQN algorithm in training step i:

[0036]

[0037] in It means Q(s t ,a t ) The parameters of the initial network, Represents the parameters of the target network.

[0038] Furthermore, the target Q value is calculated by the following formula:

[0039]

[0040] Furthermore, both the initial competence and the target competence are composed of the learners' knowledge, and both the initial competence and the target competence are tested through exercises.

[0041] Furthermore, in the deep neural network model, the state space is set. The information contained in the state s∈S needs to be sufficient for the deep neural network model to determine the action with the lowest learning cost that can achieve the target competency in the current state. The information includes the following four parts: learner l i , learners' initial competence i , learners' target competencies i And the student's curriculum system collection

[0042] After selecting the learner's information, the agent starts from the starting point and passes through the learner l i , learners' initial competence i , learners' target competencies i And the student's curriculum system collection Get the current state of step t, expressed as

[0043]

[0044] Given the state s at step t t , select an action from the action space to execute. In the action space, the action of step t consists of the learner and a knowledge point selected by the learner. After the current action is determined, the next course is selected. The action of step t is expressed as represents the learner corresponding to the tth step, Represents the knowledge point selected in the tth step; after the agent takes the tth step, the environment will calculate the reward value of the tth step as feedback for the current action.

[0045] Furthermore, the decision made by the agent will be affected by the reward function. After learning a course, the greater the improvement in competence of the learner, the greater the reward will be. The reward r at step t is t for:

[0046]

[0047] Before learning the course containing this knowledge point, learners are evaluated through two tests. The score of the test before learning is grade init , the score of the test after learning is grade final , the total score is grade total .

[0048] Furthermore, in the low-cost deep neural network model, two convolution operations are performed to extract features of all the courses that the learner is going to learn. After two layers of convolution, the features obtained are rearranged as the input of the subsequent neural network containing three layers of fully connected layers, and finally the Q(s) corresponding to the optional actions in the current state is obtained. t ,a t ) value, the agent will choose Q(s t ,a t ) is the action with the largest value, which is taken as the action a of the current step t t . Compared with the prior art, the present invention has at least the following beneficial effects: the present invention constructs a low-cost deep neural network model, so that each learner can reach the target competency from the initial competency by learning the recommended courses. The advantage of the present invention is that it can recommend courses to each learner fairly and individually. When recommending courses to each learner, it does not rely on historical data. Through the reinforcement learning agent, the best course that can achieve the target competency is selected from a series of courses each time, so that the course has the greatest impact on the learner. Secondly, since the course selected each time is the best, the course path required for the learner to reach the target competency is the shortest, which greatly saves the learner's learning time. Compared with the prior art, learners do not need to learn many courses at one time, but only need to learn the minimum number of courses to reach the target competency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 is a flow chart of Embodiment 1 of the present invention;

[0050] Figure 2 It is a flowchart for training low-cost deep learning networks;

[0051] Figure 3 It is a structural diagram of a low-cost deep learning network. DETAILED DESCRIPTION

[0052] The present invention is further described below with reference to the accompanying drawings and examples, but the present invention is not limited to the following embodiments.

[0053] Example 1

[0054] refer to Figure 1 The research method for intelligent navigation of knowledge points in the learner competency model of this embodiment includes the following steps:

[0055] (1) Dataset Construction

[0056] A data set including multiple learners and multiple courses that each learner needs to learn is constructed. The data set includes a set of learners, a set of course systems for each student, initial competence, target competence, and learning cost.

[0057] The learner set L is as follows:

[0058] L = {i|i represents the i-th learner, 1≤i≤n};

[0059] The student curriculum set SC is as follows:

[0060] SC={C1,C2,...,C i ,...,C n};

[0061] Among them, C i It is the set of course systems of learner i.

[0062] represents the jth course studied by the i-th student, 1≤j≤|C i |},|C i |Indicates the total number of courses that the student is going to study.

[0063] Initial competence i for:

[0064]

[0065] in, represents the initial knowledge of learner i,

[0066] Target competency i for:

[0067]

[0068] in, Representation is the target knowledge of learner i.

[0069] The learning cost V is the number of courses that learners need to take to reach their target competence from their initial competence.

[0070] (2) Quantifying initial competencies and target competencies

[0071] Both the initial competence and the target competence are composed of the learner’s knowledge. The vector of knowledge points is recorded as: K = {K1, K2, ..., K c}, c represents the total number of knowledge points, which is a finite positive integer. The knowledge that learners possess can be tested through exercises.

[0072] (3) Build a deep neural network model with low learning cost

[0073] The deep neural network model can enable learners to improve their competence towards the target competence with the lowest cost after learning a corresponding series of courses, and eventually convert the original long learning path into a shorter path. Learners can learn in sequence and eventually reach the target competence. The deep neural network model divides the learning path into two aspects: one is the knowledge point level, and the other is the course learning level. The knowledge point level needs to consider which knowledge points learners need to master to reach the target competence, and the course learning level needs to consider which courses need to be watched to master a certain knowledge point.

[0074] In the deep neural network model, the problem to be considered is transformed into a Markov decision process, and a deep reinforcement learning agent is used to iteratively select the knowledge points to be selected, thereby obtaining the required courses, so that the students' competence can be improved and the target competence can be achieved. The problem can be described as a state s t 、Action a t , Reward t The Markov decision process consists of elements. By exploring the Markov decision process, we learn how to generate a course route for the courses learned to achieve the target competency. In the learning process, the current state is used as the input of the agent, the agent outputs an action, and gets a reward for doing this action in the current state, so as to stimulate the agent to learn.

[0075] Status t : is a collection of states, Indicates embedding the record of the current learner’s knowledge point learning into a vector, It represents the knowledge point learned by the i-th student at time t. Where φ is the knowledge point embedding function, L i It is a record of the knowledge points of the learner's history learning.

[0076] Action a t : is a collection of actions, an action a t Corresponding to a knowledge point

[0077] Reward t : The reward function is related to the current action and the next state, that is, r t (s t ,a t ,s t+1 ). The accumulated sum of rewards is expressed as follows:

[0078] R=r t +r t+1

[0079] In this model, from the knowledge point level, the initial competency information of a learner and the target knowledge point to be learned to achieve the target competency are first obtained. Based on this information, the model will recommend a knowledge point that the learner can currently learn. The learner enters the course that contains the knowledge point to learn, and takes tests before and after learning. The reward obtained by the model after executing the current action is calculated according to the reward function, and after the learning is completed, the recommendation strategy is optimized according to the reward obtained. Subsequently, the model recommends the next knowledge point to the learner based on the reward obtained, and repeats continuously. Finally, the learner will learn the target knowledge point and achieve the target competency.

[0080] The general process of the deep neural network model with low learning cost is as follows:

[0081] 11) The learner already has a known initial competency before learning, and the learner’s target competency is also known.

[0082] 12) The low-learning-cost deep neural network model recommends courses covering knowledge points that are currently suitable for learners to learn based on their initial competencies and target competencies.

[0083] 13) After completing the recommended courses, learners will take the recommended course exercise test and save the test records. This test will affect the learner's competency and enable the learner to reach a new competency.

[0084] 14) Use the learner’s initial competence and the new competence gained after completing each course to calculate the learner’s progress after completing each course as a reward.

[0085] 15) After a learner finishes a course, the historical learning record changes. After receiving the new learning record and reward, the model recommends the next suitable knowledge point to the learner, and the learner continues to learn the next course.

[0086] 16) Each time you finish a course, put the currently completed course into a new course collection.

[0087] 17) After the deep neural network model with low learning cost obtains the learner's new competence, it compares the new competence with the target competence. If the target competence is reached, the learning ends; if the target competence is not reached, steps 13) to 16) are repeated.

[0088] 18) Learning does not end until the current learner's new competencies reach the target competencies.

[0089] (4) Knowledge Point Navigation

[0090] This embodiment designs a knowledge point navigation method, which selects a set of candidate knowledge points suitable for learning at present according to the user's knowledge point learning record and knowledge structure diagram. The knowledge point navigation method can reduce unnecessary exploration of the model in the environment.

[0091] The nodes of the known knowledge point structure graph G correspond to the knowledge points, and the edges correspond to the precedence relationship between the knowledge points. It is a directed acyclic graph. The set of learned knowledge points K a , the set of knowledge points that have not been learned yet K e , target knowledge point set K t , the steps to determine the candidate knowledge point set K are:

[0092] 21) All the learned knowledge points K a And knowledge point K a The edge of is deleted from the knowledge point structure graph G;

[0093] 22) The knowledge points that have not been learned yet K e Put it into the candidate knowledge point set K and delete K from the knowledge point structure graph G e and its edges;

[0094] 23) Traverse all nodes in the knowledge point structure graph G. If the knowledge point set K that has not been learned e and the target knowledge point set K t If there is no path between them, the unlearned knowledge point will be deleted from the knowledge point structure graph G.

[0095] 24) Traverse all nodes in the knowledge point structure graph G, and store all nodes with in-degree 0 into the candidate knowledge point set K.

[0096] 25) Output the candidate knowledge point set K.

[0097] (5) Training deep neural network models with low learning costs

[0098] The loss function of the training process of the deep neural network model with low learning cost is as follows:

[0099]

[0100] where x i is the target Q value used by the DQN algorithm in training step i:

[0101]

[0102] in It means Q(s t ,a t ) The parameters of the initial network, Represents the parameters of the target network.

[0103] The core algorithm of the low-learning-cost deep neural network model is the DQN (Deep Q-Network) algorithm. The specific method of training the low-learning-cost deep neural network model is as follows:

[0104] 1) Randomly initialize the parameters F of the deep neural network model with low learning cost;

[0105] 2) Initialize the experience pool H;

[0106] 3) Initialize a new course collection Used to store the courses that the learner has studied and has improved his competence. Initially empty;

[0107] 4) Randomly select a learner l i And obtain its target knowledge points

[0108] 5) Initialize knowledge point learning records And obtain the initial state s0;

[0109] 6) Obtain a set of candidate knowledge points through a knowledge point navigation method.

[0110] 7) Use the Epsilon-Greedy algorithm to explore and select a knowledge point from the candidate knowledge point set as the current action Then enter the course that contains this knowledge point To study;

[0111] 8) will complete action a t The new competence generated is compared with the initial competence to obtain the corresponding reward under the current state Update the knowledge point learning record L and obtain the next step status s t+1 ;

[0112] 9) Select the course you want to Put in In the collection;

[0113] 10) will (s t ,a t ,r t ,s t+1 ) into the experience pool H.

[0114] 11) Status t+1 Replaces t ;

[0115] 12) Randomly collect a sample (s) from the experience replay pool H i ,a i ,ri ,s i+1 );

[0116] 13) Calculate the target Q value as follows:

[0117]

[0118] 14) Calculation error To train the network F;

[0119] 15) If d i -t i = 0, the learner reaches the target competency and the process ends; if d i <t i , then go to step 7);

[0120] (6) Testing low-cost deep neural network models

[0121] The test set is input into a trained low-cost deep neural network model to output the course path learned to achieve the target competency.

[0122] (7) Low-cost deep neural network performance evaluation

[0123] A low-cost deep neural network model is used to assess whether each learner has achieved the target competency after completing a series of courses and to list the course paths to achieve the target competency.

[0124] Example 2

[0125] The process of training a low-cost deep learning network in this embodiment includes the following steps:

[0126] 1) Randomly initialize the parameters of this low learning cost deep neural network model;

[0127] 2) Initialize the experience pool H;

[0128] 3) Initialize a new course collection Used to store the courses that the learner has studied and that have improved his / her competence. Initially empty;

[0129] 4) Randomly select a learner l i And obtain its target knowledge points

[0130] 5) Initialize knowledge point learning records And obtain the initial state s0;

[0131] 6) Obtain a set of candidate knowledge points through knowledge point navigation.

[0132] 7) Use the Epsilon-Greedy algorithm to explore and select a knowledge point from the candidate knowledge point set as the current action Then enter the course that contains this knowledge point To study;

[0133] 8) will complete action a t The new competence generated is compared with the initial competence to obtain the corresponding reward under the current state Update the knowledge point learning record L and obtain the next step status s t+1 ;

[0134] 9) Select the course you want to Put in In the collection;

[0135] 10) will (s t ,a t ,r t ,s t+1 ) into the experience pool H;

[0136] 11) Status t+1 Replaces t ;

[0137] 12) Randomly collect a sample (s) from the experience replay pool H i ,a i ,r i ,s i+1 );

[0138] 13) Calculate the target Q value as follows:

[0139]

[0140] 14) Calculation error To train model F;

[0141] 15) If d i -t i = 0, the learner reaches the target competency and the process ends; if d i <t i , then go to step 7);

[0142] Example 3

[0143] The research method for intelligent navigation of knowledge points in the learner competency model of this embodiment includes the following steps: Steps (1) to (7) are the same as those in Embodiment 1;

[0144] The low-cost deep neural network model construction of this embodiment includes the following steps:

[0145] 1) The learner already has a known initial competency before learning, and the learner’s target competency is also known.

[0146] 2) The model recommends courses covering the knowledge points that are currently suitable for learners to learn based on their initial competencies and target competencies.

[0147] 3) After completing the course, learners will take the course test and save the test record. This test will affect the learner's competency and enable the learner to reach a new competency.

[0148] 4) The learner’s initial competence and the new competence gained after completing each course are used to calculate the learner’s progress after completing each course as a reward for the model.

[0149] 5) After a learner finishes a course, the historical learning record will change. After receiving the new learning record and reward, the model will recommend the next suitable knowledge point to the learner and continue to learn the next course.

[0150] 6) Each time you finish a course, put it into a new course collection.

[0151] 7) After the model obtains the learner's new competency, it compares it with the target competency. If the target competency is reached, the learning ends; if the target competency is not reached, steps 3 to 6 are repeated.

[0152] 8) Learning ends when the learner reaches the target competency.

[0153] Aiming at the problem of students' ideal learning efficiency and learning effect, a knowledge point intelligent navigation method for learners' competency model is proposed. Intelligent knowledge point navigation plays an important role in the cultivation of students' competence. First, by integrating and analyzing teaching data, it provides students with more accurate and personalized learning paths, which helps to improve students' learning efficiency and effect; secondly, intelligent knowledge point navigation can provide a visual presentation of the internal relationship of knowledge points, help students build deep cognition, and further improve their understanding and application capabilities. In addition, intelligent knowledge point navigation can also enable teachers to better understand students' learning situation, so as to provide them with more accurate guidance. Teachers can also recommend customized exercises and wrong question consolidation content based on students' learning situation and learning ability. In the era of artificial intelligence, the application of intelligent knowledge point navigation can also train students' thinking mode and improve their digital competence.

Claims

1. A knowledge point intelligent navigation method for learner competency model, characterized in that: Based on the low-cost deep neural network model, from the knowledge point level, the initial competency information of a learner and the target knowledge point to be learned to achieve the target competency are obtained, and a knowledge point that can be learned at present is recommended to the learner. The learner enters the course containing the knowledge point to learn, and takes tests before and after learning. The reward obtained by completing the current action is calculated according to the reward function, and after the learning is completed, the next knowledge point is recommended to the learner based on the reward obtained, and it is repeated continuously until the learner learns the target knowledge point and achieves the target competency, and the course path that the learner needs to learn to achieve the target competency is obtained; the low-cost deep neural network model transforms the whole process from state s t 、Action a t And reward r t The Markov decision process composed of the above two parts is used to learn how to generate the course route of the courses learned to achieve the target competency by exploring the Markov decision process. In the learning process, the current state is used as the input of the intelligent agent, the intelligent agent outputs an action, and the reward for doing this action in the current state is used to stimulate the intelligent agent to continue learning; among them, the knowledge points that can be learned at present are obtained from the candidate knowledge point set, and the candidate knowledge point set is obtained through the knowledge point navigation method.

2. The method for intelligent navigation of knowledge points based on learner competency model according to claim 1 is characterized in that: The specific method of training a deep neural network model with low learning cost includes the following steps: S1, randomly initializes the parameters of the deep neural network model with low learning cost; S2, initialize the experience pool H; S3, initialize a new course collection It is used to store the courses that a learner has studied and has improved his competence. Initially empty; S4, randomly select a learner l i And obtain its target knowledge points S5, initialize knowledge point learning record And obtain the initial state s0; S6, obtaining a candidate knowledge point set through a knowledge point navigation method; S7, select a knowledge point from the candidate knowledge point set as the current action Then enter the course that contains this knowledge point To study; S8, will complete action a t The new competence generated is compared with the initial competence to obtain the corresponding reward under the current state Update the knowledge point learning record L and obtain the next step status s t+1 ; S9, the course selected at this time Put in In the collection; S10, (s t ,a t ,r t ,s t+1 ) into the experience pool H; S11, using state s t+1 Replaces t ; S12, randomly collect a sample (s i ,a i ,r i ,s i+1 ); S13, calculate the target Q value: S14, calculation error Used to train model parameters; S15, if d i -t i =0, the learner reaches the target competency; if d i <t i , then return to step S7 and execute in sequence until the learner reaches the target competency.

3. The knowledge point intelligent navigation method for learner competency model according to claim 2 is characterized in that: When selecting a knowledge point from the candidate knowledge point set as the current action, the Epsilon-Greedy algorithm is used for exploration.

4. The method for intelligent navigation of knowledge points based on learner competency model according to claim 2, characterized in that: The candidate knowledge point set obtained by the knowledge point navigation method includes: According to the user's knowledge point learning record and knowledge structure diagram, a set of candidate knowledge points suitable for learning is screened out. The knowledge point navigation method can reduce unnecessary exploration of the model in the environment. The nodes of the known knowledge point structure graph G correspond to the knowledge points, and the edges correspond to the precedence relationship between the knowledge points. It is a directed acyclic graph. The set of learned knowledge points K a , the set of knowledge points that have not been learned yet K e , target knowledge point set K t , the steps to determine the candidate knowledge point set K are: S21, all the learned knowledge points K a And knowledge point K a The edge of is deleted from the knowledge point structure graph G; S22, the knowledge points K that have not been learned yet e Put it into the candidate knowledge point set K and delete K from the knowledge point structure graph G e and its edges; S23, traverse all nodes in the knowledge point structure graph G, if the knowledge point set K that has not been learned e and the target knowledge point set K t If there is no path between them, delete the unlearned knowledge point in the knowledge point structure graph G; S24, traverse all nodes in the knowledge point structure graph G, and store all nodes with in-degree 0 into the candidate knowledge point set K; S25, output the candidate knowledge point set K.

5. The method for intelligent navigation of knowledge points based on learner competency model according to claim 2, characterized in that: Loss function of deep neural network model training process as follows: where x i is the target Q value used by the DQN algorithm in training step i: in It means Q(s t ,a t ) The parameters of the initial network, Represents the parameters of the target network.

6. The method for intelligent navigation of knowledge points based on learner competency model according to claim 5, characterized in that: The target Q value is calculated by the following formula:

7. The method for intelligent navigation of knowledge points based on learner competency model according to claim 1, characterized in that: Both initial competencies and target competencies are composed of learners' knowledge, and both initial competencies and target competencies are tested through exercises.

8. The method for intelligent navigation of knowledge points based on learner competency model according to claim 1, characterized in that: In the deep neural network model, the state space is set. The information contained in the state s∈S needs to be sufficient for the deep neural network model to determine the action with the lowest learning cost that can achieve the target competency in the current state. The information includes the following four parts: learner l i , learners' initial competence i , learners' target competencies i And the student's curriculum system collection After selecting the learner's information, the agent starts from the starting point and passes through the learner l i , learners' initial competence i , learners' target competencies i And the student's curriculum system collection Get the current state of step t, expressed as Given the state s at step t t , select an action from the action space to execute. In the action space, the action of step t consists of the learner and a knowledge point selected by the learner. After the current action is determined, the next course is selected. The action of step t is expressed as represents the learner corresponding to the tth step, Represents the knowledge point selected in the tth step; after the agent takes the tth step, the environment will calculate the reward value of the tth step as feedback for the current action.

9. The method for intelligent navigation of knowledge points based on learner competency model according to claim 1, characterized in that: The decision made by the agent will be affected by the reward function. After learning a course, the greater the improvement in competence of the learner, the greater the reward will be. The reward r at step t is t for: Before learning the course containing this knowledge point, learners are evaluated through two tests. The score of the test before learning is grade init , the score of the test after learning is grade final , the total score is grade total .

10. The method for intelligent navigation of knowledge points based on learner competency model according to claim 1, characterized in that: In the low-cost deep neural network model, two convolution operations are performed to extract features of all the courses that the learner needs to learn. After two layers of convolution, the features obtained are rearranged as the input of the subsequent neural network containing three layers of fully connected layers, and finally the Q(s) corresponding to the optional actions in the current state is obtained. t ,a t ) value, the agent will choose Q(s t ,a t ) is the action with the largest value, which is taken as the action a of the current step t t .

Citation Information

Patent Citations

  • A self-adaptive learning path planning system based on reinforcement learning

    CN109948054A

  • Course recommendation method and device based on attribution analysis, equipment and medium

    CN113420261A

  • Reinforcement learning method and system in adaptive learning path recommendation

    CN113434563A

  • Individualized learning path recommendation method based on reinforcement learning

    CN116521997A

  • Personalized and adaptive math learning system

    US20200258420A1