A method, system and device for recommending diversified exercises that combine difficulty and weak knowledge points

Abstract

A method, system and device for recommending diversified exercises that combine difficulty and weak knowledge points. The method comprises: firstly encoding a student's historical answer sequence to obtain model input data in a standard format; using an e-DKT model to model the exercises to obtain the probability of the student's correct answer to the exercises, and using a k-DKT model to model the knowledge points to obtain the student's knowledge point mastery; based on the correct answer probability and the knowledge point mastery, calculating the individual difficulty of the exercises and the student's weak knowledge points, using a filtering module to select exercises with moderate difficulty and containing weak knowledge points from an exercise set to construct a candidate subset; using a simulated annealing algorithm to select exercises with the largest spatial distance from the candidate subset to generate a recommended exercise list; designing an evaluation index to measure the effect of the recommendation, so that the exercise recommendations have accuracy, novelty and diversity; the system and device are used to recommend diversified exercises that combine difficulty and weak knowledge points; the present invention simultaneously ensures the accuracy, novelty and diversity of the exercise recommendations, and has the advantages of practicality and efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of educational data mining technology, and in particular relates to a method, system and device for recommending diversified exercises that combine difficulty and weak knowledge points. Background Art

[0002] With the continuous development of online learning, recommendation algorithms have also been applied to learning resources. Exercises, as a learning resource, play a vital role in students' personalized learning. Doing exercises is an effective way to test students' mastery of target knowledge points. Existing exercise recommendation algorithms can be roughly divided into two categories: traditional recommendation models, such as collaborative filtering, which draw on the recommendation philosophy of the e-commerce field; and recommendation models based on student ability assessment, such as cognitive diagnosis and knowledge tracking.

[0003] Drawing on the application of traditional recommendation algorithms in e-commerce, students on online learning platforms can be considered users in e-commerce, exercise resources can be considered commodities, and students' performance on the exercises can be considered user ratings of the commodities. Therefore, collaborative filtering algorithms can be applied for recommendation. To address the information shortage problem in collaborative filtering algorithms, Wu et al. proposed a multi-classifier-based transfer bagging exercise recommendation model. Segal et al. used User-CF to rank similar students and construct an exercise difficulty ranking for each student to assist teachers in assigning appropriate exercises.

[0004] However, recommendation algorithms in the e-commerce field often start from the user's interests, aiming to match recommended products with user preferences. This often results in recommended exercises that are not appropriately difficult for students. However, exercise recommendation, as a form of learning resource recommendation, differs from other recommendation areas in that it is not intended to predict or cater to students' potential behavior. Instead, it should help students discover learning resources that match their personalized parameters in a reasonable way during their learning process through recommended content, thereby addressing their weaknesses and improving learning efficiency.

[0005] Cognitive Diagnostic (CD) and Knowledge Tracing (KT) are currently widely used in the field of personalized exercise recommendation. Cognitive diagnosis mainly studies the overall learning data of students over a period of time to obtain the students' current static knowledge mastery level, while knowledge tracing can dynamically capture the students' knowledge mastery level over time. Zhu Tianyu et al. proposed an exercise recommendation model that combines cognitive diagnosis and probabilistic matrix decomposition. The student knowledge mastery obtained by the cognitive diagnosis model is applied to the probabilistic decomposition model, allowing the model to simultaneously consider the personality of the recommended students and the common characteristics of the group. Ma Xiaorui et al. proposed an exercise recommendation algorithm based on deep knowledge tracing. The student knowledge level matrix obtained by knowledge tracing is regarded as the user-item matrix in collaborative filtering. The accuracy rate of the exercises is calculated by combining the mean of the most similar user vectors and the weight of the vector itself, and exercises within a certain difficulty range are recommended.

[0006] However, these exercises recommendation algorithms based on student cognitive level only consider exercises of moderate difficulty based on students' knowledge level, that is, they only consider the accuracy of recommendations. For recommendation algorithms, considering only accuracy is limited, resulting in overly focused recommendations, which fails to bring freshness to users and reduces user experience satisfaction.

[0007] Patent application CN202210833098.3 proposes a deep knowledge tracking method and exercise recommendation method that integrates multiple features. The deep knowledge tracking method that integrates multiple features is used to predict the personalized difficulty of each exercise, and then screen them according to the expected difficulty. This invention only considers the difficulty information of the recommended exercises. Patent application CN202111537852.0 proposes a method for recommending exercises based on weaknesses. Based on the historical answer data of a single student, its weak knowledge points are sorted out, and the student’s current learning status is analyzed. Different recommended content is divided according to certain thresholds. This invention uses traditional rule matching methods and cannot fully conduct in-depth data mining. The above two inventions do not consider the individual difficulty of exercises and weak knowledge points at the same time, and neither considers the diversification of recommended exercise content. Summary of the Invention

[0008] In order to overcome the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a diversified exercise recommendation method, system and device that combines difficulty and weak knowledge points, uses the exercise-level DKT model to obtain the probability of students' correct answers to each exercise, and at the same time, uses the knowledge point-level DKT model to obtain the students' knowledge point mastery vector; then, based on the probability of correct answers to exercises and the knowledge point mastery vector, an exercise filtering module is constructed to filter out exercises with moderate difficulty and containing weak knowledge points, ensuring the accuracy and novelty of recommendations; finally, considering that the diversity of recommendation results helps to improve learning interest, a simulated annealing algorithm is used to construct an exercise screening module, and the filtered exercises are screened twice, and the exercises are regarded as points in space to solve the maximum spatial distance combinatorial optimization problem to ensure the diversity of recommendations; and combined with the characteristics of the exercise field and the recommendation evaluation method, evaluation indicators are designed to measure the effect of recommendations, so that exercise recommendations have both accuracy, novelty and diversity.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A method for recommending diversified exercises that combines difficulty and weak knowledge points, specifically comprising the following steps:

[0011] Step 1: Encode the student history question answer sequence at the input layer to obtain model input data in a standard format;

[0012] Step 2: Model the input exercise data in the prediction layer to obtain the probability of students answering the exercises correctly and the degree of students' mastery of knowledge points;

[0013] Step 3: Based on the correct answer probability and knowledge mastery level of the exercises obtained by the prediction layer, the filtering module is used to select candidate subsets of appropriate difficulty and containing students' weak knowledge points from the entire exercise set according to the individual difficulty of the exercises and the students' weak knowledge points;

[0014] Step 4: Use the simulated annealing algorithm to select the exercises with the largest spatial distance from the candidate subset to generate a recommended exercise list; and output the generated recommended exercise list REL;

[0015] Step 5: Evaluate the output recommended exercise list REL based on accuracy, novelty, and diversity indicators.

[0016] Furthermore, the specific process of step one is:

[0017] The student's historical answer sequence algorithm is used as input data and divided into the exercise level or knowledge point level to model the student status in the knowledge tracking field:

[0018] X={x1,x2,x3,…,x t} represents the student's history answer sequence, where e t Exercises that indicate answers, Indicates the answer to the exercise, or where k t Indicates the knowledge points of the answer, Indicates the answer status of knowledge points, 1 indicates a correct answer, and 0 indicates an incorrect answer;

[0019] Based on this, the student history answer sequence is expressed as <exercise, result> or <knowledge point, result>.

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] The prediction layer is divided into two sub-modules: the exercise-level knowledge tracking model and the knowledge point-level knowledge tracking model;

[0022] The exercises are modeled through the exercise-level knowledge tracking model, and its input corresponds to The output is the probability of the student answering each exercise correctly;

[0023] The knowledge points are modeled through the knowledge point hierarchical knowledge tracking model, and its input corresponds to The output is the students’ mastery of each knowledge point;

[0024] Both submodules in step 2 use the DKT model based on the LSTM network structure as the knowledge tracking model. The specific method is as follows:

[0025] The DKT (Deep Knowledge Tracing) model first introduced deep learning into the field of knowledge tracing. The DKT model inputs the student’s history answer sequence X = {x1, x2, x3, …, x t}, x t After feature cross encoding, it is transformed into the input of the model:

[0026]

[0027] e t is the one-hot encoding of the exercise answered at time t, 0 is the same as e t Zero vectors of equal length;

[0028] Input vector x t After the RNN hidden layer h t Feature extraction, get the output vector y t ,y t Represents the probability of answering the exercise correctly:

[0029] h t =tanh(W hx xt +W hh h t-1 +b h )

[0030] y t =σ(W yh h t +b y )

[0031] The loss function of the DKT model is as follows, where l is the binary cross entropy, δ(e t+1 ) is Exercise e t+1 One-hot encoding:

[0032]

[0033] The prediction layer uses the question-level DKT model (e-DKT) to obtain the probability of students’ correct answers to exercises. Similarly, the knowledge point-level DKT model (k-DKT) is used to obtain the students’ mastery of knowledge points.

[0034] Furthermore, the specific process of step three is as follows:

[0035] ES (Exercise Set) represents the entire exercise set, CS (Candidate Subset) represents the candidate exercise subset selected by the filtering module, P (K) represents the probability vector of students’ knowledge points mastered by the k-DKT model, and p e(K) represents the probability of correct answers to student exercises obtained by the e-DKT model; where K represents the set of knowledge points, and the knowledge points contained in exercise e are represented by e(K):

[0036] e(K)=[e(k1),e(k2),…,e(k n )]

[0037] n represents the number of knowledge point set K, e(k i )=1 means the exercise contains the corresponding knowledge point i, e(k i )=0 means not included;

[0038] Define the student's weak knowledge point vector W(K) = 1-P(K), and the individual difficulty of the exercise d e(K) =p e(K) , we hope that the vector similarity between the exercises and the students’ weak knowledge points is large, and at the same time we hope that the distance between the expected difficulty δ and the actual difficulty is small, so the filtering objectives are as follows:

[0039]

[0040] The high similarity between the vectors of exercises and students’ weak knowledge points is equivalent to the low similarity between the vectors of exercises and students’ mastered knowledge points, which is as follows:

[0041]

[0042] The similarity calculation uses cosine similarity and the distance metric uses subtraction. Finally, we can calculate the d of each exercise. j (K)'s score

[0043]

[0044] Then, based on the score of each exercise in the exercise set ES Sort and select the N exercises with the smallest values ​​to construct the candidate exercise set ES:

[0045]

[0046] in Indicates the Nth score after ascending sorting.

[0047] Furthermore, the filtering algorithm used in step 3 has the following specific process:

[0048] (1) Select Exercise e j (K), calculate the similarity cossim(e j (K), P(K)) and distance Get the final score

[0049] (2) Repeat step (1) to calculate the scores of all exercises in the exercise set ES;

[0050] (3) Sort the exercises in ascending order of scores;

[0051] (4) Select the top N exercises with the lowest scores to construct the subsequent subset CS.

[0052] Furthermore, the specific process of step 4 is as follows:

[0053] Each exercise contains one or more knowledge points. Exercises are considered as points in a high-dimensional space. The spatial dimension is the number of knowledge points in the dataset, n. Based on this, the task of generating a diverse exercise recommendation list can be transformed into solving M points with large distances in the high-dimensional space. M represents the number of exercises in the REL. The solution is solved using the simulated annealing algorithm (SA). The specific steps are as follows:

[0054] a. First, randomly select M exercises from the candidate exercise subset CS, initialize the current exercise list L, and calculate the exercise distance matrix DL of the current exercise list. DL is an M×M dimensional matrix, where each value represents the Euclidean distance between two exercises in the exercise list:

[0055]

[0056] DL (i,j) represents the value of row i and column j in the DL matrix, e i (K), e j (K) represents the knowledge point vector of the corresponding exercise in the exercise list L. It can be seen that DL (i,j) The larger the value of , the greater the distance between the two exercises in the high-dimensional space. At the same time, the average matrix distance is defined to measure the spatial distance of the entire exercise list L:

[0057]

[0058] The larger the mean(DL), the larger the spatial distance of the exercise list L, which means that the knowledge points contained are more diverse;

[0059] b. Randomly select an exercise from the candidate exercise subset CS to replace one of the current exercise list L, obtaining a new exercise list L′. At the same time, calculate the distance matrix DL′ and the average matrix distance mean(DL′) of the new exercise list;

[0060] c. Set the rules for accepting the new exercise list L′ according to the Metropolis acceptance criterion. When mean(DL′)>mean(DL), that is, the new exercise list L′ has a high degree of diversity, use the new exercise list L′ to update the current exercise list L. Otherwise, update the current exercise list L with the new exercise list L′ with probability p:

[0061]

[0062] Where T represents temperature, κ B represents the Boltzmann constant, κ B =1.380649×10 -23 J / K, the specific acceptance method is as follows:

[0063]

[0064] Where r=random(0,1) represents a random number between 0 and 1;

[0065] d. It is divided into two layers of iteration, the outer layer iteration is controlled by the temperature T. Each iteration will cool down with a specific attenuation coefficient c. The initial temperature is T0. After multiple iterations, when the temperature is less than the termination temperature T f If the number of iterations is s, the iteration stops, otherwise, the steps b and c are continued. The inner iterations are set to a fixed number s, so that multiple searches are performed at the same temperature T.

[0066] At the same time, in each iteration, the exercise list DL that maximizes the average matrix distance mean(DL) is solved, as shown below. When the entire iteration is completed, REL is output as the recommendation system list.

[0067] REL←argmax L (mean(DL))

[0068] Furthermore, the time complexity of the simulated annealing algorithm described in step 4 depends on the number of iterations of the inner and outer loops. Try to find the global optimal solution in a limited search. The specific algorithm flow is as follows:

[0069] 1) Randomly select M exercises, initialize the exercise list L, initialize the starting temperature T = T0, and the ending temperature T f , attenuation coefficient c, number of internal cycles s;

[0070] 2) Randomly replace the exercises in the exercise list L to generate a new exercise list L ′ ;

[0071] 3) Calculate the distance matrix DL and DL ′ ;

[0072] 4) Calculate the average matrix distance mean(DL) and mean(DL′);

[0073] 5) If mean(DL′)>mean(DL), update L using L′; otherwise, calculate the acceptance probability p and generate a random number y between 0 and 1. If p>y, update L using L′; otherwise, keep L unchanged.

[0074] 6) Update the global optimal REL = argmax L (mean(DL));

[0075] 7) Repeat 2)3)4)5)6)s times;

[0076] 8) Update temperature T = c × T;

[0077] 9) Repeat 7) and 8) until T <T f .

[0078] The specific method of step five is:

[0079] Accuracy is defined as the appropriate difficulty level of the recommended exercises for the students. The smaller the distance between the difficulty level of the exercises in the recommended list and the students' expected difficulty level, the more appropriate the difficulty level and the more accurate the recommendation:

[0080]

[0081] Among them L MRepresents the recommended exercise list REL, M represents the number of exercises included in the list, represents the difficulty of the exercise for the student, δ represents the expected difficulty, Accuracy(L M ) value is between 0 and 1, the larger the value, the higher the recommendation accuracy;

[0082] Novelty is defined as the recommended exercises containing knowledge points that students have not answered correctly or cannot answer correctly:

[0083]

[0084] where e(K)=[e(k1),e(k2),…,e(k n )] represents the knowledge point vector of the exercise, e(k i ) = 1 means that the exercise contains the corresponding knowledge point, Pass(K) represents the probability of a student answering the knowledge point correctly in the history question answering sequence, and the probability of a correct answer for each knowledge point is equal to the number of correct answers divided by the total number of answers;

[0085]

[0086] Diversity is defined as the large differences between exercises in the recommendation list, and is represented by the average similarity between exercises in the recommendation list:

[0087]

[0088] where different(e i (K),e j (K)) represents the difference between exercises, and the cosine similarity function is used to calculate the similarity between exercises to measure the difference:

[0089] different(e i (K),e j (K))=1-cossim(e i (K),e j (K))

[0090] Furthermore, the recommendation system based on the above-mentioned diversified exercise recommendation method combining difficulty and weak knowledge points includes:

[0091] Prediction module, used to predict the probability of students correctly answering exercises and the degree of their mastery of knowledge points;

[0092] The filtering module is used to filter out a subset of exercises that are moderately difficult and contain students' weak knowledge points;

[0093] The filtering module is used to filter out a list of recommended exercises containing rich knowledge points;

[0094] A device for recommending diversified exercises that combine difficulty and weak knowledge points, including:

[0095] memory for storing computer programs;

[0096] A program processor is used to store a computer program and implement the diversified exercise recommendation based on the difficulty and weak knowledge points as described in any one of steps 1 to 5 when executing the computer program.

[0097] The beneficial effects of the present invention are as follows:

[0098] 1. The present invention simultaneously models the exercises and knowledge points through step 2, and uses the DKT model to obtain the probability of students correctly answering the exercises and the degree of mastery of the knowledge points.

[0099] 2. The present invention designs an exercise filtering method through step three, selecting exercises that meet students' difficulty expectations and include their weak knowledge points from the entire exercise set, ensuring the accuracy and novelty of the recommendations.

[0100] 3. The present invention designs a diversified secondary screening method through the simulated annealing algorithm in step 4, treating exercises as points in space, and screening out a recommended list of exercises containing rich knowledge points from the candidate subset to solve the maximum spatial distance combinatorial optimization problem, thereby ensuring the diversity of recommendations; the diversity of recommendation results helps to improve learning interest.

[0101] 4. The recommendation evaluation method designed in step five of the present invention ensures the accuracy, novelty, and diversity of exercise recommendations, assisting students in finding learning resources that match their personalized parameters during the learning process, thereby making up for students' weak knowledge points and improving learning efficiency, further improving the effectiveness of the recommendation method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 This is a diagram of the overall architecture of the algorithm provided by an embodiment of the present invention.

[0103] Figure 2 This is a structural diagram of the DKT model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0104] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0105] Step 1. First, encode the student history question answer sequence to obtain the model input data in a standard format.

[0106] The student's historical answer sequence algorithm is used as input data and divided into the exercise level or knowledge point level to model the student status in the knowledge tracking field:

[0107] X={x1,x2,x3,…,x t} represents the student's history answer sequence, where e t Exercises that indicate answers, Indicates the answer to the exercise, or where k t Indicates the knowledge points of the answer, Indicates the answer status of knowledge points, 1 indicates a correct answer, and 0 indicates an incorrect answer;

[0108] Based on this, the student history answer sequence is expressed as <exercise, result> or <knowledge point, result>.

[0109] Step 2. Model the input exercise data in the prediction layer. Use the e-DKT model to model the exercises to obtain the probability of students answering the exercises correctly. Use the k-DKT model to model the knowledge points to obtain the students' mastery of the knowledge points.

[0110] The prediction layer is divided into two sub-modules: the exercise-level knowledge tracking model and the knowledge point-level knowledge tracking model;

[0111] The exercises are modeled through the exercise-level knowledge tracking model, and its input corresponds to The output is the probability of the student answering each exercise correctly;

[0112] The knowledge points are modeled through the knowledge point hierarchical knowledge tracking model, and its input corresponds to The output is the students’ mastery of each knowledge point.

[0113] Both submodules of the prediction layer use the DKT model based on the LSTM network structure as the knowledge tracking model. The specific method is as follows:

[0114] The DKT (Deep Knowledge Tracing) model first introduced deep learning into the field of knowledge tracing. The DKT model inputs the student’s history answer sequence X = {x1, x2, x3, …, x t}, x t After feature cross encoding, it is transformed into the input of the model:

[0115]

[0116] e t is the one-hot encoding of the exercise answered at time t, 0 is the same as e t Zero vectors of equal length;

[0117] Input vector x t After the RNN hidden layer h t Feature extraction, get the output vector y t ,y t Represents the probability of answering the exercise correctly:

[0118] h t =tanh(W hx x t +W hh h t-1 +b h )

[0119] y t =σ(W yh h t +b y )

[0120] The loss function of the DKT model is as follows, where l is the binary cross entropy, δ(e t+1 ) is Exercise e t+1 One-hot encoding:

[0121]

[0122] The prediction layer uses the question-level DKT model (e-DKT) to obtain the probability of students’ correct answers to exercises. Similarly, the knowledge point-level DKT model (k-DKT) is used to obtain the students’ mastery of knowledge points.

[0123] Step 3. Based on the probability of correct answers and the degree of knowledge mastery, calculate the individual difficulty of the exercises and the students' weak knowledge points. Use the filtering module to select exercises with moderate difficulty and containing weak knowledge points from the exercise set to construct a candidate subset to ensure the accuracy and novelty of the recommendations.

[0124] ES (Exercise Set) represents the entire exercise set, CS (Candidate Subset) represents the candidate exercise subset selected by the filtering module, P (K) represents the probability vector of students’ knowledge points mastered by the k-DKT model, and p e(K) represents the probability of correct answers to student exercises obtained by the e-DKT model; where K represents the set of knowledge points, and the knowledge points contained in exercise e are represented by e(K):

[0125] e(K)=[e(k1),e(k2),…,e(k n )]

[0126] n represents the number of knowledge point set K, e(k i )=1 means the exercise contains the corresponding knowledge point i, e(k i )=0 means not included;

[0127] Define the student's weak knowledge point vector W(K) = 1-P(K), the individual difficulty of the exercise de (K) =pe (K) , we hope that the vector similarity between the exercises and the students’ weak knowledge points is large, and at the same time we hope that the distance between the expected difficulty δ and the actual difficulty is small, so the filtering objectives are as follows:

[0128]

[0129] The high similarity between the vectors of exercises and students’ weak knowledge points is equivalent to the low similarity between the vectors of exercises and students’ mastered knowledge points, which is as follows:

[0130]

[0131] The similarity calculation uses cosine similarity and the distance metric uses subtraction. Finally, we can calculate the e of each exercise. j (K)'s score

[0132]

[0133] Then, based on the score of each exercise in the exercise set ES Sort and select the N exercises with the smallest values ​​to construct the candidate exercise set ES:

[0134]

[0135] in Indicates the Nth score after ascending sorting.

[0136] Filtering algorithm, the specific process is as follows:

[0137] (1) Select Exercise e j (K), calculate the similarity cossim(e j (K), P(K)) and distance Get the final score

[0138] (2) Repeat step (1) to calculate the scores of all exercises in the exercise set ES;

[0139] (3) Sort the exercises in ascending order of scores;

[0140] (4) Select the top N exercises with the lowest scores to construct the subsequent subset CS.

[0141] Step 4. Use the simulated annealing algorithm to select the exercises with the largest spatial distance from the candidate subset and generate a recommended exercise list REL for output.

[0142] Each exercise contains one or more knowledge points. Exercises are considered as points in a high-dimensional space. The spatial dimension is the number of knowledge points in the dataset, n. Based on this, the task of generating a diverse exercise recommendation list can be transformed into solving M points with large distances in the high-dimensional space. M represents the number of exercises in the REL. The solution is solved using the simulated annealing algorithm (SA). The specific steps are as follows:

[0143] a. First, randomly select M exercises from the candidate exercise subset CS, initialize the current exercise list L, and calculate the exercise distance matrix DL of the current exercise list. DL is an M×M dimensional matrix, where each value represents the Euclidean distance between two exercises in the exercise list:

[0144]

[0145] DL (i,j) represents the value of row i and column j in the DL matrix, e i (K), e j (K) represents the knowledge point vector of the corresponding exercise in the exercise list L. It can be seen that DL (i,j) The larger the value of , the greater the distance between the two exercises in the high-dimensional space. At the same time, the average matrix distance is defined to measure the spatial distance of the entire exercise list L:

[0146]

[0147] The larger the mean(DL), the larger the spatial distance of the exercise list L, which means that the knowledge points contained are more diverse;

[0148] b. Randomly select an exercise from the candidate exercise subset CS to replace one of the current exercise list L, obtaining a new exercise list L′. At the same time, calculate the distance matrix DL′ and the average matrix distance mean(DL′) of the new exercise list;

[0149] c. Set the rules for accepting the new exercise list L′ according to the Metropolis acceptance criterion. When mean(DL′)>mean(DL), that is, the new exercise list L′ has a high degree of diversity, use the new exercise list L′ to update the current exercise list L. Otherwise, update the current exercise list L with the new exercise list L′ with probability p:

[0150]

[0151] Where T represents temperature, κ B represents the Boltzmann constant, κ B=1.380649×10 -23 J / K, the specific acceptance method is as follows:

[0152]

[0153] Where r=random(0,1) represents a random number between 0 and 1;

[0154] d. It is divided into two layers of iteration, the outer layer iteration is controlled by the temperature T. Each iteration will cool down with a specific attenuation coefficient c. The initial temperature is T0. After multiple iterations, when the temperature is less than the termination temperature T f If the number of iterations is s, the iteration stops, otherwise, the steps b and c are continued. The inner iterations are set to a fixed number s, so that multiple searches are performed at the same temperature T.

[0155] At the same time, in each iteration, the exercise list DL that maximizes the average matrix distance mean(DL) is solved, as shown below. When the entire iteration is completed, REL is output as the recommendation system list.

[0156] REL←argmax L (mean(DL))

[0157] The time complexity of the simulated annealing algorithm depends on the number of iterations of the inner and outer loops. The algorithm tries to find the global optimal solution in a limited search. The specific algorithm flow is as follows:

[0158] 1) Randomly select M exercises, initialize the exercise list L, initialize the starting temperature T = T0, and the ending temperature T f , attenuation coefficient c, number of internal cycles s;

[0159] 2) Randomly replace the exercises in the exercise list L to generate a new exercise list L′;

[0160] 3) Calculate the distance matrices DL and DL′ of the problem;

[0161] 4) Calculate the average matrix distance mean(DL) and mean(DL′);

[0162] 5) If mean(DL′)>mean(DL), update L using L′; otherwise, calculate the acceptance probability p and generate a random number y between 0 and 1. If p>y, update L using L′; otherwise, keep L unchanged.

[0163] 6) Update the global optimal REL = argmax L (mean(DL));

[0164] 7) Repeat 2)3)4)5)6)s times;

[0165] 8) Update temperature T = c × T;

[0166] 9) Repeat 7) and 8) until T <T f .

[0167] Step 5. Evaluate the output recommended exercise list REL based on accuracy, novelty, and diversity indicators:

[0168] Accuracy

[0169] Accuracy requires that the recommended exercises are of appropriate difficulty for students. This can be defined as the distance between the difficulty of the exercises in the recommended list and the student's expected difficulty level. The smaller the distance, the more appropriate the difficulty level and the more accurate the recommendation:

[0170]

[0171] Among them L M Represents the recommended exercise list REL, M represents the number of exercises included in the list, represents the difficulty of the exercise for the student, δ represents the expected difficulty, Accuracy(L M ) value is between 0 and 1, and the larger the value, the higher the recommendation accuracy.

[0172] Novelty

[0173] Novelty requires that the recommended exercises contain knowledge points that students have not answered correctly or cannot answer correctly:

[0174]

[0175] where e(K)=[e(k1),e(k2),…,e(k n )] represents the knowledge point vector of the exercise, e(k i )=1 means that the exercise contains the corresponding knowledge point, Pass(K) represents the probability of correctly answering the knowledge point in a student's history question answering sequence, and the probability of correctly answering each knowledge point is equal to the number of correct answers divided by the total number of answers.

[0176]

[0177] Diversity

[0178] Diversity requires that the recommended exercise list itself contains more diverse knowledge points, that is, the exercises in the recommended list are very different. This can be represented by the average similarity between the exercises in the recommended list:

[0179]

[0180] where different(e i(K),e j (K)) represents the difference between exercises, and the cosine similarity function is used to calculate the similarity between exercises to measure the difference:

[0181] different(e i (K),e j (K))=1-cossim(e i (K),e j (K))

[0182] To verify the effectiveness of the present invention, comparative experiments were conducted on multiple data sets:

[0183] Dataset

[0184] ASSIST12: This dataset is derived from student learning data collected by the ASSISTments online education platform. Exercises without skill labels, exercises with fewer than three answers, and users with fewer than 15 interaction records were removed. The final preprocessed dataset contains 26,875 students, 42,088 questions, 265 knowledge points, and 1,451,899 records.

[0185] EdNet: This dataset is the largest publicly available dataset in the education field to date. It randomly selected 1,764,267 records from 5,000 students, containing 13,169 questions and 188 knowledge points.

[0186] CodeForces: This dataset is collected from the CodeForces website and contains 8087 exercises, 36 knowledge points, 4765 users, and 3758223 submission records.

[0187] Comparison Model

[0188] SB-CF: A student-based collaborative filtering algorithm leverages similarities between students to make recommendations. It first constructs a student-exercise interaction matrix based on students' historical answer sequence data. It then calculates similarities between students and selects the top-ranked similar students. It then predicts the student's performance based on the performance of similar students. Finally, it selects exercises of appropriate difficulty for recommendation.

[0189] MF-CF: A collaborative filtering algorithm based on matrix decomposition can alleviate the problem of data sparsity and learn latent vectors for students and exercises. The latent vectors can be viewed as projections of students and exercises in the same feature space. Using the latent vectors, students can be associated with exercises and their performance can be predicted.

[0190] DKT-CF: An algorithm based on DKT and CF uses the DKT model to obtain the probability vector of students' knowledge points. At the same time, based on this vector, a collaborative filtering algorithm is used to find students with similar knowledge status, and the knowledge status of these students is used to update the own knowledge status, so as to achieve the effect of considering both the individual knowledge status of students and the commonalities of the student group.

[0191] SB-CF and MF-CF use classic collaborative algorithms and do not model students' knowledge status. They cannot obtain the individual difficulty of exercises, resulting in the inability to calculate accuracy indicators. Therefore, the static difficulty of the exercises is used as a substitute in the experiment, that is, the probability of correct answer to the exercise in the dataset.

[0192] Experimental results

[0193] accuracy:

[0194]

[0195] Novelty:

[0196]

[0197]

[0198] Diversity:

[0199]

[0200] Experimental results show that the mean and standard deviation of accuracy, novelty and diversity indicators of this method are optimal in the three datasets of ASSIST12, EdNet and CodeForces, and the improvement is more obvious.

[0201] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with this technical field within the technical scope disclosed by the present invention and within the spirit and principles of the present invention should be covered by the scope of protection of the present invention.

Claims

1. A method for recommending diversified exercises that combines difficulty and weak knowledge points, characterized by: The specific steps include: Step 1: Encode the student history question answer sequence at the input layer to obtain model input data in a standard format; Step 2: Model the input exercise data in the prediction layer to obtain the probability of students answering the exercises correctly and the degree of students' mastery of knowledge points; Step 3: Based on the correct answer probability and knowledge mastery level of the exercises obtained by the prediction layer, the filtering module is used to select candidate subsets of appropriate difficulty and containing students' weak knowledge points from the entire exercise set according to the individual difficulty of the exercises and the students' weak knowledge points; Step 4: Use the simulated annealing algorithm to select the exercises with the largest spatial distance from the candidate subset to generate a recommended exercise list; and output the generated recommended exercise list REL; Each exercise contains one or more knowledge points. Exercises are considered as points in a high-dimensional space. The spatial dimension is the number of knowledge points in the dataset, n. Based on this, the task of generating a diverse exercise recommendation list can be transformed into solving M points with large distances in the high-dimensional space. M represents the number of exercises in the REL. The solution is solved using the simulated annealing algorithm (SA). The specific steps are as follows: a. First, randomly select M exercises from the candidate exercise subset CS, initialize the current exercise list L, and calculate the exercise distance matrix DL of the current exercise list. DL is an M×M dimensional matrix, where each value represents the Euclidean distance between two exercises in the exercise list: DL (i,j) represents the value of the i-th row and j-th column in the DL matrix, e i (K), e j (K) represents the knowledge point vector of the corresponding exercise in the exercise list L. It can be seen that DL (i,j) The larger the value of , the greater the distance between the two exercises in the high-dimensional space. At the same time, the average matrix distance is defined to measure the spatial distance of the entire exercise list L: The larger the mean(DL), the larger the spatial distance of the exercise list L, which means that the knowledge points contained are more diverse; b. Randomly select an exercise from the candidate exercise subset CS to replace one in the current exercise list L, obtaining a new exercise list L', and calculate the distance matrix DL' and the average matrix distance mean(DL') of the new exercise list; c. Set the rules for accepting the new exercise list L' according to the Metropolis acceptance criterion. When mean(DL')>mean(DL), that is, the new exercise list L' is highly diverse, use the new exercise list L' to update the current exercise list L. Otherwise, update the current exercise list L with the new exercise list L' with probability p: Where T represents temperature, κ B represents the Boltzmann constant, κ B =1.380649×10 -23 J / K, the specific acceptance method is as follows: Where r=random(0,1) represents a random number between 0 and 1; d. It is divided into two layers of iteration, the outer layer iteration is controlled by the temperature T. Each iteration will cool down with a specific attenuation coefficient c. The initial temperature is T0. After multiple iterations, when the temperature is less than the termination temperature T f If the number of iterations is s, the iteration stops, otherwise, the steps b and c are continued. The inner iterations are set to a fixed number s, so that multiple searches are performed at the same temperature T. At the same time, in each iteration, we solve the list of exercises DL that maximizes the average matrix distance mean(DL), as shown below. When the entire iteration is completed, the list REL is output as the recommendation system list. REL←argmax L (mean(DL)); Step 5: Evaluate the output recommended exercise list REL based on accuracy, novelty, and diversity indicators.

2. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 1, characterized in that: The specific process of step one is: The student's historical answer sequence algorithm is used as input data and divided into the exercise level or knowledge point level to model the student status in the knowledge tracking field: X={x1,x2,x3,…,x t } represents the student's history answer sequence, where e t Exercises that indicate answers, Indicates the answer to the exercise, or where k t Indicates the knowledge points of the answer, Indicates the answer status of knowledge points, 1 indicates a correct answer, and 0 indicates an incorrect answer; Based on this, the student history answer sequence is expressed as <exercise, result> or <knowledge point, result>.

3. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 1, characterized in that: The specific process of step 2 is: The prediction layer is divided into two sub-modules: the exercise-level knowledge tracking model and the knowledge point-level knowledge tracking model; The exercises are modeled through the exercise-level knowledge tracking model, and its input corresponds to The output is the probability of the student answering each exercise correctly; The knowledge points are modeled through the knowledge point hierarchical knowledge tracking model, and its input corresponds to The output is the students’ mastery of each knowledge point; The exercise-level knowledge tracking model and the knowledge point-level knowledge tracking model both use the DKT model based on the LSTM network structure as the knowledge tracking model. The specific methods are as follows: The DKT (Deep Knowledge Tracing) model first introduced deep learning into the field of knowledge tracing. The DKT model inputs the student’s history answer sequence X = {x1, x2, x3, …, x t }, x t After feature cross encoding, it is transformed into the input of the model: e t is the one-hot encoding of the exercise answered at time t, 0 is the same as e t Zero vectors of equal length; Input vector x t After the RNN hidden layer h t Feature extraction, get the output vector y t ,y t Represents the probability of answering the exercise correctly: h t =tanh(W hx x t +W hh h t-1 +b h ) y t =σ(W yh h t +b y ) The loss function of the DKT model is as follows, where l is the binary cross entropy, δ(e t+1 ) is Exercise e t+1 One-hot encoding: The prediction layer uses the question-level DKT model (e-DKT) to obtain the probability of students’ correct answers to exercises. Similarly, the knowledge point-level DKT model (k-DKT) is used to obtain the students’ mastery of knowledge points.

4. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 1, characterized in that: The specific process of step three is: ES (Exercise Set) represents the entire exercise set, CS (Candidate Subset) represents the candidate exercise subset selected by the filtering module, P (K) represents the probability vector of students’ knowledge points mastered by the k-DKT model, and p e(K) represents the probability of correct answers to student exercises obtained by the e-DKT model; where K represents the set of knowledge points, and the knowledge points contained in exercise e are represented by e(K): e(K)=[e(k1),e(k2),…,e(k n )] n represents the number of knowledge point set K, e(k i )=1 means the exercise contains the corresponding knowledge point i, e(k i )=0 means not included; Define the student's weak knowledge point vector W(K) = 1-P(K), and the individual difficulty of the exercise d e(K) =p e(K) , we hope that the vector similarity between the exercises and the students’ weak knowledge points is large, and at the same time we hope that the distance between the expected difficulty δ and the actual difficulty is small, so the filtering objectives are as follows: The high similarity between the vectors of exercises and students’ weak knowledge points is equivalent to the low similarity between the vectors of exercises and students’ mastered knowledge points, which is as follows: The similarity calculation uses cosine similarity and the distance metric uses subtraction. Finally, we can calculate the e of each exercise. j (K)'s score Then, based on the score of each exercise in the exercise set ES Sort and select the N exercises with the smallest values ​​to construct the candidate exercise set ES: in Indicates the Nth score after ascending sorting.

5. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 4, characterized in that: The filtering algorithm used in step 3 has the following specific process: (1) Select Exercise e j (K), calculate the similarity cossim(e j (K), P(K)) and distance Get the final score (2) Repeat step (1) to calculate the scores of all exercises in the exercise set ES; (3) Sort the exercises in ascending order of scores; (4) Select the top N exercises with the lowest scores to construct the subsequent subset CS.

6. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 1, characterized in that: The time complexity of the simulated annealing algorithm described in step 4 depends on the number of iterations of the inner and outer loops. Try to find the global optimal solution in a limited search. The specific algorithm flow is as follows: 1) Randomly select M exercises, initialize the exercise list L, initialize the starting temperature T = T0, and the ending temperature T f , attenuation coefficient c, number of internal cycles s; 2) Randomly replace the exercises in the exercise list L to generate a new exercise list L'; 3) Calculate the distance matrices DL and DL' of the exercise; 4) Calculate the average matrix distance mean(DL) and mean(DL'); 5) If mean(DL')>mean(DL), update L using L'; otherwise, calculate the acceptance probability p and generate a random number y between 0 and 1. If p>y, update L using L'; otherwise, keep L unchanged. 6) Update the global optimal REL = argmax L (mean(DL)); 7) Repeat 2)3)4)5)6)s times; 8) Update temperature T = c × T; 9) Repeat 7) and 8) until T <T f .

7. The method for recommending diversified exercises that combines difficulty and weak knowledge points according to claim 1, characterized in that: The specific method of step five is: Accuracy is defined as the appropriate difficulty level of the recommended exercises for the students. The smaller the distance between the difficulty level of the exercises in the recommended list and the students' expected difficulty level, the more appropriate the difficulty level and the more accurate the recommendation: Among them L M Represents the recommended exercise list REL, M represents the number of exercises included in the list, represents the difficulty of the exercise for the student, δ represents the expected difficulty, Accuracy(L M ) value is between 0 and 1, the larger the value, the higher the recommendation accuracy; Novelty is defined as the recommended exercises containing knowledge points that students have not answered correctly or cannot answer correctly: where e(K)=[e(k1),e(k2),…,e(k n )] represents the knowledge point vector of the exercise, e(k i ) = 1 means that the exercise contains the corresponding knowledge point, Pass(K) represents the probability of a student answering the knowledge point correctly in the history question answering sequence, and the probability of a correct answer for each knowledge point is equal to the number of correct answers divided by the total number of answers; Diversity is defined as the large differences between exercises in the recommendation list, and is represented by the average similarity between exercises in the recommendation list: where different(e i (K),e j (K)) represents the difference between exercises, and the cosine similarity function is used to calculate the similarity between exercises to measure the difference: different(e i (K),e j (K))=1-cossim(e i (K),e j (K))。 8. A recommendation system based on the method for recommending diversified exercises combining difficulty and weak knowledge points as described in any one of claims 1 to 7, characterized in that: include: Prediction module, used to predict the probability of students correctly answering exercises and the degree of their mastery of knowledge points; The filtering module is used to filter out a subset of exercises that are moderately difficult and contain students' weak knowledge points; The filtering module is used to filter out a recommended list of exercises containing rich knowledge points.

9. A recommendation device based on the method for recommending diversified exercises combining difficulty and weak knowledge points according to any one of claims 1 to 7, characterized in that: include: Memory for storing computer programs; The program processor is used to store computer programs and, when executing the computer programs, recommend diversified exercises based on difficulty and weak knowledge points.

Images