Deep reinforcement learning and large model-based circuit principle course customized exercise recommendation and auxiliary guidance method

Abstract

The application discloses a circuit principle course customized exercise recommendation and auxiliary guidance method based on deep reinforcement learning and a large model, and the method comprises the following steps: a Transformer model is established to predict the mastery of circuit principle course knowledge by a learner, and a Mamba model is established to predict the knowledge point that the learner should learn next; a reliable exercise difficulty evaluation model is established to calculate the difficulty coefficient of each exercise, wherein a five-point attribution method is used to assign values to and classify different types of circuit principle exercises; a Markov decision process model for circuit principle exercise recommendation is established, a deep reinforcement learning algorithm is used to generate an optimal circuit principle exercise set for recommendation, problem solving guidance prompts for exercises are customized according to the learning situation of the learner, and a large model interface is called to generate problem solving guidance for recommended exercises. The method can effectively customize the problem solving guidance outline of circuit principle exercises according to the learning situation of the learner.

Description

TECHNICAL FIELD

[0001] The present application relates to the customization of auxiliary teaching of circuit principle courses, in particular to a circuit principle course exercise recommendation strategy based on deep reinforcement learning and large models. BACKGROUND

[0002] Circuit principle is one of the core courses in engineering disciplines, but due to its complex content and tedious mathematical derivation, students often face challenges in the learning process. Therefore, mastering circuit principles through exercise practice is crucial for students to master complex concepts and solve practical problems. Existing exercise recommendation models use pre-defined rules or simple logical judgments to recommend exercises. For example, according to the student's historical answering situation (such as accuracy), choose the appropriate difficulty of the question. This method is simple and direct, but has poor flexibility and is difficult to adapt to the learning progress and knowledge structure of individual students. Or based on the collaborative filtering technology widely used in recommendation systems, use the answering data between similar students to recommend exercises. But it relies on a large amount of data, and when the data is sparse or the student group is large, the recommendation effect will decrease significantly. Some systems use shallow machine learning algorithms such as decision trees, random forests or support vector machines to predict the most suitable exercise type based on the student's answering performance. Although these algorithms improve the accuracy of the recommendation to some extent, they often rely on manual feature engineering and are difficult to capture the complex learning behavior and habits of students.

[0003] Due to the uniqueness of the circuit principle course and the complexity of the principle mastery evaluation, the existing recommendation model cannot complete the exercise recommendation task well, the main problems include: the existing exercise recommendation system usually relies on pre-set rules or simple algorithms, and cannot effectively adapt to individual differences and dynamic learning needs of students; the difficulty division of the circuit principle knowledge involved in the exercise is not reasonable; for the advanced concepts and knowledge points involved in the circuit principle, the existing system has limited ability to analyze and quantify this part; for the advanced concepts and multi-step derivation involved in the circuit principle, the performance of the existing system in exercise generation and analysis is limited, and the auxiliary teaching effect is not obvious. SUMMARY

[0004] The purpose of the present application is to provide a circuit principle course customization auxiliary teaching method based on deep reinforcement learning and large models, so as to help circuit principle teaching through exercise recommendation to improve teaching and practice effect.

[0005] To achieve the above purpose, the specific scheme of the present application is as follows:

[0006] Step S101, a Transformer model is established to predict the mastery of circuit course learners to circuit course knowledge points;

[0007] The input of the Transformer model is a learner id, a sequence of circuit principle exercises done, and a binary vector b of the answer to the question t ∈ [0, 1], b t The value of b is 0 or 1, where 0 and 1 represent incorrect and correct answers to the circuit exercise, respectively, x = {id, e j (k), b t , P t}, where all k knowledge points contained in the circuit principle course are represented as K = {k1, k2,..., k k}, and the exercises are represented as e j (k) = [e j (k1), e j (k2), e j (k3),..., e j (k k )], j represents the jth exercise in the exercise library, e j (k i ) is 0 or 1, where 0 represents that the question does not contain the i-th knowledge point concept, and 1 represents that the question contains it, and the position encoding P t is embedded in the input of the Transformer model. The output of the model is a k-dimensional vector O t representing the mastery of each knowledge point in the circuit principle course. The optimization loss function for a single learner is represented as

[0008]

[0009] where t represents the tth position in the sequence, and · represents dot product.

[0010] Step S102, a Mamba model is established to predict the knowledge point that the learner should learn next,

[0011] The Mamba model is used to predict the knowledge point that the learner should learn next, with the input being a set of exercises done by a student within a certain time period E, and the mastered knowledge points K t , and the output being a k-dimensional probability vector representing the occurrence of all knowledge points in the next exercise. The model is trained using a binary cross-entropy loss function; the training loss function is

[0012]

[0013] where t represents the tth position in the sequence, and · represents dot product.

[0014] The teacher gives a weight vector ω(k) = [ω(k1), ω(k2),..., ω(k k)], the final knowledge point coverage prediction P(k) is obtained by using the predicted k-dimensional probability vector and weight vector dot product, and the calculation is as follows:

[0015] P(k i )=p(k i )ω(k i ) (10)

[0016] Wherein, P(k i )∈P(k);

[0017] Step S103, a credible exercise difficulty evaluation model is established to calculate the difficulty coefficient of each exercise,

[0018] Step (3.1): first, use five-point attribution method to assign values to different types of circuit principle exercises and classify them, and get the difficulty and quantity of each type of exercise: in terms of exercise type, the understanding of basic circuit knowledge is divided into single, complex and integrated multiple concept integration, and is assigned values of 0, 0.5 and 1 respectively, the content corresponds to single circuit element or simple circuit analysis, integrated multiple concept integration and open knowledge point examination, the corresponding assignment is 0, 0.5 and 1 respectively; in terms of knowledge content, one knowledge point is assigned 0, two knowledge points are assigned 0.5, and more than two knowledge points are assigned 1; in terms of knowledge type, it is divided into formula type and circuit diagram, and is assigned 0.5 and 1 respectively; in terms of reasoning ability, one-step reasoning to get the answer is assigned 0, two-step reasoning to get the answer is assigned 0.5, and complex reasoning multi-step reasoning is assigned 1; in terms of cognitive level, memorizing circuit knowledge theorem is assigned 0, understanding circuit law and circuit diagram is assigned 0.5, and applying and mastering is assigned 1;

[0019] Step (3.2): according to the situation of the exercise group, the weighted average method is used to calculate the quantitative value of each influencing factor, and the weighted average formula of each factor is as follows:

[0020]

[0021] Wherein, p i (i=1,2,3,4,5) is the difficulty value of the i-th difficulty factor in the five difficulty factors, n ij represents the number of exercises of the j-th (j=1,2,3) level of the i-th difficulty factor in a group of exercises, l ij represents the assignment value of the j-th level of the i-th difficulty factor, and n represents the total number of the group of exercises. After calculating the values of the above five difficulty factors, the difficulty value P in the range of [0, 1] is obtained;

[0022] Step (3.3): calculate the error rate of each exercise in a natural class of a grade, and adjust the initial difficulty coefficient of the exercise according to the error rate;

[0023] Step (3.4): Calculate the final difficulty coefficient d by multiplying P with the initial difficulty coefficient of the question, and the weight given by the teacher or student.

[0024] Step S104, establish a Markov decision process model for circuit principle exercise recommendation,

[0025] Establish a Markov decision process model for circuit principle exercise recommendation, the state includes the current exercise question e t , target T, the current learner's mastery of all k knowledge points contained in the circuit course h t , the next question knowledge point coverage and the difficulty coefficient d of the question, where the target T is represented by the knowledge point vector that the teacher or student needs to master and learn; the action space is to select the candidate exercise set A, and the candidate exercise is obtained by matching the exercise set through the predicted next step of learning knowledge points, and the reward value at each time is the weighted average value of the average mastery of knowledge points p t and difficulty difference | θ-d |, θ is the expected difficulty coefficient, and the weight is given by the student or teacher, where p t is the average value of each element in h t , which is calculated at each time as follows;

[0026] R t = αp t + β| θ-d | (12)

[0027] Where, α and β are self-defined weights, and p t is the average value of each element in h t .

[0028] Step S105, generate the recommended optimal circuit principle exercise set using deep reinforcement learning algorithm,

[0029] Step (5.1): Use a policy network π θ and a value network V φ , use the policy network π θ to select the action a, calculate the reward value through the reward value, and use the value network to estimate the long-term value after taking the action, so as to obtain the target value y t

[0030] y t = R t + γV φ (S t+1 ) (13)

[0031] Where γ represents the discount factor;

[0032] Step (5.2): The policy network optimizes the policy by maximizing the objective function, and the objective function is

[0033]

[0034] where r t (θ) represents the ratio between the old and new policies, S t+1 represents the state at time t+1, S t represents the state at time t, represents the advantage function, which measures the goodness between actions, and ε is a hyperparameter that controls the update range between policies.

[0035] The advantage function in step (5.3) is:

[0036]

[0037] where Q(S t ,a t ) is defined as the long-term cumulative return of the learner after choosing the exercise a t under the current knowledge state S t , which is predicted by the established Transformer model; V(S t ) is calculated by the value network estimation to obtain the knowledge mastery at each action a t .

[0038] Step S106: Customized prompt words are given and personalized problem solving guidance is given through a large model interface,

[0039] Step (6.1): The prompt words contain the learner's mastery of circuit course knowledge points, the learner's answer history record, and all circuit knowledge points are spliced;

[0040] Step (6.2): In the prompt words, the principle for generating the problem solving guidance is explained as "highlighting the weak knowledge points mastered by the learner and covering less knowledge points in the problem solving guidance";

[0041] Step (6.3): Based on the above, the problem solving guidance prompt words of the recommended exercises are given, and the large model interface is called in a zero shot way to generate the problem solving guidance of the recommended exercises.

[0042] Compared with existing exercise recommendation methods and auxiliary guidance methods, the present application has the following advantages:

[0043] 1. The present application can recommend the next learning content for each learner based on the historical answer data and knowledge point mastery of the learner through the Transformer and Mamba models, optimize the learning path, and improve the learning efficiency.

[0044] 2. The application establishes a reliable problem difficulty evaluation model, which comprehensively considers the characteristics of the problems and the error rate within the class, accurately calculates the problem difficulty coefficient, ensures that the recommended problems are moderately difficult, neither too simple nor too difficult, and helps learners effectively improve.

[0045] 3. The application can track the current learning state of the learner in real time through the Markov decision process model, dynamically adjust the recommended problems according to the knowledge point mastery and problem difficulty, and ensure that the learning process progresses steadily.

[0046] 4. The application combines deep reinforcement learning algorithm and large model interface to generate customized problem solving guidance, highlights the weak knowledge points of the learner and covers less knowledge points, helps the learner to more targetedly improve his own weaknesses, and enhances the learning effect. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The application provides a circuit principle course customized problem recommendation and auxiliary guidance method flowchart based on deep reinforcement learning and large model.

[0048] Figure 2 The application provides a Transformer structure diagram.

[0049] Figure 3 The application provides a Mamba structure diagram.

[0050] Figure 4 The application provides a five-point attribution method problem difficulty classification model diagram.

[0051] Figure 5 The application provides a customized problem solving guidance flowchart. DETAILED DESCRIPTION

[0052] In order to make the features and advantages of the application more obvious and easy to understand, the application will be described in detail below in combination with the drawings and specific embodiments.

[0053] Figure 1 The application provides a circuit principle course customized problem recommendation and auxiliary guidance method flowchart based on deep reinforcement learning and large model. As shown in Figure 1 The circuit principle course customized problem recommendation and auxiliary guidance method based on deep reinforcement learning and large model, the method comprises:

[0054] Step S101: Establish a Transformer model to predict the learner's mastery of the circuit course knowledge points;

[0055] Step S102: Establish a Mamba model to predict the knowledge points that the learner should learn next;

[0056] Step S103: Establish a reliable problem difficulty evaluation model to calculate the difficulty coefficient of each problem;

[0057] Step S104: Establish a Markov decision process model for circuit principle problem recommendation;

[0058] Step S105: Use deep reinforcement learning algorithm to solve the recommended optimal circuit problem set;

[0059] Step S106: Customized prompt words and personalized problem solving guidance are given by using large model interface;

[0060] Step S101, specifically includes:

[0061] The Transformer model is shown in Figure 2 The input is the learner id, the sequence of completed circuit principle problems, and the binary vector b t ∈[0,1] of the answer to the question, b t The values of 0 and 1 represent incorrect and correct answers to the circuit problem, respectively, x = {id, e j (k), b t , P t}, where all k knowledge points contained in the circuit principle course are represented as K = {k1, k2,..., k k}, the problem is represented as e j (k) = [e j (k1), e j (k2), e j (k3),..., e j (k k )], j represents the jth problem in the problem library, e j (k i ) takes values of 0 or 1, where 0 indicates that the question does not contain the i-th knowledge point concept, and 1 indicates that the question contains it. Position encoding P t is embedded in the input of the Transformer model. The output of the model is a k-dimensional vector O t , which represents the mastery level of each knowledge point in the circuit principle course. The optimization loss function for a single learner is represented as

[0062]

[0063] Where t represents the tth position in the sequence, and · represents dot product.

[0064] Step S102, specifically includes:

[0065] The Mamba model is used to predict the knowledge point that the learner should learn next. As shown in Figure 3 , the input is a set of exercises E performed by a student within a certain period of time, the mastered knowledge points K t , and the output is a k-dimensional probability vector p(k) representing the occurrence of all knowledge points in the next exercise, and the model is trained using a binary cross-entropy loss function; the training loss function is

[0066]

[0067] where t represents the t-th position in the sequence, and · represents dot product;

[0068] The teacher gives a weight vector ω(k) = [ω(k1), ω(k2), ···, ω(k k )], and the final knowledge point coverage prediction P(k) is obtained by dot product of the predicted k-dimensional probability vector and the weight vector, which is calculated as follows:

[0069] P(k i ) = p(k i ) ω(k i ) (19)

[0070] where P(k i ) ∈ P(k);

[0071] Step S103, specifically comprising:

[0072] Step (3.1): First, use the five-point attribution method to assign values to different types of circuit principle exercises and classify them, as shown in Figure 4 , to get the difficulty and quantity of each type of question: in terms of question type, the understanding of basic circuit knowledge is divided into single, complex and integrated multiple concept integration, respectively assigned as 0, 0.5 and 1; in terms of knowledge content, a knowledge point is assigned 0, two knowledge points are assigned 0.5, and more than two knowledge points are assigned 1; in terms of knowledge type, it is divided into formula type and circuit diagram, respectively assigned as 0.5 and 1; in terms of reasoning ability, one-step reasoning to get the answer is assigned 0, two-step reasoning to get the answer is assigned 0.5, and complex reasoning multi-step reasoning is assigned 1; in terms of cognitive level, memorizing circuit knowledge theorem is assigned 0, understanding circuit law and understanding circuit diagram is assigned 0.5, and applying and mastering is assigned 1;

[0073] Step (3.2): According to the situation of the question group, the quantitative value of each influencing factor is calculated by weighted average method, and the weighted average formula of each factor is as follows:

[0074]

[0075] ​where p i (i=1, 2, 3, 4, 5) is the difficulty value of the i-th difficulty factor in the five difficulty factors, n ij represents the number of questions of the j-th (j=1, 2, 3) level of the i-th difficulty factor in the set of questions, l ij represents the assignment of the j-th level of the i-th difficulty factor, n t is the total number of questions in the set, and the difficulty value P in the range of [0, 1] is obtained after calculating the above five difficulty factor values.

[0076] Step (3.3): Set the initial difficulty coefficient of each question in a natural class of a grade to 0.5, and then calculate the error rate of each question in the class, and adjust the initial difficulty coefficient of the question according to the error rate of each question;

[0077] Step (3.4): Calculate the final difficulty coefficient d by multiplying the average value of P and the initial difficulty coefficient of the question.

[0078] Step S104, specifically comprising:

[0079] A Markov decision process model for recommending circuit principle exercises is established, the state includes the currently practiced exercises e t , the target T, the current learner's mastery of all k knowledge points contained in the circuit course h t , the next question knowledge point coverage and the difficulty coefficient d of the question, wherein the target T is represented by a knowledge point vector given by the teacher or student; the action space is to select the candidate exercise set A, the candidate exercise is obtained by matching the exercise set through the predicted next step of learning knowledge points, and the reward value at each time is the weighted average value of the average mastery of knowledge points p t and the difficulty difference |θ-d|, θ is the expected difficulty coefficient, and the weight is given by the student or teacher, wherein p t is the average value of each element in h t , which is calculated as follows at each time.

[0080] R t = αp t + β|θ-d| (21)

[0081] Where, α and β are self-defined weights, p t is the average value of each element in h t .

[0082] Step S105, specifically comprising:

[0083] Step (5.1): Use a policy network π θ and a value network V φ , use the policy network π θThe action a is selected, a return value is obtained by return value calculation, and a value network is used to estimate the long-term value after taking the action, so as to obtain a target value y t

[0084] y t = R t + γV φ (S t+1 ) (22)

[0085] wherein γ represents a discount factor;

[0086] Step (5.2): The policy network optimizes the policy by maximizing the objective function

[0087]

[0088] wherein r t (θ) represents a ratio between new and old policies, represents an advantage function, which measures the pros and cons between actions, and ε is a hyperparameter that controls the update range between policies.

[0089] Step (5.3): The advantage function is:

[0090]

[0091] wherein Q(S t ,a t ) is defined as the long-term cumulative return of the learner after selecting the exercise a t under the current knowledge state S t , which is predicted by the established Transformer model and weightedly averaged to obtain the knowledge mastery degree of the learner at time t; V(S t ) is calculated by the value network to estimate each knowledge mastery degree, and the knowledge mastery degree at the time when the exercise a t is not completed is weightedly averaged.

[0092] Step S106, specifically comprising:

[0093] Customized prompt words are used to give personalized problem solving guidance through a large model interface, as shown in the flow Figure 5 The prompt words contain the learner's mastery of circuit course knowledge points, the learner's answer history record, and all circuit knowledge points. The principle for generating the problem solving guidance in the prompt words is "highlighting the weak knowledge points mastered by the learner and covering less knowledge points in the problem solving guidance". Based on the above, the problem solving guidance prompt words of the recommended exercises are given, and the large model interface is called in a zero shot manner to generate the problem solving guidance of the recommended exercises.

Claims

1. A circuit principle course customization problem recommendation and auxiliary guidance method based on deep reinforcement learning and large model, characterized in that, Comprise the following steps: Step S101: Establish a Transformer model to predict the learner's mastery of circuit principle course knowledge points, the input data is the splicing vector of embedded position coding, including the learner id, the binary vector of the circuit principle exercise done and the answer of the question, the output of the model is a k-dimensional vector containing the mastery degree of each knowledge point, and the binary cross entropy is used as the loss function of the model training; Step S102: Establish a Mamba model to predict the knowledge points that the learner should learn next, the input is the exercise sequence done by a student in a certain time period and the mastered knowledge points, the output of the model is a k-dimensional probability vector of all knowledge points appearing in the next question, and the teacher gives a weight vector of each knowledge point, and the two vectors are multiplied to get the coverage probability of each knowledge point in the next question; Step S103: Establish a reliable exercise difficulty evaluation model to calculate the difficulty coefficient of each exercise, first use five-point attribution method to assign values to different types of circuit principle exercises and classify them to get the difficulty and quantity of each type of question, calculate the weighted average value of the question in the whole question set, and set the initial difficulty coefficient of each question in a natural class of a grade to 0.5, then adjust the initial difficulty coefficient according to the error rate of the past answerers of the question, and the average value of the two is the final difficulty coefficient; Step S104: Establish a Markov decision process model for circuit principle exercise recommendation, the state includes the current exercise question, the target, the learner's mastery of circuit course knowledge points, the next question knowledge point coverage and the difficulty coefficient of the question, the action space is to select the candidate question, and the reward value is the weighted average value of the difference between the average mastery of knowledge points and the expected difficulty; Step S105: Use deep reinforcement learning algorithm to solve the recommended optimal circuit principle exercise set, use PPO algorithm, define a policy network and a value network, select actions through the policy network, calculate the reward value through the reward value defined in S104, and estimate the long-term value after taking the action through the value network, so as to obtain the target value; Then define the advantage function and design the objective function to train the model; Step S106: Customized prompt words and personalized problem solving guidance are given by using large model interface, the prompt words include the learner's mastery of circuit course knowledge points, the learner's answer history record, and all circuit knowledge points are spliced, and the principle of generating problem solving guidance in the prompt words is "highlighting the weak knowledge points and covering less knowledge points in the problem solving guidance", and the problem solving guidance prompt words of the recommended exercises are given in the above way, and the large model interface is called in zero shot way to generate the problem solving guidance of the recommended exercises.

2. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The Transformer model for predicting the learner's mastery of circuit principle course knowledge points in step S101 comprises: The input of the Transformer model is learner id, the sequence of done circuit principle exercises, and the binary vector b of the answer to the question t ∈ [0, 1], b t The value of b is 0 or 1, where 0 and 1 represent incorrect and correct answers to the circuit exercise respectively, x = {id, e j (k), b t , P t}, where all k knowledge points contained in the circuit principle course are represented as K = {k1, k2,..., k k}, and the exercises are represented as e j (k) = [e j (k1), e j (k2), e j (k3),..., e j (k k )], j represents the jth exercise in the exercise library, e j (k i ) takes the value of 0 or 1, where 0 represents that the question does not contain the i-th knowledge point concept, and 1 represents that the question contains P t is the embedded position encoding P t , and the output of the model is a k-dimensional vector O t containing the mastery level of each knowledge point in the circuit principle course.

3. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The Mamba model for predicting the knowledge points that the learner should learn next in step S102 comprises: Using Mamba model to predict the next knowledge point the learner should learn, input is a sequence of exercise sets E done by a student in a certain period of time, mastered knowledge points K t , output is k-dimensional probability vector representing the occurrence of all knowledge points in the next exercise, using binary cross-entropy loss function to train the model; The teacher gives the weight vector ω(k) = [ω(k1), ω(k2), ···, ω(k k )] of the circuit principle knowledge, and the final knowledge point coverage prediction P(k) is obtained by dot multiplication of the predicted k-dimensional probability vector and the weight vector, which is calculated as follows: P(k i ) = p(k i ) ω(k i ) (1) P(k) = {P(k i )∈P(k).

4. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The circuit principle exercise difficulty classification model based on five-point attribution method in step S103 comprises: Based on the five-point attribution method, a circuit principle question difficulty classification model is constructed, and the difficulty factors, levels, explanations and assignments are classified. It involves multiple knowledge points, understanding, application and practice. The model is summarized as follows: in terms of question type, the understanding of basic circuit knowledge is divided into single, composite and integrated multiple concepts, and assigned values of 0, 0.5 and 1 respectively; in terms of knowledge content, one knowledge point is assigned a value of 0, two knowledge points are assigned a value of 0.5, and more than two knowledge points are assigned a value of 1; in terms of knowledge type, it is divided into formula type and circuit diagram, and assigned values of 0.5 and 1 respectively; in terms of reasoning ability, one step reasoning to get the answer is assigned a value of 0, two step reasoning to get the answer is assigned a value of 0.5, and complex reasoning is assigned a value of 1; in terms of cognitive level, memorizing circuit knowledge theorem is assigned a value of 0, understanding circuit law and circuit diagram is assigned a value of 0.5, and applying and mastering is assigned a value of 1.

5. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The method for calculating the trusted difficulty coefficient described in step S103 specifically includes: According to the five-point attribution method, different types of circuit principle exercises are assigned and classified to obtain the difficulty and quantity of each type of question. Then, the weighted average method is used to calculate the quantitative value of each influencing factor. The weighted average formula of each factor is as follows: where p i (i = 1, 2, 3, 4, 5) is the difficulty value of the ith difficulty factor in the five difficulty factors, n ij represents the number of questions of the jth (j = 1, 2, 3) level of the ith difficulty factor in the set of questions, l ij represents the assignment of the jth level of the ith difficulty factor, n that is, the total number of questions in the set, and the difficulty value P in the range of [0, 1] is obtained after the calculation of the above five difficulty factor values; the error rate of each question is counted in a natural class of a grade, the weighted average value is calculated by multiplying P and the error rate of the question to obtain the final difficulty coefficient d, and the weight is given by the teacher or the student.

6. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The Markov decision process model for establishing circuit principle exercise recommendation described in step S104 specifically includes: A Markov Decision Process model is established for recommending circuit principle exercises, the state includes the current practiced exercise e t , target T, current learner's mastery of all k knowledge points contained in the circuit course O t , the next question knowledge point coverage and the difficulty coefficient d of the exercise, where the target T is represented by the knowledge point vector given by the teacher or student that needs to be mastered and learned; the action space is to select the candidate exercise set A, the candidate exercise is obtained from the exercise set matching through the predicted next step of the knowledge point to be learned, and the reward value at each time is the weighted average value of the current time knowledge point average mastery p t and difficulty difference | θ-d |, θ is the expected difficulty coefficient, and the weight is given by the student or teacher.

7. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The calculation of the reward value of the Markov decision process model for circuit principle exercise recommendation described in step S104 specifically includes: The return value is the average mastery of the knowledge point p t The weighted average of the difficulty difference |θ-d|, θ is the expected difficulty coefficient, and the return value at each time is calculated as follows: R t = αp t + β|θ - d| (3) where a and β are self-defined weights, the weights are given by students or teachers, p t is O t the average of each element in the middle.

8. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The deep reinforcement learning algorithm is used to solve the recommended optimal circuit exercise set described in step S105: using a policy network π θ and a value network V φ , by selecting an action a through the policy network π θ , by computing a return value for the current recommended exercise, and by using the value network to estimate the long-term value of selecting the current recommended exercise, to obtain a target value y t y t = R t + γV φ (S t+1 ) (4) where R t represents the return value at time t, γ represents the discount factor, S t+1 represents the state at time t+1; The policy network optimizes the policy by maximizing the objective function, and the objective function is where r t (θ) represents the ratio between the new and old policy, represents the advantage function, measuring the goodness of actions, and ε is a hyperparameter controlling the range of updates between policies.

9. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The advantage function described in step S105 is: The advantage function is: where Q(S t ,a t ) is defined as the long-term cumulative return after the learner chooses the exercise a t under the current knowledge state S t , which is predicted by the established Transformer model; V(S t ) is calculated by the value network estimation of the current learner's mastery of each knowledge point, and the mastery of the knowledge points before the unfinished action a t is obtained by weighted average.

10. The circuit principle course customization exercise recommendation and auxiliary guidance method based on deep reinforcement learning and large model according to claim 1, characterized in that The problem solving guidance outline for recommending circuit principle exercises based on large model prompt words described in step S106 specifically includes: The prompt words include the learner's mastery of circuit course knowledge points, the learner's answer history record, and the splicing of all circuit knowledge points. The principle for generating the problem solving guidance in the prompt words is "highlighting the weak knowledge points and covering less knowledge points mastered by the learner in the problem solving guidance". Based on the above, the problem solving guidance prompt words for recommending exercises are given, and the large model interface is called in zero shot mode to generate the problem solving guidance for recommending exercises.

Images