Learning recommendation method and device based on artificial intelligence and knowledge tracking and medium

Abstract

The invention discloses a learning recommendation method and device based on artificial intelligence and knowledge tracking and a medium, and can be applied to the technical field of smart learning. After the to-be-selected exercise knowledge set is constructed according to the exercise knowledge document and the reinforcement learning framework is constructed, novelty retrieval is performed in the to-be-selected exercise knowledge set through the reinforcement learning framework to generate the cue word template, and the cue word template is input into the large language model to generate the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises. Predicting a first knowledge mastering level of the student on the knowledge points, and then inputting the learning state of the current student, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the exercise difficulty into a pre-trained difficulty matching knowledge tracking model to obtain a second knowledge mastering level of the current student; and the recommendation question set is updated by combining the first knowledge mastering level, so that learning recommendation can be performed by effectively combining artificial intelligence feedback and knowledge tracking characteristics, and the learning recommendation accuracy is further improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent learning technology, and in particular to a learning recommendation method, device and medium based on artificial intelligence and knowledge tracking. Background Art

[0002] Among the related technologies, deep knowledge tracking has a strong perception ability to perceive the current learning status of students, but lacks certain decision-making ability; while reinforcement learning has decision-making ability, but lacks the perception ability of learning status. By combining the perception ability of deep knowledge tracking with the decision-making ability of reinforcement learning, deep knowledge tracking perceives the learning status of students, and reinforcement learning makes decisions based on the perceived learning status to obtain the best learning efficiency and effect goals, which can obtain the learning path recommendation effect with the best performance. At present, the technical development in the field of reinforcement learning shows a trend: reinforcement learning based on artificial intelligence feedback and reinforcement learning based on knowledge tracking feedback. These two intelligent learning methods have developed independently and are relatively mature. It can be seen that although artificial intelligence feedback and knowledge tracking each show significant advantages in improving model performance, there are relatively few reinforcement learning technologies that deeply integrate artificial intelligence and knowledge tracking, resulting in the inability of existing methods to use the characteristics of artificial intelligence and knowledge tracking to further improve the accuracy of learning recommendations.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose a learning recommendation method, device and medium based on artificial intelligence and knowledge tracking, which can effectively improve the accuracy of learning recommendations.

[0005] To achieve the above objectives, one aspect of an embodiment of the present application proposes a learning recommendation method based on artificial intelligence and knowledge tracing, the method comprising the following steps:

[0006] Construct a knowledge set of exercises to be selected according to the exercise knowledge document;

[0007] Build a reinforcement learning framework;

[0008] Performing a novel search in the candidate exercise knowledge set through the reinforcement learning framework to generate a prompt word template;

[0009] Input the prompt word template into the large language model, generate recommended exercises, the knowledge points and exercise difficulty of the recommended exercises, and predict the student's first knowledge mastery level of the knowledge points;

[0010] Inputting the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student;

[0011] The recommended exercise set is updated according to the first knowledge mastering level and the second knowledge mastering level.

[0012] In some embodiments, the step of constructing a set of to-be-selected exercise knowledge based on the exercise knowledge document includes:

[0013] Collect exercise knowledge documents;

[0014] Splitting the exercise knowledge document into a plurality of text blocks of preset sizes;

[0015] Converting the plurality of text blocks into question vectors through an embedding model;

[0016] The exercise vectors are stored in a vector database as the knowledge set of exercises to be selected.

[0017] In some embodiments, the reinforcement learning framework includes an evaluation network and a policy network; and constructing the reinforcement learning framework includes:

[0018] Initializing a first learning rate and a first network parameter of the evaluation network, and initializing a second learning rate and a second network parameter of the policy network;

[0019] The following steps are executed repeatedly until the number of cycles reaches the preset number:

[0020] The agent interacts with the environment and collects a first sequence, the first sequence including a state, an action, and a reward;

[0021] Calculate a first loss function of the evaluation network and a second loss function of the policy network according to the first sequence;

[0022] The first network parameters are updated according to the first loss function, and the second network parameters are updated according to the second loss function.

[0023] In some embodiments, the step of performing a novel search in the candidate exercise knowledge set through the reinforcement learning framework to generate a prompt word template includes:

[0024] Obtaining an exercise extraction request of an agent in the reinforcement learning framework;

[0025] Converting the exercise extraction request into a query vector;

[0026] Searching the vector database for knowledge text or historical conversation record information similar to the query vector;

[0027] The prompt word template is constructed according to the knowledge text or historical dialogue record information corresponding to the current step exercise extraction request and the previous step exercise extraction request.

[0028] In some embodiments, the step of inputting the prompt word template into a large language model, generating recommended exercises, knowledge points and exercise difficulty of the recommended exercises, and predicting the student's first knowledge mastery level of the knowledge points includes:

[0029] Inputting the prompt word template into a large language model to perform prompt word parsing to obtain the recommended exercises;

[0030] Analyzing key words, phrases and sentence structures in the recommended exercises;

[0031] Determine a first knowledge point according to the keyword, the phrase and the sentence structure;

[0032] Matching the first knowledge point with the knowledge points in the knowledge set of the to-be-selected exercises to determine the knowledge point corresponding to the recommended exercises;

[0033] Obtaining historical answer record information of each exercise in the knowledge set of exercises to be selected;

[0034] Determining the difficulty of the recommended exercises according to the historical answer record information;

[0035] Input the exercise data of the current time step and the historical answer record information of the exercise corresponding to the current time step into the large language model, and predict the learning effect of the next time step as the student's first knowledge mastery level of the knowledge point;

[0036] The difficulty of the exercises includes the difficulty of the problem and the difficulty of the knowledge concept. The calculation formula of the difficulty of the problem is as follows:

[0037]

[0038] The calculation formula of the difficulty of the knowledge concept is as follows:

[0039]

[0040] In the formula, QS represents the difficulty of the question; KC represents the difficulty of the knowledge concept; S i Indicates that the question q is answered j or knowledge point k m A collection of students, a ij = = 1 and a im =1 means the question is answered correctly; C qs represents the difficulty of the predefined problem, C kc and knowledge concept difficulty.

[0041] In some embodiments, the step of inputting the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises, and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student includes:

[0042] Perform multi-layer perception on the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises, and the difficulty of the exercises to obtain the first difficulty enhanced question embedding of the current time step;

[0043] Embedding the first difficulty-enhanced problem into an input adaptive sequence neural network to obtain a first knowledge state at a current time step;

[0044] Get the second difficulty enhanced question embedding for the next time step;

[0045] An inner product is performed on the first knowledge state and the second difficulty-enhanced problem embedding to obtain the second knowledge mastery level of the current student.

[0046] In some embodiments, updating the recommended exercise set according to the first knowledge mastery level and the second knowledge mastery level includes:

[0047] Taking a weighted average of the first knowledge mastery level and the second knowledge mastery level to obtain a comprehensive level prediction value of the current student;

[0048] Get the knowledge target state value;

[0049] Calculating the recommendation probability of the exercises in the knowledge set of the to-be-selected exercises according to the comprehensive level prediction value and the knowledge target state value;

[0050] The exercises in the recommended exercise set are updated according to the recommendation probability and the recommendation threshold.

[0051] To achieve the above purpose, another aspect of the embodiment of the present application provides a learning recommendation device based on artificial intelligence and knowledge tracking, the device comprising:

[0052] The first module is used to construct a set of knowledge of exercises to be selected according to the knowledge documents of exercises;

[0053] The second module is used to build a reinforcement learning framework;

[0054] The third module is used to perform a novel search in the candidate exercise knowledge set through the reinforcement learning framework to generate a prompt word template;

[0055] The fourth module is used to input the prompt word template into the large language model, generate recommended exercises, the knowledge points and exercise difficulty of the recommended exercises, and predict the student's first knowledge mastery level of the knowledge points;

[0056] A fifth module is used to input the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student;

[0057] The sixth module is used to update the recommended exercise set according to the first knowledge mastering level and the second knowledge mastering level.

[0058] To achieve the above objective, another aspect of the present application provides a computer device, including:

[0059] at least one processor;

[0060] at least one memory for storing at least one program;

[0061] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0062] To achieve the above objective, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0063] The embodiments of the present application include at least the following beneficial effects: The present application provides a learning recommendation method, device and medium based on artificial intelligence and knowledge tracking. The scheme constructs a knowledge set of selected exercises according to the exercise knowledge document and constructs a reinforcement learning framework. Then, the reinforcement learning framework is used to perform novel retrieval in the knowledge set of selected exercises to generate a prompt word template. The prompt word template is input into the large language model to generate recommended exercises, knowledge points of the recommended exercises and exercise difficulty, and the student's first knowledge mastery level of the knowledge points is predicted. Then, the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the exercise difficulty are input into the pre-trained difficulty matching knowledge tracking model to obtain the current student's second knowledge mastery level. Then, the recommended exercise set is updated according to the first knowledge mastery level and the second knowledge mastery level, so that the characteristics of artificial intelligence feedback and knowledge tracking can be effectively combined to perform learning recommendations, thereby effectively improving the accuracy of learning recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flow chart of a learning recommendation method based on artificial intelligence and knowledge tracking provided in an embodiment of the present application;

[0065] Figure 2 is a flow chart of RAG provided in an embodiment of the present application;

[0066] Figure 3It is a schematic diagram of the CBOW model provided in the embodiment of the present application;

[0067] Figure 4 It is a schematic diagram of the architecture of the reinforcement learning framework provided in the embodiment of the present application;

[0068] Figure 5 It is a data processing schematic diagram of the DIMKT model provided in an embodiment of the present application;

[0069] Figure 6 It is a complete flow chart of the learning recommendation method based on artificial intelligence and knowledge tracking provided in the embodiment of the present application;

[0070] Figure 7 It is a structural diagram of a learning recommendation device based on artificial intelligence and knowledge tracking provided in an embodiment of the present application;

[0071] Figure 8 It is a schematic diagram of the hardware structure of the computer device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application.

[0073] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0074] Before describing the embodiments of the present application in detail, some nouns and terms involved in the embodiments of the present application are first described. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0075] Reinforcement learning is a machine learning technique that learns how to make optimal decisions by interacting with the environment. In the field of intelligent education, reinforcement learning is used to explore the interactive relationship between students and learning resources and learn how to generate personalized learning recommendations for students.

[0076] Retrieval-augmented generation (RAG) refers to optimizing the output of large language models so that they can reference authoritative knowledge bases outside the training data source before generating a response.

[0077] Large Language Models (LLMs) are trained with massive amounts of data and use billions of parameters to generate original output for tasks such as answering questions, translating languages, and completing sentences. Building on the powerful capabilities of LLMs, RAG extends them to access internal knowledge bases of specific domains or organizations.

[0078] The Actor-Critic method in reinforcement learning is an algorithm that combines policy learning and value function learning, where the Actor uses a policy function to generate actions and interact with the environment, while the Critic uses a value function to evaluate the Actor's performance and guide the Actor's actions in the next stage. In a personalized learning recommendation system, the Actor is responsible for generating recommended actions based on the student's status and interacting with the environment, while the Critic evaluates the effects of these recommended actions and guides the Actor to optimize its strategy through feedback, jointly promoting the personalization and accuracy of the recommendation system.

[0079] The embodiments of the present application provide a learning recommendation method, device and medium based on artificial intelligence and knowledge tracking. After constructing a knowledge set of exercises to be selected according to the exercise knowledge document and constructing a reinforcement learning framework, the present application performs a novel search in the knowledge set of exercises to be selected through the reinforcement learning framework to generate a prompt word template, inputs the prompt word template into a large language model to generate recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises, and predicts the students' first knowledge mastery level of the knowledge points. Then, the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises are input into a pre-trained difficulty matching knowledge tracking model to obtain the current student's second knowledge mastery level, and then the recommended exercise set is updated according to the first knowledge mastery level and the second knowledge mastery level, so that the characteristics of artificial intelligence feedback and knowledge tracking can be effectively combined to perform learning recommendations, thereby effectively improving the accuracy of learning recommendations.

[0080] The learning recommendation method based on artificial intelligence and knowledge tracking provided in the embodiment of the present application relates to the field of intelligent learning technology. The learning recommendation method based on artificial intelligence and knowledge tracking provided in the embodiment of the present application can be applied to a terminal, can also be applied to a server, and can also be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or it can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured to provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms and other basic cloud computing services. The server can also be a node server in the blockchain network; the software can be an application that implements a learning recommendation method based on artificial intelligence and knowledge tracking, etc., but is not limited to the above forms.

[0081] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0082] Figure 1 is an optional flowchart of a learning recommendation method based on artificial intelligence and knowledge tracking provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S110 to S160:

[0083] Step S110, constructing a set of to-be-selected exercise knowledge according to the exercise knowledge document;

[0084] Step S120, constructing a reinforcement learning framework;

[0085] Step S130, performing a novel search in the knowledge set of the selected exercises through a reinforcement learning framework to generate a prompt word template;

[0086] Step S140, inputting the prompt word template into the large language model, generating recommended exercises, the knowledge points and exercise difficulty of the recommended exercises, and predicting the student's first knowledge mastery level of the knowledge points;

[0087] Step S150, inputting the current student's learning status, answer information corresponding to the recommended exercises, knowledge points of the recommended exercises and exercise difficulty into the pre-trained difficulty matching knowledge tracking model to obtain the current student's second knowledge mastery level;

[0088] Step S160: Update the recommended exercise set according to the first knowledge mastery level and the second knowledge mastery level.

[0089] In the embodiment of the present application, the embodiment uses the retrieval enhancement technique (RAG) to optimize the output of the large language model so that it can refer to the authoritative knowledge base other than the training data source before generating a response. The flowchart of RAG is as follows Figure 2 As shown. It can be understood that when constructing the knowledge set of exercises to be selected, this embodiment can collect the exercise knowledge documents and then divide the exercise knowledge documents into multiple text blocks of preset sizes to more efficiently process and retrieve information. Then, after converting the multiple text blocks into exercise vectors through the embedding model, the exercise vectors are stored in the vector database as the knowledge set of exercises to be selected. Specifically, the embedding model of this embodiment can adopt the Word2Vec model. Among them, Word2Vec is a word embedding method based on neural network, which is used to map words to low-dimensional vector space. There are two main training methods: CBOW (Continuous Bag of Words) and Skip-gram. This embodiment uses the CBOW model. The CBOW model is suitable for smaller data sets and common vocabulary. Its training input is the word vector corresponding to the context-related words of a certain feature word, and the output is the word vector of this specific word. The CBOW model predicts the central word through the context words, that is, using x n-2 x n-2 ,x n-1 ,x n+1 x n+2 , to predict x n It predicts the central word by summing up the context word vectors and inputting them into a neural network. The CBOW model diagram is as follows Figure 3 This embodiment embeds the exercise text block into the CBOW model, and stores all generated vectors in the vector database as a set of exercise knowledge to be selected, provides an exercise knowledge base for subsequent recommendations, and optimizes the output of the large language model LLM.

[0090] In the embodiment of the present application, the reinforcement learning framework can adopt the actor-critic framework. Among them, the actor-critic algorithm is a type of reinforcement learning method that combines policy gradient and value function estimation, and trains a policy and a value function at the same time. The structure diagram is as follows Figure 4 As shown. Figure 4 It can be seen that the reinforcement learning framework of this embodiment includes a critic network and an actor network. The actor-critic framework selects recommended actions through actors, and critics evaluate the quality of recommended actions and provide feedback to improve the recommendation strategy, ultimately achieving the recommendation of personalized exercise resources for students.

[0091] It is understandable that the process of constructing the reinforcement learning framework in this embodiment includes but is not limited to the following steps:

[0092] Initialize the first learning rate α1 and the first network parameter θ of the evaluation network q , and initialize the second learning rate α2 and the second network parameter θ of the policy network π ;

[0093] Then the following steps are executed repeatedly until the number of cycles reaches the preset number:

[0094] The agent interacts with the environment for n steps and collects the first sequence {s0, a0, r1, s1, a1, r2, …}, where the first sequence includes states, actions, and rewards;

[0095] Calculate the first loss function of the evaluation network Critic according to the first sequence And the second loss function of the policy network Actor

[0096]

[0097]

[0098] Update the first network parameter θ according to the first loss function q , and update the second network parameter θ according to the second loss function π :

[0099]

[0100]

[0101] In an embodiment of the present application, the intelligent agent of this embodiment completes the interaction with the environment by extracting exercises from the above-mentioned knowledge set of exercises to be selected, extracting one exercise at a time, and then handing it over to the large language model LLM in the form of a prompt word template. It can be understood that the intelligent agent's exercise extraction request will be treated as a user query, and the exercise extraction request will be input into the embedding model for vectorization processing to obtain a query vector. Then, the vector database is searched for knowledge texts or historical conversation records that are semantically similar to the query vector. Finally, the exercise extraction request of the current step is combined with the information retrieved in the previous step (knowledge text or historical conversation record information) to construct a prompt word template, which is input into the large language model, and wait for the model to output exercises.

[0102] In the embodiment of the present application, the embodiment obtains the knowledge points and difficulties of the exercises by using the large language model, and performs learning effect prediction to obtain the first knowledge mastery level. Specifically, the embodiment obtains the recommended exercises q by inputting the prompt word template into the large language model at time step t to perform prompt word analysis. t After that, the keywords, phrases and sentence structures in the recommended exercises are analyzed, and the first knowledge point is determined based on the keywords, phrases and sentence structures. Then, the first knowledge point is matched with the knowledge points in the knowledge set of the selected exercises to determine the knowledge point k corresponding to the recommended exercises. m ; At the same time, obtain the historical answer record information of each exercise in the knowledge set of the selected exercises, determine the difficulty of the recommended exercises based on the historical answer record information, and then convert the exercise data (content q t , knowledge concept k m 、Question Difficulty QS t , Knowledge Concept Difficulty KC t ) and the historical answer record information of the exercise corresponding to the current time step are input into the large language model, and the learning effect of the next time step t+1 is predicted as the student’s first knowledge mastery level y of the knowledge point t+1 .

[0103] It is understandable that the difficulty of exercises includes the difficulty of questions and the difficulty of knowledge concepts. The calculation formula of question difficulty is as follows:

[0104]

[0105] The calculation formula for knowledge concept difficulty is as follows:

[0106]

[0107] In the formula, QS represents the difficulty of the question; KC represents the difficulty of the knowledge concept; S i Indicates that the question q is answered j or knowledge point k m A collection of students, a ij = = 1 and aim =1 means the question is answered correctly; C qs represents the difficulty of the predefined problem, C kc and knowledge concept difficulty.

[0108] In the embodiment of the present application, the difficulty matching knowledge tracking model is used to track the current knowledge state of the student and predict the learning effect. Specifically, the difficulty matching knowledge tracking (DIMKT) model is a knowledge tracking model that measures the problem difficulty effect by establishing a relationship between the student's knowledge state and the problem difficulty level, and uses the inner product of the student's knowledge state and the problem embedding to simulate the process of students applying knowledge to answer questions and predict the student's performance. Figure 5 As shown, this embodiment uses the DIMKT model to establish the relationship between the student's knowledge status and the problem difficulty level, evaluate the student's dynamic knowledge status and predict the learning effect.

[0109] It can be understood that this embodiment uses the difficulty of the question and the difficulty of the knowledge concept to enhance the question embedding. Specifically, in time step t, the recommended exercises q obtained in the above steps are t , the current student's learning status, the answer information corresponding to the recommended exercises, and the knowledge points k of the recommended exercises m , Exercise Difficulty QS t , Knowledge Concept Difficulty KC t Input multi-layer perception (MLP) to get the first difficulty enhancement question embedding x at the current time step t , the specific formula is as follows:

[0110]

[0111] In the formula, ⊕ is a cascade operation, is the weight matrix, is the deviation term, d k It's a dimension.

[0112] Then the first difficulty enhancement problem is embedded into the input adaptive sequence neural network to obtain the first knowledge state h at the current time step t Specifically, the adaptive sequential neural network is used to capture the connection between knowledge state and problem difficulty in DIMKT, including a subjective difficulty perception module before practice, a personalized knowledge acquisition module during practice, and a knowledge state update module after practice:

[0113] Subjective difficulty feeling module before practice: In order to calculate students' subjective feelings about the difficulty of the problem, the difference between the difficulty-enhanced problem embedding and the students' previous knowledge state is considered, that is, if the students' knowledge state does not meet the requirements of the problem, they will feel difficult. The subjective feeling SDF can be obtained t , as shown in the following formula:

[0114]

[0115] In the formula, tanh is a nonlinear activation function, σ is a sigmoid activation function, is the weight matrix. Here, is the slope, It is the direct output of subjective feelings, and further application Select and retain important features, and finally get the students' subjective difficulty perception SDF t .

[0116] Personalized knowledge acquisition module in practice: students’ subjective difficulty perception SDF t and students' answers t Merge, gaining knowledge PKA in a similar way to the subjective difficulty module t , as shown in the following formula:

[0117]

[0118] In the formula, is the weight matrix, is the slope, It is the direct output of individual knowledge and can be further applied Select and retain important features, and finally obtain the knowledge PKA of individual students t .

[0119] Post-exercise knowledge status update module: define knowledge indicators According to the students' previous knowledge t-1 、Answer t and Exercise Difficulty KC t , QS t , to update the student's knowledge status h t , as shown in the following formula:

[0120]

[0121] In the formula, is the weight matrix, is the slope.

[0122] Then get the second difficulty enhanced question embedding for the next time step, and transform the knowledge state h of time step t into t and the second difficulty enhanced question embedding at time step t+1 Perform inner product to simulate the practical process of students applying the knowledge they have learned to answer exercises. Use the sigmoid activation function to infer the student's performance y at time step t+1 from the inner product result t+1 As the second knowledge mastery level, the specific formula is as follows:

[0123] y t+1 = σ(∑(h t ·q t+1 ))。

[0124] In the embodiments of the present application, after obtaining the first knowledge mastery level predicted by the LLM and y1, and the second knowledge mastery level predicted by the DIMKT as y2, the first knowledge mastery level and the second knowledge mastery level are weighted and averaged to obtain the predicted value y of the student's comprehensive knowledge mastery level, as shown in the following formula:

[0125] y = w1·y1 + w2·y2;

[0126] In the formula, w1 and w2 are the weights of the predicted values of the LLM and the DIMKT respectively, and w1 + w2 = 1.

[0127] Set a threshold s t (The value range can be set between 0.5 and 1) as the target state value of the knowledge point, which is used to define the reward value Reward in the actor-critic framework.

[0128] Calculate the recommendation probability of the exercises in the set of candidate exercise knowledge according to the comprehensive level predicted value and the knowledge target state value. Specifically, determine the reward feedback Reward as: after performing a certain recommended action from the current state, the student's comprehensive knowledge mastery level y reaches the target state value s of the knowledge point t , the reward value Reward is 1, otherwise Reward is 0; the specific formula is as follows:

[0129]

[0130] The critic network judges whether the current knowledge point mastery degree reaches the target value and judges the behavior score of the actor; the actor network modifies the recommendation probability p based on the critic score.

[0131] Then, set a recommended action threshold rec, and the value range of rec can be set between 0.5 and 1. If p ≥ rec, put the corresponding exercise into the recommended exercise set, and then continue with step S130, and the agent extracts the next exercise from the exercise knowledge base; if p < rec, directly proceed to step S130.

[0132] In some embodiments, the complete implementation process of the method provided in the embodiments of the present application, as Figure 6 shown, includes but is not limited to the following steps:

[0133] Step S610, construct an exercise knowledge base and embed the model to form a vector database as a set of candidate exercises;

[0134] Step S620: construct a reinforcement learning actor-critic framework, where the intelligent agent extracts exercises from the set of exercises to be selected, and then gives them to the large language model LLM in the form of prompt templates.

[0135] Step S630: The large language model LLM generates a recommended exercise e, and outputs the knowledge points and difficulty of the exercise e, as well as the predicted value y1 of the student's mastery level of the current knowledge point.

[0136] Step S640: For a certain student, the learning status s, the answer to the exercise e, the knowledge points and the difficulty of the exercise are input into the trained DIMKT model, the output is the predicted value y2 of the mastery level of the current knowledge point, and the student's learning status is updated.

[0137] Step S650: The critic network determines whether the mastery of the current knowledge point reaches the target value and judges the actor's behavior score; the actor network modifies the recommended behavior probability based on the critic score, puts the exercises corresponding to the recommended behavior probability exceeding the threshold into the recommended exercise set, and then continues to execute step S620.

[0138] In summary, the method of the embodiment of the present application provides an innovative reinforcement learning framework that can integrate artificial intelligence feedback and knowledge tracking technology to provide highly personalized learning resource recommendations for students.

[0139] Reference Figure 7 , the embodiment of the present application provides a learning recommendation device based on artificial intelligence and knowledge tracking, the device comprising:

[0140] The first module 710 is used to construct a set of to-be-selected exercise knowledge according to the exercise knowledge document;

[0141] The second module 720 is used to build a reinforcement learning framework;

[0142] The third module 730 is used to perform a novel search in the knowledge set of the selected exercises through a reinforcement learning framework to generate a prompt word template;

[0143] The fourth module 740 is used to input the prompt word template into the large language model, generate recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises, and predict the students' first knowledge mastery level of the knowledge points;

[0144] The fifth module 750 is used to input the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into the pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student;

[0145] The sixth module 760 is used to update the recommended exercise set according to the first knowledge mastering level and the second knowledge mastering level.

[0146] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0147] The embodiment of the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned learning recommendation method based on artificial intelligence and knowledge tracking when executing the computer program. The computer device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0148] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0149] See also Figure 8 , Figure 8 The hardware structure of a computer device according to another embodiment is shown, and the computer device includes:

[0150] The processor 810 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0151] The memory 820 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 820 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 820, and the processor 810 calls and executes the learning recommendation method based on artificial intelligence and knowledge tracking in the embodiment of this application;

[0152] Input / output interface 830, used to implement information input and output;

[0153] The communication interface 840 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0154] bus 850 , which transmits information between the various components of the device (e.g., processor 810 , memory 820 , input / output interface 830 , and communication interface 840 );

[0155] The processor 810 , the memory 820 , the input / output interface 830 and the communication interface 840 are connected to each other in communication within the device via a bus 850 .

[0156] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned learning recommendation method based on artificial intelligence and knowledge tracking.

[0157] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0158] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0160] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0161] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0162] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0163] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0164] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0165] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0166] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0168] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0169] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A learning recommendation method based on artificial intelligence and knowledge tracing, characterized in that: The method comprises the following steps: Construct a knowledge set of exercises to be selected according to the exercise knowledge document; Build a reinforcement learning framework; Performing a novel search in the candidate exercise knowledge set through the reinforcement learning framework to generate a prompt word template; Input the prompt word template into the large language model, generate recommended exercises, the knowledge points and exercise difficulty of the recommended exercises, and predict the student's first knowledge mastery level of the knowledge points; Inputting the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student; The recommended exercise set is updated according to the first knowledge mastering level and the second knowledge mastering level.

2. The method according to claim 1, characterized in that The step of constructing a set of to-be-selected exercise knowledge according to the exercise knowledge document includes: Collect exercise knowledge documents; Splitting the exercise knowledge document into a plurality of text blocks of preset sizes; Converting the plurality of text blocks into question vectors through an embedding model; The exercise vectors are stored in a vector database as the knowledge set of exercises to be selected.

3. The method according to claim 2, characterized in that The reinforcement learning framework includes an evaluation network and a strategy network; the construction of the reinforcement learning framework includes: Initializing a first learning rate and a first network parameter of the evaluation network, and initializing a second learning rate and a second network parameter of the policy network; The following steps are executed repeatedly until the number of cycles reaches the preset number: The agent interacts with the environment and collects a first sequence, the first sequence including a state, an action, and a reward; Calculate a first loss function of the evaluation network and a second loss function of the policy network according to the first sequence; The first network parameters are updated according to the first loss function, and the second network parameters are updated according to the second loss function.

4. The method according to claim 3, characterized in that The process of performing a novel search in the knowledge set of the selected exercises through the reinforcement learning framework to generate a prompt word template includes: Obtaining an exercise extraction request of an agent in the reinforcement learning framework; Converting the exercise extraction request into a query vector; Searching the vector database for knowledge text or historical conversation record information similar to the query vector; The prompt word template is constructed according to the knowledge text or historical dialogue record information corresponding to the current step exercise extraction request and the previous step exercise extraction request.

5. The method according to claim 1, characterized in that The step of inputting the prompt word template into a large language model, generating recommended exercises, knowledge points and exercise difficulty of the recommended exercises, and predicting the student's first knowledge mastery level of the knowledge points includes: Inputting the prompt word template into a large language model to perform prompt word parsing to obtain the recommended exercises; Analyzing key words, phrases and sentence structures in the recommended exercises; Determine a first knowledge point according to the keyword, the phrase and the sentence structure; Matching the first knowledge point with the knowledge points in the knowledge set of the to-be-selected exercises to determine the knowledge point corresponding to the recommended exercises; Obtaining historical answer record information of each exercise in the knowledge set of exercises to be selected; Determining the difficulty of the recommended exercises according to the historical answer record information; Input the exercise data of the current time step and the historical answer record information of the exercise corresponding to the current time step into the large language model, and predict the learning effect of the next time step as the student's first knowledge mastery level of the knowledge point; The difficulty of the exercises includes the difficulty of the problem and the difficulty of the knowledge concept. The calculation formula of the difficulty of the problem is as follows: The calculation formula of the difficulty of the knowledge concept is as follows: In the formula, QS represents the difficulty of the question; KC represents the difficulty of the knowledge concept; S i Indicates that the question q is answered j or knowledge point k m A collection of students, a ij = = 1 and a im =1 means the question is answered correctly; C qs represents the difficulty of the predefined problem, C kc and knowledge concept difficulty.

6. The method according to claim 1, characterized in that The step of inputting the learning status of the current student, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student includes: Perform multi-layer perception on the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises, and the difficulty of the exercises to obtain the first difficulty enhanced question embedding of the current time step; Embedding the first difficulty-enhanced problem into an input adaptive sequence neural network to obtain a first knowledge state at a current time step; Get the second difficulty enhanced question embedding for the next time step; An inner product is performed on the first knowledge state and the second difficulty-enhanced problem embedding to obtain the second knowledge mastery level of the current student.

7. The method according to claim 1, characterized in that The updating of the recommended exercise set according to the first knowledge mastering level and the second knowledge mastering level includes: Taking a weighted average of the first knowledge mastery level and the second knowledge mastery level to obtain a comprehensive level prediction value of the current student; Get the knowledge target state value; Calculating the recommendation probability of the exercises in the knowledge set of the to-be-selected exercises according to the comprehensive level prediction value and the knowledge target state value; The exercises in the recommended exercise set are updated according to the recommendation probability and the recommendation threshold.

8. A learning recommendation device based on artificial intelligence and knowledge tracking, characterized in that: The device comprises: The first module is used to construct a set of knowledge of exercises to be selected according to the knowledge documents of exercises; The second module is used to build a reinforcement learning framework; The third module is used to perform a novel search in the candidate exercise knowledge set through the reinforcement learning framework to generate a prompt word template; The fourth module is used to input the prompt word template into the large language model, generate recommended exercises, the knowledge points and exercise difficulty of the recommended exercises, and predict the student's first knowledge mastery level of the knowledge points; A fifth module is used to input the current student's learning status, the answer information corresponding to the recommended exercises, the knowledge points of the recommended exercises and the difficulty of the exercises into a pre-trained difficulty matching knowledge tracking model to obtain the second knowledge mastery level of the current student; The sixth module is used to update the recommended exercise set according to the first knowledge mastering level and the second knowledge mastering level.

9. A computer device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images