A learning path recommendation method and device, electronic equipment, and storage medium

Abstract

The application discloses a learning path recommendation method and device, electronic equipment and a storage medium, the method comprises the following steps: constructing a project hierarchical graph according to the association relationship between each learning project; high-order information and difficulty information between learning projects are aggregated through a hierarchical graph neural network to obtain project representation of each exercise project; a hierarchical reinforcement learning framework driven by difficulty is designed, and the framework comprises two intelligent agents, wherein the upper intelligent agent is responsible for selecting the next stage of knowledge projects to be learned in the knowledge project layer, and the lower intelligent agent is responsible for selecting the next step of exercise projects to be practiced in the exercise project layer; and the upper intelligent agent determines the initial difficulty of the exercise project, and controls the maximum number of exercises of the learner according to the learning project difficulty information transmitted from the upper intelligent agent, a communication mechanism between the two intelligent agents is realized, and thus an effective smooth learning path recommendation method is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data mining, and particularly relates to a learning path recommendation method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the development of education technology and the popularity of online learning platforms, learners can more easily obtain learning resources. However, with the explosive growth of knowledge and learning resources in the information age, it has also brought new pain points for people to obtain new knowledge, that is, how learners can accurately obtain learning resources suitable for their own needs from a large number of complex learning resources and efficiently learn in the right learning order.

[0003] Therefore, the current learning path recommendation technology for learners has emerged as the times require. The current learning path method mainly determines the knowledge items that the learner needs to learn, that is, the knowledge points or skills to be learned, according to the predecessor-successor relationship of each knowledge item, that is, the learning order relationship of the knowledge items, and provides all the exercise items related to the knowledge items to the learner for practice, that is, the two items are provided to the learner for practice, so that the learner can learn and master the corresponding knowledge items.

[0004] However, since this method only considers the predecessor-successor relationship between the items and the examination relationship, it does not consider the item difficulty, which makes the recommended learning path not smooth enough, that is, the learning path is "inefficient", that is, the same learning amount is used to learn items of different difficulties, resulting in unnecessary time spent on mastered knowledge or ineffective mastery of knowledge after completing the learning amount, and the learning path is "bumpy", that is, after mastering the exercises of high difficulty, the learner is still recommended to learn items of low difficulty. SUMMARY

[0005] Based on the deficiencies of the prior art, the present application provides a learning path recommendation method and device, an electronic device and a storage medium to solve the problem of the non-smooth learning path recommended by the prior art.

[0006] In order to achieve the above purpose, the present application provides the following technical solutions:

[0007] The first aspect of the present application provides a learning path recommendation method, comprising:

[0008] According to the association relationship between each learning item, a project hierarchical diagram is constructed; wherein the learning items include a plurality of knowledge items and each corresponding exercise item; the association relationship includes a predecessor-successor relationship and an examination relationship;

[0009] The hierarchical graph neural network aggregates high-order information and difficulty information among the learning items to obtain item representations of each of the practice items;

[0010] The high-level agent determines a current knowledge item from each of the target knowledge items on the item hierarchical graph according to a current knowledge state of the user and a previous practice number of times, determines a current initial difficulty and a current practice limit number of times, and passes them to the low-level agent;

[0011] The low-level agent selects a current practice item of a corresponding difficulty from each of the practice items corresponding to the current knowledge item according to the current knowledge state of the user from the current initial difficulty, provides the current practice item to the user for learning, accumulates a current stage practice number of times, and updates the current knowledge state of the user according to the item representation of the current practice item and learning feedback until the current stage practice number of times reaches the current practice limit number of times or the current knowledge state of the user reaches a state threshold;

[0012] The low-level agent feeds back the current stage practice number of times updated most recently and the current knowledge state of the user updated most recently to the high-level agent to return to execute the determination of the current knowledge item from each of the target knowledge items on the item hierarchical graph according to the current knowledge state of the user and the previous practice number of times, the determination of the current initial difficulty and the current practice limit number of times, and the like until the user masters each of the target knowledge items.

[0013] Optionally, in the learning path recommendation method, the aggregation of high-order information and difficulty information among the learning items by the hierarchical graph neural network to obtain the item representations of each of the practice items includes:

[0014] Based on historical learning records, a ratio of a number of incorrect answers to a total number of answers of each of the learning items is multiplied by a corresponding preset difficulty level to obtain a difficulty of each of the learning items;

[0015] A knowledge item embedding matrix and a practice item embedding matrix are generated, and a knowledge item difficulty embedding matrix and a practice item difficulty embedding matrix are generated according to the difficulty of each of the learning items; wherein the knowledge item embedding matrix and the practice item embedding matrix are respectively used to represent all of the knowledge items and the practice items;

[0016] respectively, the combined embedding vector of each of the knowledge items is obtained by connecting the item vector corresponding to the knowledge item in the knowledge item embedding matrix and the difficulty vector corresponding to the knowledge item in the knowledge item difficulty embedding matrix, multiplying the connection operation result by a first weight matrix, and adding a first bias term, and respectively for each of the exercise items, the combined embedding vector of each of the exercise items is obtained by connecting the item vector corresponding to the exercise item in the exercise item embedding matrix and the difficulty vector corresponding to the exercise item in the exercise item difficulty embedding matrix, multiplying the connection operation result by a second weight matrix, and adding a second bias term.

[0017] respectively, the item representation of each of the exercise items is obtained by connecting the combined embedding vector of the exercise item with the mean of the combined embedding vectors of each of the knowledge items adjacent to the exercise item.

[0018] Optionally, in the learning path recommendation method described above, the determining, by the high-level agent, of the current knowledge item from each of the target knowledge items on the item hierarchical graph, the current initial difficulty, and the current exercise limit number according to the current knowledge state of the user and the number of exercises in the last stage, comprises:

[0019] The current state of the high-level agent is obtained by connecting the current knowledge state of the user with the encoding vector of each of the target knowledge items by the high-level agent.

[0020] The current knowledge item is determined from the current knowledge item dynamic space based on the current state of the high-level agent by using a proximal policy optimization algorithm, wherein the current knowledge item dynamic space is composed of the target knowledge items on the item hierarchical graph.

[0021] The current initial difficulty is calculated using the current knowledge state of the user and the self-defined correct answer probability.

[0022] The current exercise limit number is calculated based on the current knowledge state of the user, the number of exercises in the last stage, the exercise limit number in the last stage, and the difficulty of the current knowledge item by using the trained multilayer perceptron.

[0023] Optionally, in the learning path recommendation method described above, the low-level agent selects, from the practice items corresponding to the current knowledge item, a current practice item of a corresponding difficulty according to the current knowledge state of the user, provides the user with the current practice item for learning, accumulates the current stage practice times, and updates the current knowledge state of the user according to the item representation and learning feedback of the current practice item, until the current stage practice times reach the current practice limit times or the current knowledge state of the user reaches the state threshold, comprising:

[0024] The low-level agent selects, from the practice items corresponding to the current knowledge item, a practice item of the current initial difficulty as the current practice item, and provides the user with the current practice item for learning.

[0025] Accumulate the current stage practice times and update the current knowledge state of the user according to the item representation and learning feedback of the current practice item.

[0026] Determine whether the accumulated current stage practice times reach the current practice limit times or the current knowledge state of the user reaches the state threshold.

[0027] If it is determined that the accumulated current stage practice times do not reach the current practice limit times and the current knowledge state of the user does not reach the state threshold, the current knowledge state of the user is connected with the encoding vector of the current knowledge item to obtain the current state of the low-level agent.

[0028] Using the actor-critic algorithm, the low-level agent selects a new current learning item from the practice items corresponding to the current knowledge item based on the current state of the low-level agent, provides the user with the new current learning item for learning, and returns to update the current knowledge state of the user according to the item representation and learning feedback of the current practice item.

[0029] If it is determined that the accumulated current stage practice times reach the current practice limit times or the current knowledge state of the user reaches the state threshold, the low-level agent executes the current latest updated current stage practice times as the last stage practice times and the current latest updated current knowledge state of the user to the high-level agent.

[0030] The second aspect of the present application provides a learning path recommendation device, comprising:

[0031] The constructing module is configured to construct a project hierarchical graph according to an association relationship among the learning projects, wherein the learning projects include a plurality of knowledge projects and corresponding practice projects, and the association relationship includes a predecessor-successor relationship and an examination relationship.

[0032] The aggregating module is configured to aggregate high-order information and difficulty information among the learning projects by a hierarchical graph neural network to obtain project representations of the practice projects.

[0033] The high-level agent module is configured to determine a current knowledge project from the target knowledge projects on the project hierarchical graph, and determine a current initial difficulty and a current practice limit number according to a current knowledge state of the user and a previous stage practice number by a high-level agent, and pass them to a low-level agent.

[0034] The low-level agent module is configured to select a current practice project of a corresponding difficulty from the practice projects corresponding to the current knowledge project according to the current knowledge state of the user from the current initial difficulty by a low-level agent, provide the current practice project to the user for learning, and accumulate a current stage practice number and update the current knowledge state of the user according to the project representation of the current practice project and a learning feedback until the current stage practice number reaches the current practice limit number or the current knowledge state of the user reaches a state threshold.

[0035] The low-level agent module is configured to select a current practice project of a corresponding difficulty from the practice projects corresponding to the current knowledge project according to the current knowledge state of the user from the current initial difficulty by a low-level agent, provide the current practice project to the user for learning, and accumulate a current stage practice number and update the current knowledge state of the user according to the project representation of the current practice project and a learning feedback until the current stage practice number reaches the current practice limit number or the current knowledge state of the user reaches a state threshold.

[0036] Optionally, in the learning path recommendation device, the aggregating module includes:

[0037] The difficulty calculating module is configured to multiply a ratio of a number of incorrect answers to a total number of answers of each learning project by a corresponding preset difficulty level based on historical learning records to obtain a difficulty of each learning project.

[0038] The matrix generating module is configured to generate a knowledge project embedding matrix and a practice project embedding matrix, and generate a knowledge project difficulty embedding matrix and a practice project difficulty embedding matrix according to the difficulty of each learning project, wherein the knowledge project embedding matrix and the practice project embedding matrix are used to represent all the knowledge projects and the practice projects.

[0039] a fusion module, configured to, for each of the knowledge items, connect a difficulty vector corresponding to the knowledge item in a difficulty embedding matrix of the knowledge items with a project vector corresponding to the knowledge item in a project embedding matrix of the knowledge items, multiply a result of the connection operation by a first weight matrix, and add a first bias term to obtain a combined embedding vector of each of the knowledge items, and for each of the exercise items, connect a difficulty vector corresponding to the exercise item in a difficulty embedding matrix of the exercise items with a project vector corresponding to the exercise item in a project embedding matrix of the exercise items, multiply a result of the connection operation by a second weight matrix, and add a second bias term to obtain a combined embedding vector of each of the exercise items;

[0040] a combination module, configured to, for each of the exercise items, connect a result of a connection operation between the combined embedding vector of the exercise item and a mean of the combined embedding vectors of the adjacent knowledge items of the exercise item to obtain a project representation of each of the exercise items.

[0041] Optionally, in the learning path recommendation apparatus, the high-level agent module is configured to, when determining the current knowledge item and the current initial difficulty and the current exercise limit number from the target knowledge items on the project hierarchical graph according to the current knowledge state of the user and the number of exercises in the previous stage, perform the following operations:

[0042] connecting, by the high-level agent, the current knowledge state of the user with the encoded vectors of the target knowledge items to obtain a current state of the high-level agent;

[0043] determining the current knowledge item from a current knowledge item dynamic space based on the current state of the high-level agent by using a proximal policy optimization algorithm, wherein the current knowledge item dynamic space is composed of the target knowledge items on the project hierarchical graph;

[0044] calculating the current initial difficulty by using the current knowledge state of the user and the self-defined correct answer probability;

[0045] calculating the current exercise limit number by using the current knowledge state of the user, the number of exercises in the previous stage, the exercise limit number in the previous stage, and the difficulty of the current knowledge item by using the trained multilayer perceptron.

[0046] Optionally, in the above-mentioned learning path recommendation device, the low-level agent module executes the step of selecting, from the current initial difficulty level, a current practice item of corresponding difficulty from each practice item corresponding to the current knowledge item according to the current knowledge status of the user through the low-level agent, and providing the current practice item to the user for learning, and accumulating the number of practice times in the current stage and updating the current knowledge status of the user according to the item representation and learning feedback of the current practice item, until the number of practice times in the current stage reaches the current practice limit number, or the current knowledge status of the user reaches the status threshold, and is used to:

[0047] Selecting, by the low-level agent, from the various practice items corresponding to the current knowledge item, the practice item with the current initial difficulty level as the current practice item, and providing it to the user for learning;

[0048] Accumulating the number of exercises in the current stage and updating the user's current knowledge status based on the item representation and learning feedback of the current exercise item;

[0049] Determining whether the accumulated number of exercises in the current stage reaches the current exercise limit, or whether the current knowledge status of the user reaches a status threshold;

[0050] If it is determined that the accumulated number of exercises in the current stage has not reached the current exercise limit, and the current knowledge state of the user has not reached the state threshold, then performing a concatenation operation on the current knowledge state of the user and the encoding vector of the current knowledge item to obtain the current state of the low-level agent;

[0051] Utilizing an actor-critic algorithm, based on the current state of the lower-level agent, selecting a new current learning item from the practice items corresponding to the current knowledge item, providing the new item to the user for learning, and returning to execute the accumulated number of current stage practices and updating the user's current knowledge state based on the item representation of the current practice item and learning feedback;

[0052] If it is determined that the accumulated number of exercises in the current stage reaches the current exercise limit, or the current knowledge status of the user reaches the status threshold, the low-level agent executes the most recently updated number of exercises in the current stage as the number of exercises in the previous stage, and feeds back the most recently updated current knowledge status of the user to the high-level agent.

[0053] A third aspect of the present application provides an electronic device, including:

[0054] memory and processor;

[0055] The memory is configured to store a program.

[0056] The processor is configured to execute the program, and the program, when executed, is configured to implement the learning path recommendation method according to any one of the preceding embodiments.

[0057] The fourth aspect of the present application provides a computer storage medium configured to store a computer program, and the computer program, when executed, is configured to implement the learning path recommendation method according to any one of the preceding embodiments.

[0058] The learning path recommendation method provided by the present application constructs a project hierarchical graph according to the association relationship between each learning project, wherein the learning project includes multiple knowledge projects and corresponding practice projects. The association relationship includes a predecessor-successor relationship and an examination relationship. Then, the hierarchical graph neural network is used to aggregate high-order information and difficulty information between the learning projects to obtain project representations of the practice projects, so as to represent the practice projects by the representations fused with the difficulty information, so that the difficulty of the projects is considered in the path recommendation. Then, the upper intelligent agent determines the current knowledge project from each knowledge project on the project hierarchical graph, and determines the current initial difficulty and the current practice limit number according to the current knowledge state of the user and the practice number of the last stage, and feeds them to the lower intelligent agent. Next, the lower intelligent agent selects the current practice project of the corresponding difficulty from the practice projects corresponding to the current knowledge project according to the current knowledge state of the user from the current initial difficulty, and provides it to the user for learning, and accumulates the practice number of the current stage and updates the current knowledge state of the user according to the project representation of the current practice project and the learning feedback, until the practice number of the current stage reaches the current practice limit number, or the current knowledge state of the user reaches the state threshold. The lower intelligent agent feeds the current latest updated practice number of the current stage and the current latest updated current knowledge state of the user to the upper intelligent agent as the practice number of the last stage, so as to return to the upper intelligent agent to perform the learning of the next stage, so that the upper intelligent agent and the lower intelligent agent and the interaction therebetween realize a difficulty-driven method of recommending learning knowledge projects and practice projects under the knowledge projects, and further ensure that the recommended learning path is smoother and the learning efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0060] Figure 1A flowchart of a learning path recommendation method provided for an embodiment of the present application;

[0061] Figure 2 A flowchart of a method for calculating a project representation of a practice project provided for an embodiment of the present application;

[0062] Figure 3 A flowchart of a method for determining a current knowledge project, a current initial difficulty and a current practice limit number provided for an embodiment of the present application;

[0063] Figure 4 A flowchart of a method for recommending a practice project by a low-level intelligent agent provided for an embodiment of the present application;

[0064] Figure 5 A schematic diagram of a system architecture for implementing a learning path recommendation method provided for an embodiment of the present application;

[0065] Figure 6 A schematic diagram of an architecture of a learning path recommendation device provided for an embodiment of the present application;

[0066] Figure 7 A schematic diagram of an architecture of an electronic device provided for an embodiment of the present application. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0068] In the present application, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of another identical element in the process, method, article or device including the element.

[0069] The embodiments of the present application provide a learning path recommendation method, as shown in Figure 1 The method comprises the following steps:

[0070] S101. Construct a project hierarchy diagram based on the relationship between various learning projects.

[0071] Among them, learning projects include multiple knowledge projects and their corresponding practice projects. The association relationship includes predecessor and successor relationships and examination relationships. Knowledge projects generally refer to knowledge points or skills, such as addition, multiplication, calculus, etc. There is a predecessor and successor relationship between knowledge projects, such as addition is the predecessor of multiplication, and multiplication is the successor of addition, which means that addition needs to be learned before learning multiplication. Practice projects refer to the practice content set to master the corresponding knowledge projects, which are practical applications or problems with an examination relationship with the knowledge projects, such as the problem "2+3" that examines addition. Through practice projects, learners can learn and master the corresponding knowledge projects. At the same time, each interaction between learners and practice projects can get instant feedback, which is usually right or wrong or a score. Learners can understand their own learning situation through feedback.

[0072] Therefore, in the embodiment of the present application, in order to fully consider the learning projects, a project hierarchical graph is first constructed based on the association relationship between the learning projects. The project hierarchical graph can be specifically expressed as HG = {V, E}. Among them, V represents the set of projects, which is all the nodes in the hierarchical graph, including knowledge projects and practice projects; E represents the set of project relationships, which is all the edges in the hierarchical graph, including the predecessor and successor relationships between knowledge projects and the inspection relationship between knowledge projects and practice projects.

[0073] Specifically, the edge connecting two knowledge items in E (c i , c j ) represents knowledge item c i It is a knowledge item c j Precursor knowledge and learning foundation; the edge connecting the practice items and knowledge items in E (e m , c j ) indicates exercise item e m Examined knowledge project c j .

[0074] S102. Aggregate high-level information and difficulty information between learning items through a hierarchical graph neural network to obtain item representations for each practice item.

[0075] It should be noted that in order to better consider the correlation between learning items and the difficulty of learning items during the path determination process, in an embodiment of the present application, high-order information and difficulty information between learning items are aggregated through a hierarchical graph neural network to obtain project representations of each practice item, and the project representations of the practice items are used to represent the nodes of each practice item in the project hierarchical graph, so that the correlation between learning items and the difficulty can be taken into account when exploring the learning path.

[0076] Specifically, each learning item and the difficulty of each learning item are represented by corresponding representation vectors, and the two are fused to obtain a vector after fusing the difficulty characteristics, and then the re-practice item and the fused difficulty characteristic vector representation of the knowledge item associated therewith are fused, so as to obtain the item representation of each practice item.

[0077] Optionally, in another embodiment of the present application, a specific implementation of step S102 includes: Figure 2

[0078] S201, based on the historical learning records, respectively multiplying the ratio of the number of incorrect answers to the total number of answers of each learning item by the corresponding preset difficulty level to obtain the difficulty of each learning item.

[0079] Specifically, the historical learning records of each learning item of a large number of users are collected to calculate the difficulty of each learning item according to the accuracy of the answers of each historical learning item. The total number of answers of a practice item is the total number of people who have interacted with the practice item, i.e. the total number of people who have answered the practice item. Correspondingly, the number of incorrect answers of a practice item is the number of people who have not answered the practice item correctly. The total number of answers of a knowledge item is the total number of people who have answered each practice item associated with the knowledge item. The number of incorrect answers of a knowledge item is the total number of people who have not answered each practice item associated with the knowledge item correctly. Therefore, the calculation method of the difficulty of a learning item can be as follows:

[0080]

[0081] wherein M represents the total number of answers of the learning item; a m = 0 represents that the learner cannot correctly answer the learning item, i.e. the learner who answers correctly; λ p represents the preset difficulty level.

[0082] S202, generating a knowledge item embedding matrix and a practice item embedding matrix, and generating a knowledge item difficulty embedding matrix and a practice item difficulty embedding matrix according to the difficulty of each learning item.

[0083] wherein the knowledge item embedding matrix and the practice item embedding matrix are respectively used to represent all knowledge items and practice items.

[0084] ​S203, for each knowledge item, embedding the knowledge item into a combined embedding vector of the knowledge item by connecting and multiplying the item vector corresponding to the knowledge item in the item embedding matrix and the difficulty vector corresponding to the knowledge item in the knowledge item difficulty embedding matrix by a first weight matrix, and adding a first bias term; and for each exercise item, embedding the exercise item into a combined embedding vector of the exercise item by connecting and multiplying the item vector corresponding to the exercise item in the exercise item embedding matrix and the difficulty vector corresponding to the exercise item in the exercise item difficulty embedding matrix by a second weight matrix, and adding a second bias term.

[0085] Therefore, the calculation methods of the combined embedding vector of the knowledge item and the combined embedding vector of the exercise item are as follows:

[0086]

[0087] wherein, and denote the first weight matrix and the second weight matrix, respectively; and denote the item vector corresponding to the i-th knowledge item and the difficulty vector corresponding to the i-th knowledge item, respectively; and denote the item vector corresponding to the i-th exercise item and the difficulty vector corresponding to the i-th exercise item, respectively; L and P denote the first bias term and the second bias term, respectively; denotes the connection operation.

[0088] S204, for each exercise item, obtaining an item representation of the exercise item by connecting and multiplying the combined embedding vector of the exercise item and the mean of the combined embedding vectors of the adjacent knowledge items of the exercise item.

[0089] Specifically, in the embodiments of the present application, the features of the knowledge items associated with the exercise item are fused by using simplified average aggregation, which can be specifically represented as:

[0090]

[0091] wherein, denotes the exercise item e j the aggregated feature embedding, N j denotes the e j the knowledge item neighbor nodes on the hierarchical graph HG.

[0092] S103, determining the current knowledge item from the knowledge items on the item hierarchical graph, the current initial difficulty, and the current exercise limit number by the high-level intelligent agent according to the current knowledge state of the user and the number of exercises in the previous stage.

[0093] It should be noted that in order to recommend an efficient and smooth learning path for learners under consideration of item difficulty, a difficulty-driven hierarchical reinforcement learning framework is designed based on the hierarchical graph constructed. The framework includes two agents: a high-level agent responsible for selecting the next stage of knowledge to be learned at the knowledge item level, and a low-level agent responsible for selecting the next step of practice items to be practiced at the practice item level. At the same time, in order to ensure effective coordination between the two agents and achieve effective learning path construction, a communication mechanism between the two agents is implemented. Therefore, the high-level agent determines the initial difficulty of the practice item, and the low-level agent controls the maximum number of practice times of the learner according to the learning item difficulty information transmitted from the high-level agent.

[0094] Current learning path recommendation is to use a deep knowledge tracking method to only consider the association between items to evaluate the knowledge state of the learner. However, the current method ignores the comprehensive use of item difficulty. Therefore, in the embodiment of the present application, the knowledge tracking method DIMKT is introduced, so as to consider the influence of item difficulty on the cognitive changes of the learner. And the item representation of the practice item obtained in the last section is used to replace x j to represent the practice item interacting with the learner.

[0095] Therefore, the high-level agent can determine whether to learn the next knowledge item, return to the last knowledge item for learning, or continue to learn the current knowledge item according to the current knowledge state of the user, i.e., the state of the user learning the knowledge item.

[0096] In order to enable the user to start learning from the practice item with a difficulty suitable for the current learning situation, so that the learning is more efficient, in the embodiment of the present application, the difficulty of starting learning at the current stage, i.e., the current initial difficulty, is determined based on the number of practice times in the last stage.

[0097] In addition, it is considered that the learner cannot practice the same knowledge item indefinitely to master the knowledge item, because this way is counterproductive and unreasonable. When the learner still cannot master the knowledge item after multiple practices, it indicates that the learner has not effectively mastered the previous knowledge item and needs to learn again, rather than practicing the same knowledge item indefinitely. Therefore, in the embodiment of the present application, the number of practice limits is set. In addition, each learner has different learning level and speed, so the limit of the number of practice times is personalized and dynamically adjusted. Therefore, in the embodiment of the present application, the current practice limit number is determined according to the current knowledge state and the number of practice times in the last stage.

[0098] Optionally, in another embodiment of the present application, a specific implementation of step S103 is as follows:Figure 3 As shown, comprising the following steps:

[0099] S301, by high-level agent, connecting operation of the current knowledge state of the user and the encoding vector of each target knowledge item is performed to obtain the current state of the high-level agent.

[0100] Specifically, the state of the high-level agent contains the target knowledge of the learner, that is, each knowledge item to be learned by the learner, and the state at the end of the previous stage of the learner. Since the learning of the next node has not started at this time, the knowledge state at the end of the previous stage is the current knowledge state.

[0101] Optionally, the target knowledge of the user can be encoded into G = {0, 1} K , where k is the number of knowledge items; the position corresponding to the target knowledge of the user is 1, otherwise it is 0. Therefore, the calculation process of the current state of the high-level agent can be specifically represented as:

[0102]

[0103] , where h i-1 is the knowledge state at the end of the previous stage.

[0104] S302, using a proximal policy optimization algorithm, based on the current state of the high-level agent, determining the current knowledge item from the current knowledge item dynamic space.

[0105] Wherein the current knowledge item dynamic space is composed of target knowledge items on the item hierarchical graph. It is a dynamically changing dynamic knowledge action space that can dynamically adjust the knowledge items to be determined according to the learning situation.

[0106] Specifically, the action of the high-level agent refers to recommending appropriate knowledge items for the current learning stage. Specifically, the A star algorithm can be used to generate the shortest path between the starting knowledge item and the target knowledge item as a reference path. Subsequently, when generating the learning path, the real-time learning progress of the learner is adjusted. If the learning is smooth, continue along the reference path. Otherwise, search with the current knowledge item as the root node, and then learn step by step from near to far according to the distance from the current item. Whether the learning is smooth is judged by evaluating whether the learner has reached the expected mastery state threshold after completing a certain number of practice items, that is, whether h i-1 ≥ threshold. Therefore, the candidate action space D L can be dynamically determined.

[0107] After determining the current dynamic space, the selection of knowledge items in the dynamic space can be performed according to a selection strategy. In the embodiments of the present application, a proximal policy optimization algorithm is used to select the most suitable knowledge item in the action space D L as the current knowledge item.

[0108] Specifically, a policy network is used as an actor to output action probabilities from a distribution , and a value network is used as a critic to evaluate the expected return of each state. Wherein, θ L and are parameters of related networks. Specifically, the probability distribution of the knowledge item and the expected return of each state are calculated as follows:

[0109]

[0110] Wherein, FC represents a fully connected layer.

[0111] During training, the critic's value network is specifically trained using a mean square error loss. Wherein, the loss function is as follows:

[0112]

[0113] Wherein, γ i is a discount factor. When calculating the cumulative reward, it discounts the importance of future rewards, so that the rewards far from the current time have lower weights. A higher discount factor (close to 1) indicates that more emphasis is placed on long-term rewards, while a lower discount factor (close to 0) focuses more on recent rewards. represents the reward value of the L agent at t+i.

[0114] At the same time, for the training of the actor policy network, the advantage value of each step is calculated. The specific calculation method is as follows:

[0115]

[0116] Then in the training process of the actor, the data of a round is divided into multiple iterations for training. Specifically, the actor of the kth iteration is denoted as Next, the proximal policy optimization clipping loss is used for training, and the loss function is as follows:

[0117]

[0118] Wherein, clip(x,l,h)=max(min(x,h),l), which means limiting x to [l,h]. This can make the training more stable and efficient.

[0119] The future can maximize the learning benefit of the learner, thereby achieving the best learning effect, so in the embodiment of the application, the reward function adopted by the high-level intelligent agent is:

[0120]

[0121] Wherein, i is the last learning stage, and the value is E p ; i is not the last learning stage, and the value is 0. Wherein, E p The calculation method is as follows:

[0122]

[0123] Wherein, wherein E s is the test score of the learner on the target knowledge before learning, E e is the test score of the learner on the target knowledge after learning the learning path, and E sup is the full score of the test on the target knowledge.

[0124] S303, using the current knowledge state of the user and the self-defined correct answer probability, the current initial difficulty is calculated.

[0125] Since the current knowledge state reflects the user's current knowledge situation, according to which it can be determined that the next learning should start with what difficulty, which is most suitable for the user's current learning situation, avoiding unnecessary time spent on the relatively simple content that has been mastered, and avoiding too high difficulty, which cannot learn new knowledge from it. The self-defined correct answer probability can feedback the probability of correctly answering the exercise item, so it has a high reference for selecting the difficulty of the exercise item, so the current initial difficulty is calculated using the current knowledge state of the user and the self-defined correct answer probability. Specifically, the current initial difficulty calculation process can be as follows:

[0126]

[0127] Wherein, h i represents the knowledge state of the learner at the current learning stage i, P i represents the self-defined probability of the learner correctly answering the exercise item.

[0128] S304, through the trained multi-layer perception, based on the current knowledge state of the user, the number of exercises in the last stage, the current exercise limit number in the last stage, and the difficulty of the current knowledge item, the current exercise limit number is calculated.

[0129] It should be noted that, since the practice limit number of the last stage reflects the learning state of the user in the last stage, and the learning state usually does not change greatly, it will affect the current practice limit number, so in the embodiment of the present application, the current practice limit number of the last stage is also considered. The difficulty of the current knowledge item will also affect the normal practice limit number. The higher the difficulty, the more the practice limit number is generally. The difficulty of the current knowledge item can be calculated in the manner of step S201, which will not be repeated here.

[0130] Therefore, in the embodiment of the present application, the current practice limit number is calculated as follows:

[0131] τ i =f(p i-1 ,h i-1 ,τ i-1 ,DL i )

[0132] Where p i-1 and h i-1 represent the practice limit number and the knowledge state of the last learning stage, respectively. τ i-1 and DL i represent the practice limit number of the last stage and the difficulty of the current knowledge item, respectively. f is a multi-layer perceptron, which is a trainable network. It can be trained by the historical data of p i-1 , h i-1 , τ i-1 , and DL i .

[0133] S104, judge whether the user has mastered each target knowledge item.

[0134] Optionally, according to the current knowledge state of the user and the current knowledge item, it can be judged whether the user has learned and mastered all the target knowledge items.

[0135] Wherein, if it is judged that the user has mastered each target knowledge item, step S105 is executed. If it is judged that the user has not mastered each target knowledge item, step S106 is executed.

[0136] S105, end the path recommendation of each target knowledge.

[0137] S106, pass the current knowledge item, the current initial difficulty and the current practice limit number to the lower intelligent agent.

[0138] S107, selecting, by the low-level intelligent agent, a current exercise item of a corresponding difficulty from each exercise item corresponding to the current knowledge item according to the current knowledge state of the user, starting from the current initial difficulty, providing the user for learning, accumulating the current stage exercise times, and updating the current knowledge state of the user according to the item representation and learning feedback of the current exercise item, until the current stage exercise times reach the current exercise limit times or the current knowledge state of the user reaches the state threshold.

[0139] Specifically, after the low-level intelligent agent ends the transmission of the current knowledge item, the current initial difficulty and the current exercise limit times to the high-level intelligent agent, the low-level intelligent agent selects a current exercise item of a corresponding difficulty from each exercise item corresponding to the current knowledge item with the current initial difficulty as the initial difficulty, and provides the user for learning. In addition, to avoid exceeding the limit of exercise times, the current stage exercise times are accumulated, that is, the current stage exercise times start from zero from receiving the information, and the current stage exercise times are accumulated by 1 for each exercise item provided to the user. In order to adjust the difficulty of the exercise item to be selected next according to the real-time situation of the user, so that the difficulty of the recommended exercise item matches the knowledge state of the current user, and in order to know that the current user has mastered the current knowledge item in time, the current knowledge state of the user needs to be updated according to the item representation and learning feedback of the current exercise item.

[0140] Therefore, if the current stage exercise times do not reach the current exercise limit times, and the current knowledge state of the user does not reach the state threshold, the low-level intelligent agent needs to select an exercise item of the next difficulty as the current exercise item according to the current knowledge state of the user and the corresponding selection strategy, and provide the user for continuing learning, until the current stage exercise times reach the current exercise limit times, that is, the user fails to effectively master the current knowledge item, and a new current knowledge item needs to be determined again, or the current knowledge state of the user reaches the state threshold, that is, the user has mastered the current knowledge item, and the next knowledge item can be recommended for rendering. The step S106 is executed to return to the high-level intelligent agent to determine a new current knowledge item.

[0141] Optionally, in another embodiment of the present application, a specific implementation of step S107 includes the following steps as shown in Figure 4

[0142] S401, selecting, by the low-level intelligent agent, an exercise item of a difficulty of the current initial difficulty from each exercise item corresponding to the current knowledge item as a current exercise item, and providing the user for learning.

[0143] ​It should be noted that since the current initial difficulty is analyzed by the high-level agent based on the current knowledge state, the accuracy thereof can be ensured, and therefore in the embodiment of the present application, the low-level agent does not further analyze, but directly selects the exercise item of the difficulty as the current exercise item to provide to the user for learning.

[0144] S402, accumulate the current stage exercise times and update the current knowledge state of the user according to the item representation of the current exercise item and the learning feedback.

[0145] S403, judge whether the accumulated current stage exercise times reaches the current exercise limit times or the current knowledge state of the user reaches the state threshold.

[0146] If it is judged that the accumulated current stage exercise times does not reach the current exercise limit times and the current knowledge state of the user does not reach the state threshold, step S404 is executed. If it is judged that the accumulated current stage exercise times reaches the current exercise limit times or the current knowledge state of the user reaches the state threshold, step S106 is executed.

[0147] S404, connect the current knowledge state of the user with the encoding vector of the current knowledge item to obtain the current state of the low-level agent.

[0148] That is, in the embodiment of the present application, the current state of the low-level agent contains the knowledge item c i learned by the learner at the current stage and the knowledge state h t of the learner at the current time t. Similarly, the current knowledge item can also be represented by multi-hot encoding, where k is the number of all knowledge items. Similarly, the position of the current knowledge item is 1 and the rest is 0. Therefore, the current state of the low-level agent can be represented as:

[0149]

[0150] wherein, represents the connection operation.

[0151] S405, select a new current learning item from each exercise item corresponding to the current knowledge item based on the current state of the low-level agent by using the actor-critic algorithm, and provide the user for learning.

[0152] Specifically, each exercise item corresponding to the current knowledge item can be taken as an exercise action space, and a new current learning item is selected from the exercise action space by performing an exercise action on the space. Therefore, the exercise action space can be represented as:

[0153]

[0154] wherein, Ψ is a mapping function, used to obtain the knowledge item c i associated with the exercise item set.

[0155] In the embodiments of the present application, a critic algorithm framework similar to the high-level agent is used to obtain the selection strategy of the low-level agent to select the most suitable exercise item for the learner to practice. The main difference between the actor-critic algorithm and the proximal policy optimization algorithm is that the actor-critic algorithm uses a simple policy gradient method.

[0156] In order to control the difficulty change of the current exercise item, in the embodiments of the present application, the reward function of the low-level agent is as follows:

[0157] r t p1 =L(DP t ,DP t-1 )=-(DP t -DP t-1 ) 2

[0158] wherein, DP t is the difficulty of the current exercise item at time t, and DP t-1 is the difficulty of the current exercise item at the previous time.

[0159] In addition, in order to evaluate the effectiveness of the current exercise item, another part of the reward function is designed as follows:

[0160]

[0161] wherein, h i represents the knowledge state of the learner at learning stage i, and Thre is a predefined threshold.

[0162] Finally, the reward function of the low-level agent is obtained by two balance parameters α1 and α2 to obtain the final reward function, which is calculated as follows:

[0163] r t p =α1×r t p1 +α2×r t p2 ; α1, α2 ∈ [0, 1]

[0164] It should be noted that after step S405 is executed, step S402 is returned to be executed.

[0165] S108, feeding back the current latest updated current stage exercise times as the previous stage exercise times and the current latest updated current knowledge state of the user to the high-level agent through the low-level agent.

[0166] It should be noted that since the learning of the current stage has been completed, the current knowledge state of the user and the number of practice times of the current stage need to be fed back to the high-level agent to start the learning of the next stage. Therefore, after step S106 is executed, step S103 is returned to be executed.

[0167] Therefore, in summary, the framework of the learning path recommendation method provided by the embodiments of the present application can be implemented as shown in Figure 5 The leftmost in the figure is the constructed project hierarchical diagram, and the corresponding representation is calculated based on the project hierarchical diagram. And Figure 5 the right side is the specific recommendation process, as shown in Figure 5 It is divided into multiple learning stages. For each learning stage, the high-level agent (L-Agent) determines a dynamic space from the layer of knowledge projects in the project hierarchical diagram according to the knowledge state of the user, and determines the current knowledge project from the dynamic space. At the same time, the current initial difficulty and the current practice limit number are also determined, which are fed back to the low-level agent (P-Agent) together. Correspondingly, the low-level agent selects multiple practice projects and recommends them to the user for learning, and updates the knowledge state of the user and the cumulative learning number in combination with DIMKT. Finally, the low-level agent feeds back the knowledge state of the user and the cumulative learning number to the high-level agent, completing the learning of a stage.

[0168] The learning path recommendation method provided in the application implementation provides a project hierarchical graph according to the association relationship between each learning project. The learning project includes multiple knowledge projects and corresponding practice projects. The association relationship includes a predecessor-successor relationship and an examination relationship. Then, high-order information and difficulty information between the learning projects are aggregated through a hierarchical graph neural network to obtain project representations of the practice projects, so as to represent the practice projects through the representations fused with the difficulty information, and thus the difficulty of the projects is considered in path recommendation. Then, a high-level agent determines a current knowledge project from each target knowledge project on the project hierarchical graph, and determines a current initial difficulty and a current practice limit number according to a current knowledge state and a previous practice number of a user, and delivers them to a low-level agent. Next, the low-level agent selects a current practice project of a corresponding difficulty from the practice projects corresponding to the current knowledge project according to the current knowledge state of the user from the current initial difficulty, and provides the current practice project to the user for learning, and accumulates a current stage practice number and updates the current knowledge state of the user according to the project representation of the current practice project and learning feedback, until the current stage practice number reaches the current practice limit number or the current knowledge state of the user reaches a state threshold. The low-level agent feeds back the current stage practice number updated most recently and the current knowledge state of the user updated most recently to the high-level agent as the previous practice number and the current knowledge state of the user, so as to return to the high-level agent to perform the next stage of learning, so that a difficulty-driven method of recommending a knowledge project and a practice project under the knowledge project for learning is realized through the high-level agent and the low-level agent and the interaction therebetween, and thus the recommended learning path is more smooth and the learning efficiency is improved.

[0169] Another embodiment of the application provides a learning path recommendation device, as shown in the accompanying drawings, comprising: Figure 6

[0170] The construction module 601 is configured to construct a project hierarchical graph according to the association relationship between each learning project.

[0171] The learning project includes multiple knowledge projects and corresponding practice projects. The association relationship includes a predecessor-successor relationship and an examination relationship.

[0172] The aggregation module 602 is configured to aggregate high-order information and difficulty information between the learning projects through a hierarchical graph neural network to obtain project representations of the practice projects.

[0173] The high-level agent module 603 is configured to determine a current knowledge project from each target knowledge project on the project hierarchical graph, and determine a current initial difficulty and a current practice limit number according to a current knowledge state and a previous practice number of a user through a high-level agent, and deliver them to a low-level agent. ​

[0174] The low-level agent module 604 is configured to select, by the low-level agent, a current exercise item of a corresponding difficulty from each exercise item corresponding to a current knowledge item according to a current knowledge state of the user, from a current initial difficulty, provide the user with the current exercise item for learning, and accumulate a current stage exercise frequency and update the current knowledge state of the user according to the item representation of the current exercise item and learning feedback, until the current stage exercise frequency reaches a current exercise limit frequency or the current knowledge state of the user reaches a state threshold.

[0175] The low-level agent feeds back the current stage exercise frequency updated most recently and the current knowledge state of the user most recently to the high-level agent as the last stage exercise frequency and the current knowledge state of the user most recently, so as to return to execute the determination of the current knowledge item from each target knowledge item on the project hierarchical diagram, the determination of the current initial difficulty and the current exercise limit frequency by the high-level agent according to the current knowledge state of the user and the last stage exercise frequency, until the user masters each target knowledge item.

[0176] Optionally, in the learning path recommendation device provided in another embodiment of the present application, the aggregation module comprises:

[0177] The difficulty calculation module is configured to multiply, based on the historical learning record, a ratio of the number of incorrect answers to the total number of answers of each learning item by a corresponding preset difficulty level, to obtain the difficulty of each learning item.

[0178] The matrix generation module is configured to generate a knowledge item embedding matrix and an exercise item embedding matrix, and generate a knowledge item difficulty embedding matrix and an exercise item difficulty embedding matrix according to the difficulty of each learning item. The knowledge item embedding matrix and the exercise item embedding matrix are respectively used to represent all knowledge items and all exercise items.

[0179] The fusion module is configured to, for each knowledge item, multiply a connection operation result of a project vector corresponding to the knowledge item in the knowledge item embedding matrix and a difficulty vector corresponding to the knowledge item in the knowledge item difficulty embedding matrix by a first weight matrix, and add a first bias term to obtain a combined embedding vector of each knowledge item, and for each exercise item, multiply a connection operation result of a project vector corresponding to the exercise item in the exercise item embedding matrix and a difficulty vector corresponding to the exercise item in the exercise item difficulty embedding matrix by a second weight matrix, and add a second bias term to obtain a combined embedding vector of each exercise item.

[0180] The combination module is configured to, for each exercise item, perform a connection operation on a combined embedding vector of the exercise item and a mean value of combined embedding vectors of adjacent knowledge items of the exercise item, to obtain an item representation of each exercise item.

[0181] Optionally, in the learning path recommendation device provided in another embodiment of the present application, when the high-level agent module performs the following operations: determining the current knowledge item from each target knowledge item on the project hierarchical graph, and determining the current initial difficulty and the current practice limit number of times, by the high-level agent according to the current knowledge state of the user and the number of practice times in the last stage, the device is used for:

[0182] The current state of the high-level agent is obtained by connecting and operating the current knowledge state of the user and the encoding vector of each target knowledge item through the high-level agent.

[0183] The current knowledge item is determined from the current knowledge item dynamic space based on the current state of the high-level agent by using the proximal policy optimization algorithm. The current knowledge item dynamic space is composed of target knowledge items on the project hierarchical graph.

[0184] The current initial difficulty is calculated by using the current knowledge state of the user and the self-defined correct answer probability.

[0185] The current practice limit number of times is calculated based on the current knowledge state of the user, the number of practice times in the last stage, the practice limit number of times in the last stage, and the difficulty of the current knowledge item by using the trained multilayer perceptron.

[0186] Optionally, in the learning path recommendation device provided in another embodiment of the present application, the low-level agent module performs the following operations: gradually selecting the current practice item with the corresponding difficulty from each practice item corresponding to the current knowledge item according to the current knowledge state of the user from the current initial difficulty, providing the user for learning, and accumulating the number of practice times in the current stage and updating the current knowledge state of the user according to the project representation of the current practice item and the learning feedback, until the number of practice times in the current stage reaches the current practice limit number of times, or the current knowledge state of the user reaches the state threshold, and the device is used for:

[0187] The practice item with the difficulty of the current initial difficulty is selected from each practice item corresponding to the current knowledge item by the low-level agent, and the practice item is used as the current practice item to provide the user for learning.

[0188] The number of practice times in the current stage is accumulated, and the current knowledge state of the user is updated according to the project representation of the current practice item and the learning feedback.

[0189] It is judged whether the number of practice times in the current stage reaches the current practice limit number of times, or whether the current knowledge state of the user reaches the state threshold.

[0190] If it is judged that the accumulated current stage practice number does not reach the current practice limit number, and the current knowledge state of the user does not reach the state threshold, the current knowledge state of the user is connected with the encoding vector of the current knowledge item to obtain the current state of the low-level agent.

[0191] Using the actor critic algorithm, a new current learning item is selected from each practice item corresponding to the current knowledge item based on the current state of the low-level agent, and is provided to the user for learning, and the execution of the accumulated current stage practice number and the updating of the current knowledge state of the user according to the item representation of the current practice item and the learning feedback are returned.

[0192] If it is judged that the accumulated current stage practice number reaches the current practice limit number, or the current knowledge state of the user reaches the state threshold, the current latest updated current stage practice number is executed as the last stage practice number by the low-level agent, and the current latest updated current knowledge state of the user is fed back to the high-level agent.

[0193] It should be noted that the specific working process of each unit provided in the above embodiments of the present application can be correspondingly referred to the implementation process of the corresponding steps in the above method embodiments, which will not be described here.

[0194] Another embodiment of the present application provides an electronic device, as shown in the accompanying drawings, comprising: Figure 7 as shown in the accompanying drawings, comprising:

[0195] a memory 701 and a processor 702.

[0196] The memory 701 is used to store programs.

[0197] The processor 702 is used to execute the program stored in the memory 701, and when the program is executed, it is specifically used to implement the learning path recommendation method provided in any one of the above embodiments.

[0198] Another embodiment of the present application provides a computer storage medium for storing a computer program, which is executed to implement the learning path recommendation method provided in any one of the above embodiments.

[0199] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. According to the definitions provided herein, computer readable media does not include transitory computer readable medium, such as a modulated data signal and a carrier wave.

[0200] Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality, without reference to a specific

[0201] The above description of disclosed embodiments provides information sufficient to understand how to make and use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Accordingly, the application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A learning path recommendation method characterized by, The method comprises the following steps: According to the correlation between each learning project, a project hierarchical graph is constructed; wherein, the learning project comprises multiple knowledge projects and corresponding practice projects; the correlation comprises a predecessor-successor relationship and an examination relationship; High-order information and difficulty information between the learning projects are aggregated through a hierarchical graph neural network to obtain project representations of each practice project; A high-level agent determines a current knowledge project from each target knowledge project on the project hierarchical graph according to a current knowledge state of a user and a previous practice frequency, determines a current initial difficulty and a current practice limit frequency, and passes them to a low-level agent; The low-level agent selects a current practice project of a corresponding difficulty from each practice project corresponding to the current knowledge project according to the current knowledge state of the user from the current initial difficulty, provides the user for learning, and accumulates a current stage practice frequency and updates the current knowledge state of the user according to the project representation of the current practice project and a learning feedback until the current stage practice frequency reaches the current practice limit frequency or the current knowledge state of the user reaches a state threshold; The low-level agent feeds back the current latest updated current stage practice frequency and the current latest updated current knowledge state of the user to the high-level agent to return to the execution of determining a current knowledge project from each target knowledge project on the project hierarchical graph according to a current knowledge state of a user and a previous practice frequency, determining a current initial difficulty and a current practice limit frequency, and so on until the user masters each target knowledge project.

2. The method of claim 1, wherein, The method for aggregating high-order information and difficulty information between the learning projects through a hierarchical graph neural network to obtain project representations of each practice project comprises the following steps: Based on historical learning records, the ratio of the number of incorrect answers to the total number of answers for each learning project is multiplied by the corresponding preset difficulty level to obtain the difficulty of each learning project; A knowledge project embedding matrix and a practice project embedding matrix are generated, and a knowledge project difficulty embedding matrix and a practice project difficulty embedding matrix are generated according to the difficulty of each learning project; wherein, the knowledge project embedding matrix and the practice project embedding matrix are used to represent all the knowledge projects and practice projects; For each knowledge project, the project vector corresponding to the knowledge project in the knowledge project embedding matrix and the difficulty vector corresponding to the knowledge project in the knowledge project difficulty embedding matrix are connected, the operation result is multiplied by a first weight matrix, and a first bias term is added to obtain a combined embedding vector of each knowledge project; for each practice project, the project vector corresponding to the practice project in the practice project embedding matrix and the difficulty vector corresponding to the practice project in the practice project difficulty embedding matrix are connected, the operation result is multiplied by a second weight matrix, and a second bias term is added to obtain a combined embedding vector of each practice project; The connection operation result of the combined embedding vector of each exercise item and the mean of the combined embedding vectors of the adjacent knowledge items of the exercise item is obtained to obtain the item representation of each exercise item.

3. The method of claim 1, wherein, The current knowledge item is determined from each target knowledge item on the project hierarchical graph, and the current initial difficulty and the current exercise limit number are determined according to the current knowledge state of the user and the number of exercise times in the last stage by the high-level agent, which includes: The current state of the high-level agent is obtained by connecting the current knowledge state of the user and the encoding vector of each target knowledge item through the high-level agent. The current knowledge item is determined from the current knowledge item dynamic space based on the current state of the high-level agent by using a proximal policy optimization algorithm, wherein the current knowledge item dynamic space is composed of the target knowledge items on the project hierarchical graph. The current initial difficulty is calculated using the current knowledge state of the user and the self-defined correct answer probability. The current exercise limit number is calculated based on the current knowledge state of the user, the number of exercise times in the last stage, the exercise limit number in the last stage, and the difficulty of the current knowledge item by using the trained multilayer perceptron.

4. The method of claim 1, wherein, The current exercise item of the corresponding difficulty is selected from each exercise item corresponding to the current knowledge item according to the current knowledge state of the user from the current initial difficulty by the low-level agent, and is provided to the user for learning, and the current stage exercise number is accumulated and the current knowledge state of the user is updated according to the item representation and learning feedback of the current exercise item until the current stage exercise number reaches the current exercise limit number or the current knowledge state of the user reaches the state threshold, which includes: The exercise item with a difficulty of the current initial difficulty is selected from each exercise item corresponding to the current knowledge item by the low-level agent, and is provided to the user as the current exercise item for learning. The current stage exercise number is accumulated and the current knowledge state of the user is updated according to the item representation and learning feedback of the current exercise item. It is judged whether the accumulated current stage exercise number reaches the current exercise limit number or the current knowledge state of the user reaches the state threshold. If it is judged that the accumulated current stage exercise number does not reach the current exercise limit number and the current knowledge state of the user does not reach the state threshold, the current knowledge state of the user is connected with the encoding vector of the current knowledge item to obtain the current state of the low-level agent. The new current exercise item is selected from each exercise item corresponding to the current knowledge item based on the current state of the low-level agent by using the actor critic algorithm, and is provided to the user for learning, and the accumulated current stage exercise number and the current knowledge state of the user are updated according to the item representation and learning feedback of the current exercise item, and the execution is returned. If it is determined that the accumulated number of exercises in the current stage reaches the current exercise limit, or the current knowledge status of the user reaches the status threshold, the low-level agent executes the most recently updated number of exercises in the current stage as the number of exercises in the previous stage, and feeds back the most recently updated current knowledge status of the user to the high-level agent.

5. A learning path recommendation apparatus characterized by comprising: include: A construction module is used to construct a project hierarchical diagram based on the association relationship between various learning projects; wherein the learning projects include multiple knowledge projects and their corresponding exercise projects; the association relationship includes a predecessor-successor relationship and an inspection relationship; an aggregation module, configured to aggregate high-level information and difficulty information between the learning items through a hierarchical graph neural network to obtain item representations of the practice items; A high-level agent module is configured to determine, through the high-level agent, a current knowledge item from among the target knowledge items on the item hierarchy diagram based on the user's current knowledge status and the number of exercises in the previous stage, as well as a current initial difficulty and a current limit on the number of exercises, and transmit these to the low-level agent; a low-level agent module, configured to, through the low-level agent, gradually select, starting from the current initial difficulty, current practice items of corresponding difficulty from the practice items corresponding to the current knowledge item according to the current knowledge status of the user, provide the current practice items to the user for learning, accumulate the number of practice times in the current stage, and update the current knowledge status of the user according to the item representations and learning feedback of the current practice items, until the number of practice times in the current stage reaches the current practice limit or the current knowledge status of the user reaches a status threshold; The low-level agent feeds back the most recently updated number of exercises in the current stage as the number of exercises in the previous stage and the most recently updated current knowledge status of the user to the high-level agent, so as to return the execution. The high-level agent determines the current knowledge item from each target knowledge item on the item hierarchy diagram according to the user's current knowledge status and the number of exercises in the previous stage, and determines the current initial difficulty and the current limit number of exercises until the user masters each target knowledge item.

6. The apparatus of claim 5, wherein, The aggregation module includes: A difficulty calculation module is used to calculate the difficulty of each learning item by multiplying the ratio of the number of incorrect answers to the total number of answers for each learning item by the corresponding preset difficulty level based on historical learning records; a matrix generation module, configured to generate a knowledge item embedding matrix and a practice item embedding matrix, and to generate a knowledge item difficulty embedding matrix and a practice item difficulty embedding matrix according to the difficulty of each learning item; wherein the knowledge item embedding matrix and the practice item embedding matrix are respectively used to represent the full amount of the knowledge items and the practice items; a fusion module configured to, for each of the knowledge items, obtain a combined embedding vector of the knowledge item by connecting and multiplying a project vector corresponding to the knowledge item in a knowledge item embedding matrix and a difficulty vector corresponding to the knowledge item in a knowledge item difficulty embedding matrix by a first weight matrix, and adding a first bias term; and for each of the exercise items, obtain a combined embedding vector of the exercise item by connecting and multiplying a project vector corresponding to the exercise item in an exercise item embedding matrix and a difficulty vector corresponding to the exercise item in an exercise item difficulty embedding matrix by a second weight matrix, and adding a second bias term; a combination module configured to, for each of the exercise items, obtain a project representation of the exercise item by connecting and multiplying the combined embedding vector of the exercise item and a mean of combined embedding vectors of adjacent knowledge items of the exercise item.

7. The apparatus of claim 5, wherein, The high-level agent module is configured to, when determining the current knowledge item from the target knowledge items on the project hierarchical graph, determining the current initial difficulty, and determining the current exercise limit number according to the current knowledge state of the user and the number of exercises in the last stage, the high-level agent module is configured to: connect and multiply the current knowledge state of the user and the encoding vectors of the target knowledge items by the high-level agent to obtain a current state of the high-level agent; determine the current knowledge item from the current knowledge item dynamic space based on the current state of the high-level agent by using a proximal policy optimization algorithm, wherein the current knowledge item dynamic space is composed of the target knowledge items on the project hierarchical graph; calculate the current initial difficulty based on the current knowledge state of the user and the self-defined correct answer probability; calculate the current exercise limit number based on the current knowledge state of the user, the number of exercises in the last stage, the exercise limit number in the last stage, and the difficulty of the current knowledge item by using the trained multilayer perceptron.

8. The apparatus of claim 5, wherein, The low-level agent module is configured to, when selecting the current exercise item from the exercise items corresponding to the current knowledge item according to the current knowledge state of the user from the current initial difficulty, providing the current exercise item to the user for learning, accumulating the number of exercises in the current stage, and updating the current knowledge state of the user according to the project representation of the current exercise item and the learning feedback until the number of exercises in the current stage reaches the current exercise limit number or the current knowledge state of the user reaches a state threshold, the low-level agent module is configured to: select, by the low-level agent, the exercise item with a difficulty of the current initial difficulty from the exercise items corresponding to the current knowledge item as the current exercise item, and provide the current exercise item to the user for learning; accumulate the number of exercises in the current stage and update the current knowledge state of the user according to the project representation of the current exercise item and the learning feedback; determining whether the accumulated current-stage practice number reaches the current practice limit number or whether the current knowledge state of the user reaches a state threshold value; if it is determined that the accumulated current-stage practice number does not reach the current practice limit number and the current knowledge state of the user does not reach the state threshold value, performing a connection operation on the current knowledge state of the user and the encoding vector of the current knowledge item to obtain a current state of the low-level agent; using an actor-critic algorithm, selecting a new current practice item from each of the practice items corresponding to the current knowledge item based on the current state of the low-level agent, providing the new current practice item to the user for learning, and returning to perform the accumulated current-stage practice number and update the current knowledge state of the user according to the item representation of the current practice item and the learning feedback; if it is determined that the accumulated current-stage practice number reaches the current practice limit number or the current knowledge state of the user reaches the state threshold value, performing the feedback of the current latest updated current-stage practice number and the current latest updated current knowledge state of the user to the high-level agent through the low-level agent.

9. An electronic device, comprising: comprise: a memory and a processor; wherein the memory is configured to store a program; the processor is configured to execute the program, and when the program is executed, specifically configured to implement the learning path recommendation method of any one of claims 1 to 4.

10. A computer storage medium, characterized in that, a computer program is stored, and when the computer program is executed, the computer program is used to implement the learning path recommendation method of any one of claims 1 to 4.

Images