A MOOC explainable recommendation method, a terminal device and a storage medium

Abstract

The application relates to a MOOC explainable recommendation method, a terminal device and a storage medium, and the method comprises the following steps: constructing a triple data set according to entities and the relationship between the entities in a learner historical course selection record; constructing a knowledge graph based on the triple data set, and performing vectorization representation on the entities and the relationship in the knowledge graph through a TransE model, wherein the learner and the course are represented through a coarse-grained course representation method and a fine-grained concept representation method respectively; constructing a learning path reasoning model based on a self-supervised reinforcement learning method, which is used for guiding a learning path reasoning of a recommendation agent from the learner to a target course on the knowledge graph; training the learning path reasoning model by adopting an actor-critic algorithm; and performing learning path reasoning between the learner and the target course through the trained learning path reasoning model. The application can not only construct explicit information and implicit feedback in the knowledge graph, but also perform MOOC explainable recommendation through deep reinforcement learning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of MOOC recommendation, and in particular to a MOOC explainable recommendation method, a terminal device and a storage medium. BACKGROUND

[0002] The existing popular explainable recommendation method is not applicable to the MOOC recommendation scenario, because the online education environment is usually subject to two complex conditions: one is the course sequence relationship. The course sequence relationship between courses is an important consideration factor for MOOC recommendation. Generally speaking, the prerequisite courses should be included in the course recommendation result, because the learner may lack the knowledge points of these prerequisite courses, and needs to master these knowledge points to deepen the understanding of the completed courses. The second is the knowledge structure of the learner. As known, the knowledge structure of the learner will evolve in the learning process. In this case, any fixed recommendation strategy is suboptimal, because the learners will rely on their learning ability to achieve different completion states of the recommended courses. Therefore, the recommendation strategy needs to consider the cognitive level of the learner. The constraints of these complex conditions lead to high difficulty challenges in the implementation of the MOOC explainable recommendation method. SUMMARY

[0003] In order to solve the above problems, the present application provides a MOOC explainable recommendation method, a terminal device and a storage medium.

[0004] The specific scheme is as follows:

[0005] A MOOC explainable recommendation method, comprising the following steps:

[0006] S1: collecting the historical course selection records of the learner, extracting the learner, the course, the course concept and the subject classification as entities from the historical course selection records, and extracting the relationship between the entities, constructing a triple data set based on the entities and the relationship between the entities;

[0007] S2: constructing a knowledge graph based on the triple data set, and performing vectorization representation on the entities and the relationship in the knowledge graph through a TransE model, wherein the embedding vector of the learner is represented by a coarse-grained course representation method, and the embedding vector of the course is represented by a fine-grained concept representation method;

[0008] S3: constructing a learning path reasoning model based on a self-supervised reinforcement learning method, for guiding the learning path reasoning of a recommendation agent from the learner to the target course on the knowledge graph;

[0009] The learning path inference model comprises an executor network and a path discriminator, the executor network first generates a path, and then the path discriminator distinguishes the expert demonstration path from the generated path, and the executor network attempts to cheat the path discriminator by imitating the expert demonstration path;

[0010] S4: training the learning path inference model by using the executor-critic algorithm, wherein the executor network learns the path inference strategy according to the value function of the critic network, and the critic network updates the value function by using the time difference method in a single step;

[0011] S5: performing learning path inference between the learner and the target course by using the trained learning path inference model.

[0012] Further, the method for representing the embedding vector of the learner by using the course representation method with coarse granularity is as follows: after sorting the historical course selection records of the learner u in chronological order, the embedding vector of the learner is represented as: wherein, represents the course selected by the learner u at time t, 1,…,t,…,t u represents the time from far to near.

[0013] Further, the embedding vector of the course is represented as c t :

[0014] c t ={(k,w)|(k i ,w j ),n>i>0,j>0}

[0015] wherein, n represents the number of course concepts contained in the course, i represents the serial number of the course concept, j represents the serial number of the word contained in the course concept, k i represents the embedding vector of the i-th course concept of the course, w j represents the embedding vector of the j-th word in the embedding vector k i , and k represents the embedding vector of all concepts in the course, and w represents the embedding vector of all words in the course.

[0016] Further, in step S3, the path inference problem is formulated as a Markov decision process, and the agent recommends the learning path of the learner to the target course by performing multi-hop path inference on the knowledge graph; in the Markov decision process, the initial state is set as s0=u, and the state at time t is According to the state s t , the agent performs the related action t to predict the feasible output edge of the entity e , and the action space wherein ε represents the entity set, Table cloud a knowledge graph; through the terminal revenue R e，T measures whether an agent generates a multi-hop path starting from a learner u and ending with a target course ; wherein, u represents a learner, r represents a relationship, e represents an entity, r t represents a relationship vector at time t, e t represents an entity vector at time t, t = {1, 2,..., t-1, t, t+1}, represents a revenue function.

[0017] Further, in the Markov decision process, the potential output edges are retained by weighting the action paths, and the weight of each edge in the path is set as:

[0018]

[0019] wherein, represents the path weight relative to the triple , V represents the vector e t of the head entity e t , r t represents the relationship vector r t between the head entity e p and the tail entity , and represents the vector of the tail entity ||.|| represents the L1 norm.

[0020] Further, the method for obtaining the expert demonstration path is: for all learners u and target courses , based on the weighted action path, the Dijkstra algorithm is used to generate the shortest path between the learner u and the target course on the weighted graph, to obtain a series of demonstration paths , and the expert demonstration path is obtained by randomly sampling from the demonstration paths .

[0021] Further, the path discriminator D p (s t , a t ) represents the action a t about the state s t at time t, and is specifically defined as:

[0022]

[0023]

[0024] wherein, is an intermediate variable, represents the state s tembedding of action a is the embedding of action a p in the discriminator D p，t , tanh(·) denotes the hyperbolic tangent function, and and are learned parameters, d a denotes the dimension of action embeddings in the actor network, d s denotes the dimension of state embeddings, and d d denotes the dimension of action embeddings in the path discriminator.

[0025] A MOOC interpretable recommendation terminal device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method of the above-mentioned embodiment of the application when executing the computer program.

[0026] A computer-readable storage medium stores a computer program, and the computer program implements the steps of the method of the above-mentioned embodiment of the application when executed by a processor.

[0027] The technical scheme disclosed in the present application can not only construct explicit information and implicit feedback in a knowledge graph, but also perform MOOC interpretable recommendation through deep reinforcement learning. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 Fig. 1 shows a flowchart of an embodiment of the present application.

[0029] Figure 2 Fig. 2 shows a schematic diagram of the overall framework of an embodiment of the present application. DETAILED DESCRIPTION

[0030] To further illustrate the embodiments, the present application provides drawings. These drawings are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principle of the embodiments in conjunction with the related description of the specification. Those skilled in the art should understand other possible implementations and advantages of the present application by referring to these contents.

[0031] The present application will be further described in conjunction with the drawings and specific embodiments.

[0032] Embodiment One

[0033] The embodiment of the present application provides a MOOC interpretable recommendation method, as shown in Figure 1 and Figure 2 , the method comprises the following steps:

[0034] S1: Collect the historical course selection records of the learners, extract the learners, courses, course concepts and subject categories as entities from the historical course selection records, and extract the relationships between the entities, and construct a triple data set based on the entities and the relationships between the entities.

[0035] S2: Construct a knowledge graph based on the triple data set, and obtain the embedding vectors of each entity by vectorizing the entities in the knowledge graph through a TransE model.

[0036] The knowledge graph can be constructed based on triple data (h, r, t), and semantic association between entities can be performed through the triple data, wherein h and t represent the head node and tail node of a relationship, or in other words, h and t represent the head entity and tail entity of a relationship, and r represents the relationship, such as <learner-enrolled-course>.

[0037] It should be noted that for the triple data in the constructed knowledge graph, if it is desired to actually apply it, the word texts therein need to be converted into digital form for subsequent calculation. Specifically, the mapping of the word texts in the triple data in the vector space can be obtained through distributed representation learning of the knowledge graph, that is, the corresponding vector of the word texts in the vector space, wherein the entity corresponds to the entity vector, and the relationship corresponds to the relationship vector. In this embodiment, the entities e i and the relationships r in the knowledge graph are vectorized by a TransE model to obtain the embedding vectors of each entity and the embedding vectors of the relationships wherein d E represents the dimension of the vector.

[0038] In this embodiment, in order to capture the temporal preference of the learners for the courses, a coarse-grained course representation method is used to model the learning behavior of the learners from the interaction sequence. In addition, a fine-grained concept representation method is also used to grasp the knowledge state of the learners, and these concepts are used as attribute-level information of the courses. In this way, the temporal preference of the learners can be combined with the attribute-level preference to better simulate the knowledge structure of the learners, while mining the potential factors in the relationship between the courses in the order of learning, because similar courses usually have some same course concepts.

[0039] (1) The learners are encoded by a coarse-grained course representation method.

[0040] In the course recommendation based on the knowledge graph, let U represent a set of learners, C represent a set of courses, and given the historical course selection records C u , the recommendation task aims to find the corresponding target course recommended learning path. To this end, the historical course-taking records of learner u can be formalized as where denotes the course c e C taken by learner u at time t u denotes the number of courses taken by the learner. Therefore, the learner can be encoded by a coarse-grained course representation method, i.e., the embedding vector of the learner is represented as: In this way, the embedding vector of the learner models the learning behavior of the learner by the coarse-grained course representation method, thereby capturing the temporal preference of the learner for courses.

[0041] Since the coarse-grained course representation method cannot understand and interpret the hidden vectors of each course, it is difficult to understand the knowledge level of the learner from the historical course-taking records. To this end, a fine-grained concept representation method is also proposed in this embodiment to address this challenge.

[0042] (2) Encode courses by a fine-grained concept representation method.

[0043] It is well known that the knowledge structure of a learner is composed of many knowledge points, including course concepts. In addition, each course contains multiple concepts, and there are often the same concepts between similar courses. In this case, we can use the semantic representation of course concepts to capture the knowledge level of the learner.

[0044] More precisely, the fine-grained concept representation method can capture the knowledge state of the learner by a series of course concepts, i.e., {k1,..., ki,..., kn}, where ki e K, i = 1,..., n, and K denotes the set of course concepts. i i denotes the embedding vector of the course concept in the completed course, which can be regarded as the attribute-level information of the completed course. Generally, the course concept embedding is composed of a series of word vectors. Formally, according to a series of concept embeddings, the course embedding can be composed of a set of vector pairs:

[0045] c t = {(k, w) | (k i , w j ), n > i > 0, j > 0}

[0046] where n denotes the number of course concepts contained in the course, i denotes the serial number of the course concept, j denotes the serial number of the word contained in the course concept, k i denotes the embedding vector of the i-th course concept of the course, w j denotes the embedding vector of the j-th word in the embedding vector k i , k denotes the embedding vector of all concepts in the course, and w denotes the embedding vector of all words in the course.

[0047] ​In particular, multiple similar courses can be associated with one or more same course concepts. Such connectivity can reveal potential factors of the prerequisite relationship between courses. For example, the course “Genetics” and its prerequisite course “Cell Biology” share some same course concepts, such as “gene”, “cell”, etc. In this case, if a learner takes the course “Genetics” or “Cell Biology”, such semantic perception can enable the multi-scale representation learning method to acquire the knowledge level and interest of the learner. Therefore, the method of the embodiment enriches the perceived information of semantic interaction in the knowledge graph, which helps the path reasoning of the MOOC recommendation.

[0048] S3: constructing a learning path reasoning model based on a self-supervised reinforcement learning method (such as the self-supervised module in the above embodiment), which is used to guide the learning path reasoning of the recommendation agent from the learner to the target course on the knowledge graph. Figure 2

[0049] In step S3, the embodiment is used to provide interpretable recommendations for learners. To this end, a self-supervised deep reinforcement learning method is proposed in the embodiment to guide the path reasoning on the knowledge graph constructed by the multi-scale representation method. Specifically, starting from a certain learner in the observed historical course selection record Cu, the recommendation agent performs multi-hop path reasoning on the knowledge graph, so that the recommended courses not only meet the knowledge structure of the learner, but also meet the constraint requirements of the course prerequisite relationship. The learning path reasoning model in the embodiment helps the recommendation agent to distinguish the strength of different paths in the knowledge graph to infer the preferences of the learner and find reasonable demonstrations to achieve accurate recommendations.

[0050] (1) Markov Decision Process

[0051] In the embodiment, the path reasoning problem is formulated as a Markov Decision Process (MDP). The agent attempts to recommend appropriate courses for learners by performing multi-hop path reasoning on the knowledge graph. Formally, the MDP can be defined as a 5-tuple where S represents the state space, A represents the action space, P represents the state transition probability, R represents the reward function in the environment, and γ represents the discount factor of the reward.

[0052] State: s t ∈ S represents the search state of the agent in the knowledge graph at time t. Here, it is assumed that the path discovery process encodes the multi-hop relationship between the learner u and the target course , i.e., the initial state s0 = u, and other states To enhance the path reasoning ability of the agent to obtain higher recommendation accuracy, we introduce course concepts as auxiliary information to increase path connectivity. ​

[0053] • Action: according to state s t , the agent performs the relevant action according to the policy to predict the entity e t The feasible output edges (except for the searched entities) are predicted. It is necessary to control the size of the action space here, because some entities have a large out-degree in the knowledge graph. Therefore, the weighted action is used in this embodiment to retain the potential output edges, so that the policy can be adjusted to infer the preferences of the learner. Formally, the action space can be defined as where ε represents a set of entities, and K represents a knowledge graph.

[0054] • Reward: represents the terminal reward, which measures whether the agent generates a multi-hop path starting from the learner u and ending with the target course Formally, the terminal reward at the last time T can be defined as where is an indicator function of path discovery, i.e., 1 when ; and 0 when .

[0055] (2) Self-supervised module

[0056] The self-supervised module in this embodiment includes two functions: one is the weighted action path, which helps the recommendation agent (i.e., the executor) to distinguish the strength of different paths in the knowledge graph to infer the preferences of the learner. The other is the path discriminator based on inverse reinforcement learning, which can obtain reasonable demonstration paths to achieve accurate recommendations. The implementation details of the two functions are described as follows.

[0057] 1) Weighted action path

[0058] Some studies assume that shorter paths are easier to explain recommendations, and then use the minimum multi-hop relationship to infer paths on the unweighted graph. However, the method does not fully exploit the dependency between entities and the overall semantics of the path, which can lead to unreasonable inferences. As an alternative method, the weighted operation based on the similarity of the relationship between entities can learn the dependency between entities and distinguish the strength of different paths. For a given arbitrary triple used to represent the head entity e t and the tail entity connected by the relationship r t , the weight of each edge in the path can be defined as follows.

[0059]

[0060] where, Relative to triples The edge weights, where V represents the vector e of the head entity. t Its relation vector r t The sum of Vector representing the tail entity The smaller the weight of each edge in the path, the stronger the dependency between the two entities on the path, because they are closer in the vector space.

[0061] Based on weighted action paths, Dijkstra's algorithm is used to generate learner u and target course on the weighted graph. The shortest path between all learners and the target course. Repeat this process to obtain a series of demonstration paths.

[0062]

[0063] in, Relative to triples The minimum edge weights are used. In this way, the recommending agent uses weighted action paths to adjust its policy to effectively infer learner preferences, because the path weights can explore the overall semantics of the path in the observed interactions.

[0064] 2) Path discriminator based on inverse reinforcement learning

[0065] This embodiment employs generative adversarial imitation learning to obtain reasonable demonstration paths that conform to predefined meta-paths. It uses expert demonstration paths and reward signals to incentivize the strategy for accurate recommendations. In this way, the recommendation agent can recommend courses that match the learner's knowledge level and interests, while simultaneously enhancing its reasoning ability.

[0066] Specifically, the executor network uses an adversarial approach with the path discriminator D. p The collaboration involves the actor network first generating paths, then a path discriminator distinguishing the expert demonstration paths from the generated paths, while the actor network attempts to deceive the path discriminator by mimicking the expert demonstration paths. Formally, the path discriminator D... p (s t a t ) represents the state s t Action a at time t t Specifically, it can be defined as follows.

[0067]

[0068]

[0069] in, is an intermediate variable, represents the state s t , is an embedding vector of action a p in discriminator D p，t , tanh(·) represents the hyperbolic tangent function, and σ(·) represents the logistic sigmoid function, and are learned parameters, d a represents the dimension of the action embedding in the actor network, d s represents the dimension of the state embedding, and d d represents the dimension of the action embedding in the path discriminator.

[0070] The training of path discriminator D p (s t , a t ) is used to calculate the probability that (s t , a t ) comes from the observed demonstration path. Generally, this can be achieved by minimizing the following classification loss function :

[0071]

[0072] where the action and the state are determined by the expert demonstration path, and the expert demonstration path is randomly sampled from the observed demonstration path .

[0073] When the actor network generates (s t , a t ) similar to the observed demonstration path, the path discriminator's reward R p，t can be obtained, as shown below.

[0074] R p，t = log D p (s t , a t ) - log(1-D p (s t , a t ))

[0075] In order to smoothly update the policy to find an approximation of the observed demonstration path, we define the aggregated reward R t by a linear combination of the path discovery and the path discriminator's reward.

[0076] R t = λR e，T + (1-λ)R p，t

[0077] where λ ∈ [0, 1] is the proportion of the reward R of equilibrium path finding e，T and the reward R of path discriminator p，t .

[0078] S4: training the learning path inference model by using the actor-critic algorithm, wherein the actor network learns the path inference strategy according to the value function of the critic network, and the critic network uses the time-difference method to update the value function in a single step.

[0079] (1) Actor: the actor network aims to learn the path inference strategy by calculating the probability distribution of each action in the state st. It uses the weighted action path and the expert path discriminator to effectively guide the path inference. In this embodiment, the actor network π θ (a t , s t ) is trained by using a multi-layer fully connected neural network:

[0080] h θ = ReLU(W θ，s s t )

[0081]

[0082] where ReLU(·) represents an activation function, represents the embedding vector of the action a t in the actor network, and are the parameters of the actor network to be learned, d h represents the dimension of the hidden layer, d s represents the dimension of the state embedding, and d a represents the dimension of the action embedding. Here, the actor network is optimized by the policy gradient method. For each sampled trajectory, the gradient can be calculated as follows.

[0083]

[0084] where the symbol ∝ represents “proportional to”, Q φ (s t , a t ) represents the action value function of the action a t in the state s t . Therefore, we can learn the actor network by minimizing the loss function, as shown below.

[0085]

[0086] where This indicates adherence to the executor network policy π. θ The expected values ​​of the variables given below.

[0087] (2) Critics: The critic network estimates the action value function to evaluate each action in the MDP environment. It can model the gains of path discovery and path discriminators to effectively guide the actor network. The critic network computes states s t The action value Q φ :

[0088] h φ =ReLU(W φ，s s t )

[0089] Q φ (s t a t ) = a φ，t ReLU(W φ，a h φ )

[0090] in Indicates action a in the critic network t Embedded vector, and These are the network parameters of the commentators that need to be studied.

[0091] The critic network is trained using a temporal difference method, which updates the target q step-by-step according to the Bellman equation. t As shown below.

[0092]

[0093] Where β∈[0,1] is the action-value function Q. φ (s t+1 The decay factor of a). Therefore, the critic network can be learned by minimizing the time difference error:

[0094]

[0095] By minimizing the total loss function, we jointly optimize the path discriminator D. p (s t a t ), Executor Network π θ And the critics' network Q φ Therefore, the objective function of the learning path reasoning model can be defined as follows:

[0096]

[0097] S5: inferring the learning path between the learner and the target course by the trained learning path inference model.

[0098] The embodiment adopts a multi-scale representation learning method to enhance the semantic representation and relationship of the knowledge graph for the knowledge graph environment. More specifically, the coarse-grained course representation simulates the learning behavior of the learner through user course interaction; the fine-grained concept representation can capture the knowledge state of the learner, i.e., a series of course concepts {k1,..., k i}, which are regarded as attribute-level information of the completed courses. In this way, the potential relationship between courses can be well learned.

[0099] The embodiment recommends target courses that match the knowledge level and interest of the learner by a self-supervised reinforcement learning method. The recommendation agent starts from a learner, performs multi-hop path inference on the knowledge graph, and finally recommends appropriate courses in the knowledge graph to the learner. The self-supervised module in the method includes two functions: the path discriminator based on inverse reinforcement learning can obtain reasonable demonstration paths to achieve accurate recommendation. In addition, the weighted action path can help the recommendation agent to distinguish the strength of different paths in the knowledge graph to infer the preference of the learner.

[0100] The embodiment uses the actor-critic algorithm to train the learning path inference model. It uses the reward signal (i.e., a reward R e,T for path discovery, and another reward R p,t for path discrimination) to motivate the policy evaluation of the path inference of the MOOC recommendation.

[0101] To sum up, the embodiment can not only construct explicit information (such as the learning behavior of the learner) and implicit feedback (such as the knowledge level of the learner) in the knowledge graph, but also perform interpretable MOOC recommendation through deep reinforcement learning.

[0102] Embodiment two:

[0103] The application also provides a MOOC interpretable recommendation terminal device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the above method embodiments of the first embodiment of the application when executing the computer program.

[0104] Further, as an executable solution, the MOOC interpretable recommendation terminal device can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The MOOC interpretable recommendation terminal device can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned components of the MOOC interpretable recommendation terminal device are only examples of the MOOC interpretable recommendation terminal device, and do not constitute a limitation on the MOOC interpretable recommendation terminal device, and can include more or fewer components than the above, or combine certain components, or different components, for example, the MOOC interpretable recommendation terminal device can also include an input / output device, a network access device, a bus and the like, and the embodiments of the present application do not limit this.

[0105] Further, as an executable solution, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor and the like. The processor is a control center of the MOOC interpretable recommendation terminal device, and connects various parts of the MOOC interpretable recommendation terminal device through various interfaces and lines.

[0106] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the MOOC interpretable recommendation terminal device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; the data storage area can store data created according to the use of the mobile phone and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0107] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the method of the above-mentioned embodiments of the application.

[0108] The module / unit for integrating the MOOC explainable recommendation terminal device can be stored in a computer readable storage medium if it is implemented in the form of a software function unit and sold or used as an independent product. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when being executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, an executable file, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), and a software distribution medium, etc.

[0109] Although the application is specifically shown and described in connection with the preferred embodiments, those skilled in the art should understand that various changes in form and details can be made to the application without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. A MOOC interpretable recommendation method, characterized in that, The method comprises the following steps: S1: collecting historical course selection records of learners, extracting learners, courses, course concepts and subject categories as entities from the historical course selection records, and extracting relationships between the entities, and constructing a triple data set based on the entities and the relationships between the entities; S2: constructing a knowledge graph based on the triple data set, and performing vectorization representation on the entities and the relationships in the knowledge graph through a TransE model, wherein the embedding vector of the learner is represented through a coarse-grained course representation method, and the embedding vector of the course is represented through a fine-grained concept representation method; S3: constructing a learning path reasoning model based on a self-supervised reinforcement learning method, which is used to guide a recommended agent to perform learning path reasoning from the learner to a target course on the knowledge graph; The learning path reasoning model comprises an actor network and a path discriminator, the actor network first generates a path, and then the path discriminator distinguishes the expert demonstration path from the generated path, and the actor network tries to cheat the path discriminator by imitating the expert demonstration path; S4: training the learning path reasoning model by using an actor-critic algorithm, wherein the actor network learns a path reasoning strategy according to a value function of the critic network, and the critic network uses a time difference method to update the value function step by step; S5: performing learning path reasoning between the learner and the target course through the trained learning path reasoning model; The method for representing the embedding vector of the learner by the coarse-grained course representation method is: after the historical course selection record of the learner u is sorted in time sequence, the embedding vector of the learner is represented as: Wherein, represents the course selected by the learner u at time t, 1,…,t,…,t u represents the time from far to near; The method for representing the embedding vector of the course through the fine-grained concept representation method is: representing the embedding vector of the course as c t : c t = {(k, w) | (k i ,w j ), n > 0, j > 0} where n represents the number of course concepts included in the course, i represents the serial number of the course concept, j represents the serial number of the word included in the course concept, k i represents the embedding vector of the i-th course concept of the course, w j represents the embedding vector of the j-th word in the k i -th course concept, k represents the embedding vector of all concepts in the course, and w represents the embedding vector of all words in the course; Path discriminator D p (s t , a t ) denotes the state s t at time t of the action a t , defined as: wherein, is an intermediate variable, denotes the state s t embedding vector, is the action a p embedding vector in the discriminator D p,t , tanh(·) denotes the hyperbolic tangent function, and σ(·) denotes the logistic sigmoid function, and are learned parameters, d a denotes the dimension of the action embedding in the actor network, d s denotes the dimension of the state embedding, and d d denotes the dimension of the action embedding in the path discriminator. 2.The MOOC explainable recommendation method of claim 1, wherein: In step S3, the path reasoning problem is expressed as a Markov decision process, and the agent recommends a learning path from the learner to the target course for the learner by performing multi-hop path reasoning on the knowledge graph; In the Markov decision process, the initial state is set as s0=u, and the state at time t is According to the state s t , the agent performs the relevant action to predict the feasible output edge of the entity e t , the action space where ε represents the entity set, represents a knowledge graph; the terminal revenue R e,T measures whether the agent generates a multi-hop path starting from the learner u and ending with the target course ; wherein u represents the learner, r represents the relationship, e represents the entity, r t represents the relationship vector at time t, e t represents the entity vector at time t, t={1,2,…,t-1,t,t+1}, represents the revenue function. 3.The MOOC interpretable recommendation method of claim 1, wherein: In the Markov decision process, potential output edges are retained by using weighted action paths, and the weight of each edge in the path is set as: wherein, denotes the path weight of the triple , V denotes the vector e t of the head entity, t the relationship vector r t between the head entity e t and the tail entity, denotes the sum of the relationship vectors r of the tail entity, ‖.‖ denotes the L1 norm.

4. The MOOC interpretable recommendation method of claim 1, wherein: The method for obtaining the expert demonstration path is: for all learners u and target courses Based on the weighted action path, the Dijkstra algorithm is used to generate the shortest path between the learner u and the target course on the weighted graph, and a series of demonstration paths are obtained The expert demonstration path is randomly sampled from the demonstration path .​ 5.A MOOC interpretable recommendation terminal device, characterized by comprising: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1-4.

6. A computer readable storage medium storing a computer program, characterized in that: The computer program is executed by the processor to implement the steps of the method according to any one of claims 1-4.

Images