Learning content recommendation method, device, equipment and storage medium

Abstract

The present application relates to the field of cloud technology, in particular to the field of cloud education, and provides a method, apparatus, device, and storage medium for recommending learning content. The method comprises: combining a knowledge structure diagram reflecting the overall correlation between knowledge points with a learning state diagram that dynamically reflects the changes in the learning state of a target object when learning knowledge points within a set time range, extracting a target learning ability feature with stronger expression ability and more abstractness from the knowledge structure diagram, and based on the obtained target learning ability feature of the target object at each time stage, determining the target object's mastery of the knowledge system and the target object's learning ability, and recommending target learning content that meets the target object's current learning ability to the target object, thereby greatly improving the recommendation accuracy.

Description

Technical Field

[0001] The present application relates to the field of cloud technology, in particular to the field of cloud education, and provides a method, apparatus, device, and storage medium for recommending learning content. Background Art

[0002] With the continuous development of Internet technology, the Internet has gradually become one of the main channels for the public to acquire knowledge. Therefore, online education has gradually become a new trend in the development of the education industry.

[0003] In the field of online education, the following methods are commonly used to recommend appropriate learning content to target audiences:

[0004] Collect the target object's behavior data for the current knowledge point within the first time period, determine the target object's mastery of the current knowledge point by analyzing the target object behavior data, and generate and push learning content for the current knowledge point.

[0005] However, a rough analysis of the target object's peak performance, terminal performance and stability in the first time period can only determine the target object's mastery of the current knowledge point, but cannot determine the degree of influence of the current knowledge point on other related knowledge points, and thus cannot determine the target object's mastery of the entire knowledge system. In this way, when the target object is learning content with rigorous logic and strong knowledge relevance, if the above method is continued to be used to push learning content to the target object, the recommendation accuracy will be reduced and the target object's learning experience will be affected. Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, device, and storage medium for recommending learning content to solve the problem of low recommendation accuracy caused by the inability to determine the target subject's mastery of the knowledge system.

[0007] In a first aspect, an embodiment of the present application provides a method for recommending learning content, comprising:

[0008] Obtaining a knowledge structure diagram representing the overall association relationship between various knowledge points, and a schematic diagram representing the learning status of a target subject for some knowledge points in the knowledge structure diagram within a set time range;

[0009] Performing a first feature extraction based on the knowledge structure diagram and the learning state diagram to obtain target learning ability features of the target object at each time stage within the set time range;

[0010] Based on the obtained target learning ability characteristics of each target object, corresponding target learning content is recommended to the target object.

[0011] In a second aspect, an embodiment of the present application further provides a device for recommending learning content, including:

[0012] A processing unit is used to obtain a knowledge structure diagram representing the overall association relationship between various knowledge points, and a schematic diagram representing the learning status of a target subject for some knowledge points in the knowledge structure diagram within a set time range;

[0013] Performing a first feature extraction based on the knowledge structure diagram and the learning state diagram to obtain target learning ability features of the target object at each time stage within the set time range;

[0014] The recommendation unit is used to recommend corresponding target learning content to the target object based on the obtained target learning ability characteristics.

[0015] Optionally, the device further includes a model training unit, which trains the policy model by performing the following operations:

[0016] Get multiple sample data, each sample data contains the state change information of the sample object within a set time range;

[0017] Using a loop iteration method, read each sample data in turn until the iteration stop condition is met, and output the trained strategy model and value model; wherein, each time a sample data is read, the following operations are performed:

[0018] Determining a gradient of a preset value model based on the state change information of the one sample data, and adjusting corresponding model parameters based on the gradient of the value model; and

[0019] Based on the state change information of the sample data and the gradient of the value model, the gradient of the policy model is determined, and corresponding model parameters are adjusted based on the gradient of the policy model.

[0020] Optionally, the model training unit obtains sample data by performing the following operations:

[0021] Inputting the current learning state information of the sample object into the strategy model to obtain the corresponding current learning content;

[0022] Obtaining a corresponding strategy evaluation value and next learning state information after learning the current learning content by learning the current learning content;

[0023] The current learning state information of the sample object, the corresponding current learning content and strategy evaluation value, and the next learning state information of the sample object are determined as a sample data.

[0024] In a third aspect, an embodiment of the present application further provides a computer device comprising a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of any one of the above-mentioned learning content recommendation methods.

[0025] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium comprising a program code. When the program product is run on a computer device, the program code is used to enable the computer device to execute the steps of any one of the above-mentioned learning content recommendation methods.

[0026] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of any of the above-mentioned learning content recommendation methods.

[0027] The beneficial effects of this application are as follows:

[0028] An embodiment of the present application provides a method, apparatus, device and storage medium for recommending learning content, the method comprising: obtaining a knowledge structure diagram representing the overall correlation between knowledge points, and a learning status diagram representing the target object for some knowledge points in the knowledge structure diagram within a set time range; performing a first feature extraction based on the knowledge structure diagram and the learning status diagram to obtain target learning ability characteristics of the target object at each time stage within the set time range, and then recommending corresponding target learning content to the target object based on the obtained target learning ability characteristics.

[0029] The knowledge structure diagram reflects the overall correlation between various knowledge points, and the learning status diagram dynamically reflects the changes in the learning status of the target object when learning knowledge points within the set time range. The combination of dynamic and static diagrams is conducive to extracting more expressive and abstract target learning ability characteristics, so as to determine the target object's mastery of the knowledge system and the target object's learning ability, and to screen out the target learning content that meets the target object's current learning ability from a large number of learning contents, thereby improving the recommendation accuracy.

[0030] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0032] Figure 1 This is an optional schematic diagram of an application scenario in an embodiment of the present application;

[0033] Figure 2a A schematic diagram of the architecture of the online learning system provided in an embodiment of the present application;

[0034] Figure 2b A schematic diagram of a process for recommending learning content to a target object provided in an embodiment of the present application;

[0035] Figure 2c A schematic diagram of a knowledge structure diagram provided in an embodiment of the present application;

[0036] Figure 2d A schematic diagram of the learning state provided in an embodiment of the present application;

[0037] Figure 2e A flowchart of a training and learning content recommendation model provided in an embodiment of the present application;

[0038] Figure 2f A logical diagram of the training and learning content recommendation model provided in an embodiment of the present application;

[0039] Figure 3a A logical diagram of applying the above method in the field of online education to recommend learning content to users provided by an embodiment of the present application;

[0040] Figure 3b A flowchart of applying the above method in the field of online education to recommend learning content to users provided by an embodiment of the present application;

[0041] Figure 3c A logical diagram of recommending appropriate mathematics courses to first-year junior high school students provided in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a learning content recommendation device provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0044] Figure 6 This is a schematic diagram of the structure of a computing device in an embodiment of the present application. DETAILED DESCRIPTION

[0045] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of the technical solutions of this application, but not all of them. Based on the embodiments described in this application document, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the technical solutions of this application.

[0046] The following explains some of the terms used in the embodiments of the present application to facilitate understanding by those skilled in the art.

[0047] 1. Cloud technology:

[0048] The present application relates to the field of cloud technology. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network within a wide area network or a local area network to realize data calculation, storage, processing, and sharing.

[0049] Specifically, cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. that apply the cloud computing business model. The resource pool composed of cloud technology can be used on demand and is more flexible and convenient.

[0050] Cloud computing will become a crucial support for cloud technologies, primarily addressing the massive computational and storage resources required for backend services within cloud network systems, including but not limited to video and image websites, and numerous portals. With the rapid development and application of the internet, every item will likely have its own unique identifier, requiring transmission to backend systems for logical processing. Different levels of data will be handled separately, requiring robust system backend support for all types of industry data, a feat only possible through cloud computing.

[0051] 2. Cloud Computing Education (CCEDU):

[0052] Cloud education refers to an educational platform service based on a cloud computing business model. On this platform, all educational institutions, training organizations, enrollment services, publicity agencies, industry associations, regulatory bodies, industry media, and legal structures are integrated into a centralized resource pool. These resources are then displayed and interacted with one another, enabling on-demand communication and consensus building, thereby reducing educational costs and improving efficiency.

[0053] 3. Deep Reinforcement Learning

[0054] Deep learning has strong perception capabilities but lacks certain decision-making capabilities. While reinforcement learning does have decision-making capabilities, it's unable to address perception issues. Therefore, deep reinforcement learning combines the perception capabilities of deep learning with the decision-making capabilities of reinforcement learning, leveraging their complementary strengths to provide a solution to perception and decision-making issues in complex systems. Deep reinforcement learning can directly control itself based on input images, making it an AI approach closer to human thinking.

[0055] 4. Reinforcement Learning

[0056] Reinforcement learning, also known as reinforcement learning, evaluation learning, or enhanced learning, is a subfield of machine learning and is considered one of the three machine learning paradigms alongside supervised and unsupervised learning. It focuses on how an intelligent agent should act in an environment to maximize cumulative rewards.

[0057] Reinforcement learning differs from supervised learning in that there is no need to label input / output pairs and no need to explicitly correct suboptimal actions. Instead, reinforcement learning focuses on finding a balance between exploration (unknown territory) and exploitation (current knowledge).

[0058] The environment is often represented as a Markov decision process (MDP), so in this case many reinforcement learning algorithms use dynamic programming techniques. The main difference between classic dynamic programming methods and reinforcement learning algorithms is that the latter do not assume the exact mathematical model of the Markov decision process and are suitable for large-scale Markov decision processes where exact methods are not available.

[0059] 5. Markov Decision Process:

[0060] The Markov decision process is an optimal decision-making process for stochastic dynamic systems based on Markov process theory. Markov decision processes are a major research area in sequential decision making. They are a combination of Markov processes and deterministic dynamic programming, hence the name Markov-type stochastic dynamic programming. They are a branch of mathematical programming within operations research.

[0061] A Markov decision process involves a decision maker periodically or continuously observing a random dynamic system exhibiting Markov properties and making decisions sequentially. This involves selecting an action from a set of available actions based on the state observed at each moment. The system's next (future) state is random, and its state transition probabilities exhibit Markov properties. The decision maker then makes a new decision based on the newly observed state, and this process repeats itself.

[0062] 6. Graph Neural Networks

[0063] The study of graph neural networks is closely related to graph embedding or network embedding. Graph embedding aims to represent graph vertices as low-dimensional vectors by preserving the network topology and node content information of the graph, making them easy to process using simple machine learning algorithms (e.g., support vector machine classification). Graph embedding algorithms are typically unsupervised and can be roughly divided into three categories: matrix factorization, random walks, and deep learning methods. Deep learning methods for graph embedding also fall under the category of graph neural networks, including algorithms based on graph autoencoders and unsupervised graph convolutional neural networks.

[0064] The following is a brief introduction to the design concept of the embodiment of this application:

[0065] With the continuous development of Internet technology, the Internet has gradually become one of the main channels for the public to acquire knowledge. Therefore, online education has gradually become a new trend in the development of the education industry.

[0066] In the field of online education, the following methods are commonly used to recommend appropriate learning content to target audiences:

[0067] Collect the target object's behavior data for the current knowledge point within the first time period, determine the target object's mastery of the current knowledge point by analyzing the target object behavior data, and generate and push learning content for the current knowledge point.

[0068] However, a rough analysis of the target object's peak performance, terminal performance and stability in the first time period can only determine the target object's mastery of the current knowledge point, but cannot determine the degree of influence of the current knowledge point on other related knowledge points, and thus cannot determine the target object's mastery of the entire knowledge system. In this way, when the target object is learning content with rigorous logic and strong knowledge relevance, if the above method is continued to be used to push learning content to the target object, the recommendation accuracy will be reduced and the target object's learning experience will be affected.

[0069] In view of this, an embodiment of the present application provides a method for recommending learning content. The method includes: obtaining a knowledge structure diagram representing the overall association relationship between various knowledge points, and a schematic diagram representing the learning status of a target subject for some knowledge points in the knowledge structure diagram within a set time range; performing a first feature extraction based on the knowledge structure diagram and the learning status schematic diagram to obtain target learning ability characteristics of the target subject at each time stage within the set time range; and then recommending corresponding target learning content for the target subject based on the obtained target learning ability characteristics.

[0070] The knowledge structure diagram reflects the overall correlation between various knowledge points, and the learning status diagram dynamically reflects the changes in the learning status of the target object when learning knowledge points within the set time range. The combination of dynamic and static diagrams is conducive to extracting more expressive and abstract target learning ability characteristics, so as to determine the target object's mastery of the knowledge system and the target object's learning ability, and to screen out the target learning content that meets the target object's current learning ability from a large number of learning contents, thereby improving the recommendation accuracy.

[0071] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application can be combined with each other if there is no conflict.

[0072] The embodiments of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Figure 1 A schematic diagram of one application scenario is shown, which includes two physical terminal devices 110 and a server 130. The physical terminal devices 110 and the server 130 establish a communication connection using a wired or wireless communication method.

[0073] The physical terminal device 110 in the embodiment of the present application is a computer device used by the user. The computer device can be a personal computer, mobile phone, tablet computer, notebook, e-book reader, smart home device, etc., which has certain computing capabilities.

[0074] The server 130 in the embodiment of the present application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. This application does not impose any restrictions on this.

[0075] The target object learns knowledge online through the online learning interface 120 of the physical terminal device 110. At the same time, the physical terminal device uploads the learning status information of the target object within the set time range to the server 130;

[0076] The server 130 generates a corresponding learning status diagram based on the obtained learning status information, and then inputs the pre-generated knowledge structure and learning status diagram into the recurrent neural network to obtain the target learning ability characteristics of the target object at each time stage within the set time range. Thereafter, the server 130 recommends corresponding target learning content to the target object based on the obtained target learning ability characteristics, and the target object views the target learning content through the physical terminal device 110.

[0077] Figure 2a The diagram shows the architecture of an online learning system, which includes a graph neural network, a recurrent neural network, and a learning content recommendation model. The graph neural network includes a static graph neural subnetwork for generating a knowledge structure graph and a dynamic graph neural subnetwork for generating a learning state diagram. The recurrent neural network is used to extract more expressive and abstract target learning ability features from the knowledge structure graph and learning state diagram. This helps the learning content recommendation model determine the target subject's mastery of the knowledge system and their learning ability, and selects target learning content that matches the target subject's current learning ability from a wide range of learning content, thereby improving the target subject's knowledge level.

[0078] After understanding the overall architecture of the online learning system, refer to Figure 2b The flowchart shown in FIG. 1 details the process of using the system to recommend learning content to target objects.

[0079] S201: Obtain a knowledge structure diagram representing the overall association relationship between various knowledge points, and a schematic diagram representing the learning status of a target object for some knowledge points in the knowledge structure diagram within a set time range.

[0080] Input the knowledge system of the target object's learning content into the static graph neural sub-network to obtain Figure 2c As shown in the figure, the knowledge structure graph is a graph with knowledge points as nodes and the total association relationship between knowledge points as edges. The mathematical expression of the knowledge structure graph is h v =f(x v , x co[v] ), h v Is a constructed matrix, representing the knowledge structure graph, x v Represents a knowledge point, x co[v] Represents the overall association relationship between knowledge points.

[0081] As shown in Table 1, the overall association relationships in the embodiments of the present application include but are not limited to the following relationships:

[0082] Table 1

[0083]

[0084] Input the learning state information of the target object when learning some knowledge points in the knowledge structure graph within the set time range into the dynamic graph neural sub-network to obtain Figure 2d The learning state diagram is shown in Figure 1. The nodes in the learning state diagram are knowledge points, and the edges are the sub-associations between knowledge points at the target object at the ti-th time stage. By clicking on a knowledge point, the learning state information of the target object when learning that knowledge point can be obtained.

[0085] It is understandable that in the specific implementation of this application, when it involves relevant data such as the learning status information of the target object, when the embodiment of this application is applied to a specific product or technology, it is necessary to obtain the permission or consent of the target object, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0086] As shown in Table 2, the learning status information in the embodiment of the present application includes the following:

[0087] Table 2

[0088]

[0089] The mathematical expression of the learning state diagram of the target object in period T is h T =(h t0 , h t1 ,……,h ti ). Among them, h ti is the matrix of the constructed target object at the ti-th time stage, and h ti The mathematical expression is h ti =f(x ti , x co[ti] ), h ti Represents the learning state information of the target object at the ti-th time stage, x ti represents the dynamic value of the target object at the ti-th time stage, x co[ti] Represents the sub-association relationship between each knowledge point at the ti-th time stage.

[0090] S202: Perform a first feature extraction based on the knowledge structure diagram and the learning state diagram to obtain target learning ability features of the target object at each time stage within a set time range.

[0091] During the process of the target object learning knowledge points, the learning status of the target object is constantly changing, and there is a learning sequence between knowledge points. The knowledge points learned first will affect the learning effect of the subsequent knowledge points. Therefore, the embodiment of the present application introduces a recurrent neural network, and the static knowledge structure diagram is combined with the long and short cycle learning status changes in the learning status diagram to extract more expressive and abstract target learning ability characteristics, which is convenient for the subsequent determination of the target object's mastery of the knowledge system and the target object's learning ability, and personalized recommendation of corresponding target learning content for it.

[0092] Therefore, the mathematical expression of the recurrent neural network is o = F(y, θ), y = (y πt0 ,y πt1 ,……,y πti ), θ represents the network parameters of the recurrent neural network, y represents the target learning ability characteristics of the target object at each time stage within the set time range T, and y πti Represents the target learning ability characteristics of the target object at the ti-th time stage. πti =f ti (S πti-1 , h πti ), S πti-1 =g ti (S πti-2 , h πti-1 ), h πti =(h vti , h ti ), h πti represents the learning state information and sub-association information of the ti-th time stage, h vti Represents the knowledge structure diagram of the ti-th time stage, S πti-1 Represents the intermediate state of the recurrent neural network, that is, the target learning ability characteristics of the target object in the previous time stage.

[0093] Specifically, for each time period, perform the following operations:

[0094] From the learning state diagram and knowledge structure diagram, we obtain the learning state information of the target object at a time stage ti and the sub-association relationship of the corresponding knowledge points. The sub-association relationship reflects the learning sequence between knowledge points, as well as the inclusion relationship and causal relationship between knowledge points.

[0095] If the time stage ti is the first time stage, the target learning ability feature of the target object in the time stage ti is obtained by performing the first feature extraction on the learning state information and the sub-association relationship;

[0096] If the time stage ti is not the first time stage, the corresponding target learning ability feature is obtained by performing a first feature extraction on the learning state information, the sub-association relationship, and the target learning ability feature of the target object in the previous time stage.

[0097] Among them, through the recurrent neural network, the first feature extraction of the learning state information and sub-association relationship is first performed to obtain the candidate learning ability features of the target object in the time stage ti, and then the influence of the first learned knowledge points on the learning of the subsequent knowledge points is considered. The target learning ability features of the target object in the previous time stage and the candidate learning ability features of the time stage ti are fused through the recurrent neural network to obtain the target learning ability features of the target object in the time stage ti.

[0098] S203: Based on the obtained target learning ability characteristics of each target object, recommend corresponding target learning content to the target object.

[0099] The learning content recommendation model extracts the second feature of each target learning ability characteristic and determines the target learning content recommended to the target object. The learning content recommendation model can be based on any neural network model such as reinforcement learning, deep reinforcement learning, and attention mechanism, and is not limited in this embodiment of the present application.

[0100] Taking the learning content recommendation model built based on reinforcement learning as an example, let’s first introduce the training process of the model.

[0101] The learning content recommendation model includes a strategy model and a value model. The strategy model is first used to obtain the current learning content with better strategy effect under the current learning status information of the sample object, and then the value model is used to evaluate whether the strategy model selects the learning content as a good strategy for one step. In this way, after multiple rounds of auxiliary training of the value model, the model parameters of the strategy model are continuously adjusted to obtain a strategy model with accurate strategy accuracy.

[0102] See Figure 2e The flow chart and Figure 2f The following logic diagram details the training process of the learning content recommendation model:

[0103] S2031: Acquire multiple sample data, each sample data including status change information of a sample object within a set time range.

[0104] The current state information S of the sample object t (including target object data set, real-time learning information, knowledge point learning progress and corresponding assessment scores) is input into the strategy model to obtain the corresponding current learning content a t(knowledge points to be learned, training questions, examination papers, etc.), and then obtain the corresponding strategy evaluation value r by learning the current learning content t (including learning time R1, contact results R t 、Exam results R r etc., R=α1R1+α2R t +α3R r +…+αNR N , α(1~N) is a reward parameter combination), and the next learning state information S after learning the current learning content t+1 Finally, the current learning state information of the sample object, the corresponding learning content and strategy evaluation value, and the next learning state information of the sample object are determined as sample data m, and its corresponding mathematical expression is (S t , a t , r t , S t+1 ).

[0105] S2032: Read a sample data m.

[0106] S2033: Based on the state change information of the sample data m, determine the gradient of the preset value model, and adjust the corresponding model parameters based on the gradient of the value model.

[0107] First, use formulas 1 and 2 to obtain the loss value of the value model, then calculate the corresponding gradient based on the obtained loss value, and use formula 3 to update the model parameters of the value model.

[0108] y t =r i +γQ′(S t+1 ,u′(S t+1 |θ μ’ )|θ Q ) Formula 1;

[0109]

[0110] θ μ′ ←τθ μ +(1-τ)θ μ Formula 3;

[0111] Among them, y t Represents the label value, (u′(S t+1 |θ μ’ ) represents the action prediction value of the online strategy model, Q′(S t+1 ,u′(S t+1 |θ μ’ )|θ Qrepresents the strategy evaluation value of the online value model, and γ represents the discount factor used to measure future rewards. During training, the strategy evaluation value includes both the current short-term reward and the long-term reward of the final result. Therefore, this embodiment of the application also stipulates that when γ = 0, only the rewards obtained from learning the current learning content are considered; when γ = 1, all future rewards are considered when the current action is taken.

[0112] Q(s t ,a t |θ Q ) represents the strategy evaluation value of the value model to be trained, θ μ Represents the model parameters of the policy model to be trained, θ μ′ represents the model parameters of the online strategy model, and τ represents the fusion coefficient of the network model parameters.

[0113] S2034: Based on the state change information of the sample data m and the gradient of the value model, determine the gradient of the policy model, and adjust corresponding model parameters based on the gradient of the policy model.

[0114] First, use Formula 4 to determine the gradient of the policy model, and then use Formula 5 to update the model parameters of the policy model.

[0115]

[0116] θ Q′ ←τθ Q +(1-τ)θ Q Formula 5;

[0117] in, represents the gradient of the policy model, represents the gradient of the value model to be trained, represents the gradient of the strategy model to be trained, a=μ(s t ) represents the predicted value of the strategy model to be trained, θ Q Represents the model parameters of the value model to be trained, θ Q′ Represents the model parameters of the online value model.

[0118] S2035: Determine whether the iteration stop condition is met. If so, output the trained policy model and value model; otherwise, return to step 2032.

[0119] Any of the following conditions can be set as the iteration stop condition: whether all sample data have been read, whether the set round has been reached, and whether the loss values ​​of the strategy model and the value model are both lower than the set threshold.

[0120] Next, the trained strategy model is used to perform a second feature extraction on the obtained target learning ability features, and the predicted reward value of the target object when learning each candidate learning content is obtained. The candidate learning content with a predicted reward value exceeding the set threshold is determined as the target learning content recommended to the target object.

[0121] Target learning content includes, but is not limited to, audio and video materials, text materials, exercises tailored to the knowledge points learned or to be learned, and exam questions tailored to the knowledge points learned or to be learned. Both exercises and exam questions can be in some or all of the following formats: multiple-choice, fill-in-the-blank, short-answer, uploaded audio, and other formats.

[0122] See Figure 3a The logic diagram and Figure 3b The flowchart shown introduces the process of applying the above method in the field of online education to recommend learning content to users.

[0123] S301: Obtain a knowledge structure diagram representing the overall association relationship between knowledge points, and a diagram representing the user's learning status for some knowledge points in the knowledge structure diagram within a week. Each node on the learning status diagram contains the user's data set when learning the knowledge point, real-time learning information, knowledge point learning progress, and assessment score information;

[0124] S302: Input the knowledge structure diagram and the learning state diagram into a recurrent neural network, and perform first feature extraction on the learning state information and sub-association relationships for each time period from Monday to Sunday through multiple internal convolutional layers to obtain target learning ability features for each time period;

[0125] S303: Input the obtained target learning ability characteristics into a learning content recommendation model built based on reinforcement learning, and obtain target learning content that matches the user's learning ability from the knowledge learning library, training question library, and assessment question library. The learning content recommendation model is trained based on multiple sample data. Each sample data includes the current learning state information of the sample object, the corresponding learning content and strategy evaluation value, and the next learning state information of the sample object.

[0126] S304: The target learning content is obtained and pushed to the front end for display.

[0127] See Figure 3c The logical diagram shown is for a first-year junior high school student. The user's learning records for one week on the online learning system are collected. The learning records include the user's data set (specifically including the selected online learning time and learning preferences), the knowledge points learned by the user every day, the speed, playback, exit, recording of wrong questions, and other behaviors performed when learning knowledge points, as well as the corresponding assessment scores.

[0128] The learning record is uploaded to the dynamic graph neural sub-network to obtain a schematic diagram of the user's learning status within a week, and based on the static graph neural sub-network, a knowledge structure diagram of the first-year junior high school mathematics is obtained; then the knowledge structure diagram of the first-year junior high school mathematics and the schematic diagram of the user's learning status within a week are input into the recurrent neural network to obtain the user's target learning ability characteristics from Monday to Sunday. Finally, based on the trained learning content recommendation model, feature extraction is performed on each target learning ability characteristic. The analysis result shows that the user is not good at solving word problems. According to the user's mastery of knowledge structure and learning ability, the online learning system recommends online courses related to word problems, as well as exercises such as short-answer questions, fill-in-the-blank questions and multiple-choice questions to the user, so as to help the user understand the principles of problem-solving, clarify the problem-solving ideas, and overcome the knowledge difficulties as soon as possible.

[0129] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a learning content recommendation device. Figure 4 As shown in the structural diagram, the learning content recommendation device 400 may include:

[0130] Processing unit 401 is used to obtain a knowledge structure diagram representing the overall association relationship between various knowledge points, and a schematic diagram representing the learning status of a target subject for some knowledge points in the knowledge structure diagram within a set time range;

[0131] Performing a first feature extraction based on the knowledge structure diagram and the learning state diagram to obtain the target learning ability features of the target object at each time stage within the set time range;

[0132] The recommendation unit 402 is configured to recommend corresponding target learning content to the target object based on the obtained target learning ability characteristics.

[0133] Optionally, the processing unit 401 is configured to:

[0134] For each time period, perform the following operations:

[0135] From the learning state diagram and knowledge structure diagram, respectively, the learning state information of the target object in a time period and the sub-association relationship with the corresponding knowledge points are obtained;

[0136] If a time stage is the first time stage, a target learning ability feature of the target object in the time stage is obtained by performing a first feature extraction on the learning state information and the sub-association relationship;

[0137] If a time stage is not the first time stage, the corresponding target learning ability feature is obtained by performing first feature extraction on the learning state information, the sub-association relationship, and the target learning ability feature of the target object in the previous time stage.

[0138] Optionally, the processing unit 401 is configured to:

[0139] By performing first feature extraction on the learning state information and the sub-association relationship, a candidate learning ability feature of the target object in a time period is obtained;

[0140] The target learning ability characteristics of the target object in the previous time period are integrated with the candidate target learning ability of a time period to obtain the target learning ability characteristics of the target object in a time period.

[0141] Optionally, the learning status information includes: a target object data set, the target object's knowledge point learning progress and corresponding assessment score information, and the target object's real-time learning information.

[0142] Optionally, the recommendation unit 402 is configured to:

[0143] A second feature extraction is performed on each target learning ability feature obtained through a preset strategy model to obtain the predicted reward value of the target object when learning each candidate learning content;

[0144] The candidate learning content whose predicted reward value exceeds the set threshold is determined as the target learning content recommended to the target object.

[0145] Optionally, the apparatus 400 further includes a model training unit 403, which trains the policy model by performing the following operations:

[0146] Get multiple sample data, each sample data contains the state change information of the sample object within a set time range;

[0147] Using a loop iteration method, read each sample data in turn until the iteration stop condition is met, and output the trained strategy model and value model; wherein, each time a sample data is read, the following operations are performed:

[0148] Determining the gradient of a preset value model based on state change information of a sample data, and adjusting corresponding model parameters based on the gradient of the value model; and

[0149] Based on the state change information of a sample data and the gradient of the value model, the gradient of the policy model is determined, and the corresponding model parameters are adjusted based on the gradient of the policy model.

[0150] Optionally, the model training unit 403 obtains a sample data by performing the following operations:

[0151] Input the current learning state information of the sample object into the strategy model to obtain the corresponding current learning content;

[0152] By learning the current learning content, the corresponding strategy evaluation value and the next learning state information after learning the current learning content are obtained;

[0153] The current learning state information of the sample object, the corresponding current learning content and strategy evaluation value, and the next learning state information of the sample object are determined as a sample data.

[0154] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0155] After introducing the method and apparatus for accessing the service platform according to an exemplary embodiment of the present application, a computer device according to another exemplary embodiment of the present application is introduced next.

[0156] Those skilled in the art will appreciate that various aspects of the present application can be implemented as systems, methods, or program products. Therefore, various aspects of the present application can be specifically implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation that combines hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0157] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a computer device, see Figure 5 As shown, the computer device 500 may include at least a processor 501 and a memory 502. The memory 502 stores program codes, which, when executed by the processor 501, enable the processor 501 to perform the steps of any of the above-mentioned learning content recommendation methods.

[0158] In some possible implementations, the computing device according to the present application may include at least one processor and at least one memory. The memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the method for recommending learning content according to various exemplary embodiments of the present application described above in this specification. For example, the processor may execute the following steps: Figure 2b Follow the steps shown in .

[0159] Refer to the following Figure 6 hereinafter, a computing device 600 according to this embodiment of the present application is described. Figure 6 The computing device 600 is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0160] like Figure 6As shown, computing device 600 is implemented as a general-purpose computing device. Components of computing device 600 may include, but are not limited to, at least one processing unit 601, at least one storage unit 602, and a bus 603 connecting various system components (including storage unit 602 and processing unit 601).

[0161] Bus 603 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a processor or local bus using any of a variety of bus architectures.

[0162] The storage unit 602 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 6021 and / or a cache memory unit 6022 , and may further include a read-only memory (ROM) 6023 .

[0163] The storage unit 602 may also include a program / utility 6025 having a set (at least one) of program modules 6024, such program modules 6024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0164] Computing device 600 can also communicate with one or more external devices 604 (e.g., a keyboard, pointing device, etc.), one or more devices that enable a user to interact with computing device 600, and / or any device that enables computing device 600 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 605. Furthermore, computing device 600 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 606. As shown, network adapter 606 communicates with other modules of computing device 600 via bus 603. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with computing device 600, including but not limited to microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0165] Based on the same inventive concept as the above-mentioned method embodiment, various aspects of the method for accessing the service platform provided by the present application can also be implemented in the form of a program product, which includes program code. When the program code is run on a computer device, the program code is used to enable the computer device to execute the steps of the method for recommending learning content according to various exemplary embodiments of the present application described above in this specification. For example, the computer device can execute the following steps: Figure 2b Follow the steps shown in .

[0166] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0167] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0168] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for recommending learning content, characterized in that: include: Based on the static graph neural network in the graph neural network, a knowledge structure graph representing the overall association relationship between each knowledge point is obtained; and based on the dynamic graph neural network in the graph neural network, a schematic diagram representing the learning status of the target object for some knowledge points in the knowledge structure graph within a set time range is obtained; For each time stage within the set time range, respectively performing the following: obtaining the learning state information of the target object in a time stage and the sub-association relationship with the corresponding knowledge point from the learning state diagram and the knowledge structure diagram; performing a first feature extraction on the learning state information and the sub-association relationship through a recurrent neural network to obtain the target learning ability feature of the target object in the time stage; The second feature extraction is performed on each target learning ability feature obtained by the learning content recommendation model to determine the target learning content recommended to the target object.

2. The method according to claim 1, wherein Performing a first feature extraction on the learning state information and the sub-association relationship to obtain a target learning ability feature of the target object in the time period includes: If the one time period is the first time period, obtaining a target learning ability feature of the target object in the one time period by performing a first feature extraction on the learning state information and the sub-association relationship; If the time period is not the first time period, the corresponding target learning ability feature is obtained by performing a first feature extraction on the learning state information, the sub-association relationship, and the target learning ability feature of the target object in the previous time period.

3. The method according to claim 2, wherein The step of extracting a first feature from the learning state information, the sub-association relationship, and the target learning ability feature of the target object in the previous time period to obtain a corresponding target learning ability feature includes: Obtaining a candidate learning ability feature of the target object in the time period by performing a first feature extraction on the learning state information and the sub-association relationship; The target learning ability characteristics of the target object in the previous time period are integrated with the candidate target learning ability of the time period to obtain the target learning ability characteristics of the target object in the time period.

4. The method according to claim 2 or 3, wherein: The learning status information includes: a target object data set, the target object's knowledge point learning progress and corresponding assessment score information, and the target object's real-time learning information.

5. The method according to claim 1, wherein The learning content recommendation model includes a strategy model; the second feature extraction is performed on each target learning ability feature obtained by the learning content recommendation model to determine the target learning content recommended to the target object, including: Performing a second feature extraction on each target learning ability feature obtained by the strategy model to obtain a predicted reward value of the target object when learning each candidate learning content; The candidate learning content whose predicted reward value exceeds a set threshold is determined as the target learning content recommended to the target object.

6. The method according to claim 5, wherein The learning content recommendation model also includes a strategy model; the strategy model is trained by performing the following operations: Get multiple sample data, each sample data contains the state change information of the sample object within a set time range; Using a loop iteration method, read each sample data in turn until the iteration stop condition is met, and output the trained strategy model and value model; wherein, each time a sample data is read, the following operations are performed: Determining a gradient of a preset value model based on the state change information of the one sample data, and adjusting corresponding model parameters based on the gradient of the value model; and Based on the state change information of the sample data and the gradient of the value model, the gradient of the policy model is determined, and corresponding model parameters are adjusted based on the gradient of the policy model.

7. The method according to claim 6, wherein Get a sample data by doing the following: Inputting the current learning state information of the sample object into the strategy model to obtain the corresponding current learning content; Obtaining a corresponding strategy evaluation value and next learning state information after learning the current learning content by learning the current learning content; The current learning state information of the sample object, the corresponding current learning content and strategy evaluation value, and the next learning state information of the sample object are determined as a sample data.

8. A learning content recommendation device, characterized in that: include: a processing unit configured to obtain, based on a static graph neural network in the graph neural network, a knowledge structure graph representing the overall association relationship between knowledge points, and, based on a dynamic graph neural network in the graph neural network, obtain a learning status diagram representing a target object for some knowledge points in the knowledge structure graph within a set time range; For each time stage within the set time range, respectively performing the following: obtaining the learning state information of the target object in a time stage and the sub-association relationship with the corresponding knowledge point from the learning state diagram and the knowledge structure diagram; performing a first feature extraction on the learning state information and the sub-association relationship through a recurrent neural network to obtain the target learning ability feature of the target object in the time stage; The recommendation unit is used to perform second feature extraction on each target learning ability feature obtained through a learning content recommendation model, and determine the target learning content recommended to the target object.

9. The device according to claim 8, wherein The processing unit is used for: If the one time period is the first time period, obtaining a target learning ability feature of the target object in the one time period by performing a first feature extraction on the learning state information and the sub-association relationship; If the time period is not the first time period, the corresponding target learning ability feature is obtained by performing a first feature extraction on the learning state information, the sub-association relationship, and the target learning ability feature of the target object in the previous time period.

10. The device according to claim 9, wherein The processing unit is used for: Obtaining a candidate learning ability feature of the target object in the time period by performing a first feature extraction on the learning state information and the sub-association relationship; The target learning ability characteristics of the target object in the previous time period are integrated with the candidate target learning ability of the time period to obtain the target learning ability characteristics of the target object in the time period.

11. The device according to claim 9 or 10, characterized in that The learning status information includes: a target object data set, the target object's knowledge point learning progress and corresponding assessment score information, and the target object's real-time learning information.

12. The device according to claim 8, wherein The learning content recommendation model includes a strategy model; the recommendation unit is used to: Performing a second feature extraction on each target learning ability feature obtained by the strategy model to obtain a predicted reward value of the target object when learning each candidate learning content; The candidate learning content whose predicted reward value exceeds a set threshold is determined as the target learning content recommended to the target object.

13. A computer device, characterized in that: The method comprises a processor and a memory, wherein the memory stores program codes, and when the program codes are executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 7.

14. A computer-readable storage medium, characterized in that The method comprises a program code, and when the program code is run on a computer device, the program code is used to enable the computer device to execute the steps of the method according to any one of claims 1 to 7.

15. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7.

Images