Graph attention reinforcement learning path recommendation method, system and device based on large language model embedding and storage medium

Abstract

The invention relates to the technical field of data mining recommendation, and particularly discloses a graph attention reinforcement learning path recommendation method, system and device based on large language model embedding and a storage medium. Performing large language model embedding representation; pre-training the DKT model; constructing a data knowledge graph according to the embedded representation of the large language model and the original prerequisite relationship of the data set; using a graph attention algorithm to update graph content, using multi-head attention to stabilize a learning process, and updating knowledge point embedding representation; and sequentially recommending a learning path for the student through the obtained knowledge point embedded representation in combination with the historical learning information of the student. According to the method, the large language model is introduced to perform embedded representation on the knowledge points, the graph attention algorithm and the reinforcement learning method are added, the correlation between the knowledge points is mined, and a learning path can be quickly and reasonably recommended for students.

Description

Technical Field

[0001] The present invention relates to the field of data mining recommendation technology, and in particular to a method, system, device and storage medium for recommending learning paths based on graph attention reinforcement learning embedded in a large language model. Background Art

[0002] With the rapid development of Internet technology, more and more learning resources are widely disseminated on the Internet, and online education systems have emerged. Online education provides learners with the opportunity to learn anytime and anywhere, and also provides personalized learning and training programs for different students, which has an extremely important impact on the training process of students.

[0003] However, with the explosive growth of knowledge and learning resources, how to recommend personalized learning resources to learners based on differences in their learning abilities, knowledge backgrounds, learning goals, etc., and tailor a learning path for learners that can achieve their learning goals is a new challenge at present. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a learning path recommendation method, system, device and storage medium for graph attention reinforcement learning based on large language model embedding, which can quickly and reasonably recommend learning paths for students and solve the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a learning path recommendation method for graph attention reinforcement learning based on large language model embedding, comprising the following steps:

[0006] S1, data preprocessing;

[0007] S2, using a large language model to embed representation based on the knowledge point name;

[0008] S3. Pre-train the DKT model based on the pre-processed data;

[0009] S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0010] S5. Based on the two knowledge graphs, the graph attention algorithm is used to obtain the embedded representations of the updated knowledge points and accumulate them;

[0011] S6. The knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0012] Preferably, in step S1, data preprocessing includes data segmentation, data cleaning and data merging; the data segmentation refers to segmenting the data according to different types, extracting the knowledge point names in file 1, and extracting the user ID, knowledge point name and student answer status in file 2; the data cleaning includes name deduplication, illegal character removal, data missing and data digitization processing; the data merging refers to splicing the cleaned data to form input data that can be input into the model.

[0013] Preferably, in step S4, the following is specifically included:

[0014] S41, based on the knowledge point embedding representation generated by the large language model, calculate the cosine similarity of the embedding representation of each knowledge point, and if the similarity exceeds 0.4, it is considered that there is a similar relationship, and the IDs of the two knowledge points with a similar relationship are recorded as a pair, which is recorded as a similar relationship pair;

[0015] S42, according to the knowledge point prerequisite relationship provided by the data set, recording the IDs of two knowledge points with a prerequisite relationship as a pair, recorded as a prerequisite relationship pair;

[0016] S43. Call the machine learning algorithm library torch_geometric, use the knowledge point ID as the node, similarity relationship pairs and prerequisite relationship pairs as the edges, and construct the similarity relationship knowledge graph and the prerequisite relationship knowledge graph respectively.

[0017] Preferably, in step S5, the embedded representation of each knowledge point is passed into the graph attention model for linear transformation, and then the attention scores between each pair of knowledge points are calculated and softmax operation is performed on them. After that, the neighbor node information is aggregated. At the same time, a multi-head attention mechanism is used to stabilize the attention learning process, and finally the final embedded representation of each knowledge point is output. Specifically, it includes the following:

[0018] S51, attention learning layer: According to the nodes in the two generated knowledge graphs, the embedded representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output i-th node linearly transformed embedding representation z is i As shown below:

[0019] z i =Wh i

[0020] Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix;

[0021] S52. Calculate the original attention score between each pair of nodes: According to the relationship between the edges in the two generated knowledge graph data, find the nodes at both ends of an edge. First, concatenate the embedding representations of the two nodes, then do a dot product calculation on the concatenated embedding and a learnable weight, and finally activate it using the LeakyReLU function to output the original attention score e between each pair of nodes. ij As shown below:

[0022] e ij =LeakyReLU(a T (z i ||z j ))

[0023] Among them, z i and z j They represent the embedded representations of two nodes after linear transformation, || here represents concatenation, a T is a learnable weight vector;

[0024] S53. Perform Softmax operation on the original attention scores of all incoming edges of a node according to the two generated knowledge graphs to obtain the attention weight a ij As shown below:

[0025]

[0026] Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N(i) exp(e ik ) means to add the original attention scores of each pair of nodes;

[0027] S54. According to the attention weight, aggregate the features of all neighbor nodes to obtain the aggregated final embedding representation h′ i As shown below:

[0028]

[0029] Among them, σ() is the activation function;

[0030] S55. Use multi-head attention mechanism to stabilize the learning process of attention:

[0031]

[0032] Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads of the multi-head attention mechanism.

[0033] Preferably, in step S6, the following is specifically included:

[0034] S61. Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation;

[0035] S62, randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding according to the knowledge point final embedding obtained in step S5;

[0036] S63, combining the student hidden state, combining the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron, and outputting a recommended item;

[0037] S64, putting the recommended items into the DKT model pre-trained in step S3 for testing and giving a score;

[0038] S65. Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete a project recommendation. The formula is as follows:

[0039] LOSS=-E t ∑log P

[0040] Among them, E t represents the score improvement, and P represents the probability of the project being recommended;

[0041] S66. Repeat the above steps to generate recommendation items in sequence and generate a learning path.

[0042] On the other hand, to achieve the above purpose, the present invention also provides the following technical solution: a graph attention reinforcement learning learning path recommendation system based on large language model embedding, the system includes the following modules:

[0043] Data preprocessing module;

[0044] The large language model embedding representation module uses the large language model to embed representation according to the knowledge point name;

[0045] DKT model pre-training module, pre-training the DKT model based on pre-processed data;

[0046] The knowledge graph construction module builds two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0047] The graph attention module uses the graph attention algorithm to obtain and accumulate the embedded representations of updated knowledge points based on the two knowledge graphs;

[0048] The recommendation module finally embeds the obtained knowledge points and recommends them in sequence by combining the reinforcement learning method to generate a learning path.

[0049] On the other hand, to achieve the above-mentioned purpose, the present invention also provides the following technical solution: an electronic device, the electronic device comprising: a processor; and a memory for storing one or more programs;

[0050] When the one or more programs are executed by the processor, the processor executes the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0051] On the other hand, to achieve the above-mentioned purpose, the present invention also provides the following technical solution: a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the learning path recommendation method of graph attention reinforcement learning based on large language model embedding is implemented.

[0052] The beneficial effects of the present invention are:

[0053] 1) Solution for obtaining knowledge point relevance: For different knowledge points in the data set, in the absence of other data, the present invention obtains the implicit relevance information between knowledge points through the embedding representation generation method of the large language model, so that the student's knowledge status and knowledge points are connected;

[0054] 2) A new method for learning path recommendation: Compared with previous learning path recommendation methods, the present invention incorporates the similarity and prerequisite relationships of knowledge points into each recommendation item through the graph attention algorithm and reinforcement learning rewards, and uses the pre-trained DKT test model to evaluate the quality of the path, making the learning path recommendation more accurate and reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flowchart of a method for recommending learning paths for graph attention reinforcement learning based on large language model embedding in an embodiment of the present invention;

[0056] Figure 2 A schematic diagram of the architecture of a learning path recommendation method for graph attention reinforcement learning based on large language model embedding in an embodiment of the present invention;

[0057] Figure 3 It is a schematic diagram of a learning path recommendation system module based on graph attention reinforcement learning embedded in a large language model in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention;

[0059] In the figure, 110-data preprocessing module; 120-large language model embedding representation module; 130-DKT model pre-training module; 140-knowledge graph construction module; 150-graph attention module; 160-recommendation module; 210-processor; 220-storage. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] See also Figure 1 and Figure 2 The present invention provides a technical solution: a learning path recommendation method for graph attention reinforcement learning based on large language model embedding, comprising the following steps:

[0062] S1. Data preprocessing: The Junyi dataset contains 26 million answers from 191,874 students and 712 valid knowledge points. In order to make better recommendations later, the following operations need to be performed on the data:

[0063] The data is segmented according to different types, and the knowledge point names are extracted in file 1, and the user ID, knowledge point name and student answer status are extracted in file 2;

[0064] Data cleaning includes name deduplication, illegal character removal, data missing, and data digitization;

[0065] De-duplicate knowledge point names: De-duplicate the knowledge point names in file 1 and add serial numbers to non-duplicate items.

[0066] Illegal character removal: Delete illegal characters that appear in the knowledge point names in file 1.

[0067] Data missing processing: Delete the data items with empty user ID and knowledge point name in file 2.

[0068] Data digitization processing: Using the character replacement method, the knowledge point names in file 2 are replaced with numbers in sequence according to the knowledge point names and serial numbers in file 1.

[0069] Data merging refers to the process of combining cleaned data to form input data that can be input into the model.

[0070] Group file 2 by user ID, extract the knowledge point number and student answer status of the same user ID into two rows respectively. Delete the user data with less than 2 knowledge point numbers and save the rest of the data.

[0071] S2, using a large language model to embed representation based on the knowledge point name;

[0072] In order to extract the relevance between knowledge points, the open source large language model Llama-2-7B is used to embed each knowledge point and extract the hidden information of the knowledge point, including environment deployment and embedding output, including:

[0073] 2-1) Configure the operating environment required by the Llama-2-7B large language model locally on the computer, including GPU driver, Python language environment, and run the large language model;

[0074] 2-2) Extract the knowledge point names in file 1, input them into the Llama-2-7B model in sequence according to the following input format, generate and save the embedded representation of each knowledge point in turn;

[0075] ###Instruction: Given knowledge concept in order, output the embedding of each concept.

[0076] ###Input:{knowledge concept}

[0077] ###Response:{Embedding Vector}.

[0078] S3. Pre-train the DKT model based on the pre-processed data;

[0079] In order to evaluate the rationality of the learning path, a pre-trained knowledge tracking DKT model is required to simulate the student learning process, including environment deployment, model training and testing:

[0080] 3-1) Configure the operating environment required by the DKT model locally and deploy the DKT model;

[0081] 3-2) Input the preprocessed data of file 1 into the DKT model in sequence for training and testing;

[0082] 3-3) After training is completed, save the trained DKT model parameters.

[0083] S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0084] The details include:

[0085] 4-1) Based on the knowledge point embedding representation generated by the large language model, the cosine similarity of the embedding representation of each knowledge point is calculated. If the similarity exceeds 0.4, it is considered that there is a similar relationship. The IDs of two knowledge points with similar relationships are recorded as a pair, which is recorded as a similar relationship pair;

[0086] 4-2) According to the prerequisite relationship of the knowledge points provided by the data set, the IDs of two knowledge points with a prerequisite relationship are recorded as a pair, which is recorded as a prerequisite relationship pair;

[0087] 4-3) Call the machine learning algorithm library torch_geometric, use the above knowledge point IDs as nodes, similarity relationship pairs and prerequisite relationship pairs as edges, and construct similarity relationship knowledge graphs and prerequisite relationship knowledge graphs respectively.

[0088] S5. Based on the two knowledge graphs, the graph attention algorithm is used to obtain the embedded representations of the updated knowledge points and accumulate them;

[0089] 5-1) Attention learning layer: According to the nodes in the two generated knowledge graphs, the embedding representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output embedding representation z of the i-th node after linear transformation is i As shown below:

[0090] z i =Wh i

[0091] Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix;

[0092] 5-2) Calculate the original attention score between each pair of nodes: According to the relationship between the edges in the two generated knowledge graph data, find the nodes at both ends of an edge. First, concatenate the embedding representations of the two nodes, then do a dot product calculation on the concatenated embedding and a learnable weight, and finally use the LeakyReLU function to activate and output the original attention score e between each pair of nodes. ij As shown below:

[0093] e ij =LeakyReLU(a T (z i ||z j ))

[0094] Among them, z i and z j They represent the embedded representations of two nodes after linear transformation, || here represents concatenation, a T is a learnable weight vector;

[0095] 5-3) According to the two generated knowledge graphs, perform Softmax operation on the original attention scores of all incoming edges of a node to obtain the attention weight a ij As shown below:

[0096]

[0097] Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N (i)exp(e ik ) means to add the original attention scores of each pair of nodes;

[0098] 5-4) According to the attention weight, the features of all neighbor nodes are aggregated to obtain the final aggregated embedding representation h′ i As shown below:

[0099]

[0100] Among them, σ() is the activation function;

[0101] 5-5) Use the multi-head attention mechanism to stabilize the learning process of attention:

[0102]

[0103] Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads of the multi-head attention mechanism.

[0104] S6. The knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0105] The details include:

[0106] 6-1) Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation;

[0107] 6-2) Randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding according to the final embedding of the knowledge point obtained in step S5;

[0108] 6-3) Combine the student's hidden state, combine the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron, and output a recommended item;

[0109] 6-4) Put the recommended items into the DKT model pre-trained in step S3 for testing and give a score;

[0110] 6-5) Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete an item recommendation. The formula is as follows:

[0111] LOSS=-E t ∑logP

[0112] Among them, E t represents the score improvement, and P represents the probability of the project being recommended;

[0113] 6-6) Repeat the above steps to generate recommended items in sequence and generate a learning path.

[0114] The present invention introduces a large language model to embed and represent knowledge points, adds a graph attention algorithm and a reinforcement learning method, mines the correlation between knowledge points, and can quickly and reasonably recommend learning paths for students.

[0115] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides a learning path recommendation system based on graph attention reinforcement learning embedded in a large language model, which can implement the functions provided by the above method embodiment, such as Figure 3 As shown, the system includes the following modules:

[0116] Data preprocessing module 110;

[0117] A large language model embedding representation module 120 performs embedding representation using a large language model according to the knowledge point name;

[0118] DKT model pre-training module 130, pre-training the DKT model based on the pre-processed data;

[0119] The knowledge graph construction module 140 constructs two knowledge graphs according to the embedding representation of the large language model and the prerequisite relationship of the knowledge points;

[0120] The graph attention module 150 uses a graph attention algorithm to obtain and accumulate the embedded representations of updated knowledge points according to the two knowledge graphs;

[0121] The recommendation module 160 finally embeds the obtained knowledge points into representations, recommends them in sequence in combination with the reinforcement learning method, and generates a learning path.

[0122] Based on the same inventive concept as the above method embodiment, the present application embodiment also provides an electronic device, such as Figure 4 As shown, the device includes: a processor 210; and a memory 220, for storing one or more programs;

[0123] When the one or more programs are executed by the processor 210, the processor executes the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0124] The learning path recommendation method of graph attention reinforcement learning based on large language model embedding specifically includes the following:

[0125] Data preprocessing;

[0126] Use a large language model to embed the knowledge point name;

[0127] Pre-train the DKT model based on the pre-processed data;

[0128] Based on the embedding representation of the large language model and the prerequisite relationships of knowledge points, two knowledge graphs are constructed;

[0129] Based on the two knowledge graphs, the graph attention algorithm is used to obtain the updated embedding representations of the knowledge points and accumulate them;

[0130] The obtained knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0131] Based on the same inventive concept as the above-mentioned method embodiment, the embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 210, the learning path recommendation method of graph attention reinforcement learning based on large language model embedding is implemented.

[0132] The learning path recommendation method of graph attention reinforcement learning based on large language model embedding specifically includes the following:

[0133] Data preprocessing;

[0134] Use a large language model to embed the knowledge point name;

[0135] Pre-train the DKT model based on the pre-processed data;

[0136] Based on the embedding representation of the large language model and the prerequisite relationships of knowledge points, two knowledge graphs are constructed;

[0137] Based on the two knowledge graphs, the graph attention algorithm is used to obtain the updated embedding representations of the knowledge points and accumulate them;

[0138] The obtained knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0139] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0141] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention is essentially or partly contributed to the prior art or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0142] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0143] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0144] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0145] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0146] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A learning path recommendation method based on graph attention reinforcement learning with large language model embedding, characterized in that: The steps include: S1, data preprocessing; S2, using a large language model to embed representation based on the knowledge point name; S3. Pre-train the DKT model based on the pre-processed data; S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points; S5. Based on the two knowledge graphs, the graph attention algorithm is used to obtain the embedded representations of the updated knowledge points and accumulate them; S6. The knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

2. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized in that: In step S1, data preprocessing includes data segmentation, data cleaning and data merging; the data segmentation refers to segmenting the data according to different types, extracting the knowledge point names in file 1, and extracting the user ID, knowledge point name and student answer status in file 2; the data cleaning includes name deduplication, illegal character removal, data missing and data digitization processing; the data merging refers to the splicing of the cleaned data to form input data that can be input into the model.

3. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized in that: In step S4, the specific steps include: S41, based on the knowledge point embedding representation generated by the large language model, calculate the cosine similarity of the embedding representation of each knowledge point, and if the similarity exceeds 0.4, it is considered that there is a similar relationship, and the IDs of the two knowledge points with a similar relationship are recorded as a pair, which is recorded as a similar relationship pair; S42, according to the knowledge point prerequisite relationship provided by the data set, recording the IDs of two knowledge points with a prerequisite relationship as a pair, recorded as a prerequisite relationship pair; S43. Call the machine learning algorithm library torch_geometric, use the knowledge point ID as the node, similarity relationship pairs and prerequisite relationship pairs as the edges, and construct the similarity relationship knowledge graph and the prerequisite relationship knowledge graph respectively.

4. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized in that: In step S5, the specific steps include: S51, attention learning layer: According to the nodes in the two generated knowledge graphs, the embedded representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output i-th node linearly transformed embedding representation z is i As shown below: z i =Wh i Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix; S52. Calculate the original attention score between each pair of nodes: According to the relationship between the edges in the two generated knowledge graph data, find the nodes at both ends of an edge. First, concatenate the embedding representations of the two nodes, then do a dot product calculation on the concatenated embedding and a learnable weight, and finally activate it using the LeakyReLU function to output the original attention score e between each pair of nodes. ij As shown below: and ij =LeakyReLU(a T (With i ||from j )) Among them, z i and z j They represent the embedded representations of node i and node j after linear transformation, || here represents concatenation, a T is a learnable weight vector; S53. Perform Softmax operation on the original attention scores of all incoming edges of a node according to the two generated knowledge graphs to obtain the attention weight a ij As shown below: Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N(i) exp(e ik ) means to add the original attention scores of each pair of nodes; S54, according to the attention weight, aggregate the features of all neighbor nodes to obtain the final embedding representation h' of the node aggregation i As shown below: Among them, σ() is the activation function; S55. Use multi-head attention mechanism to stabilize the learning process of attention: Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads of the multi-head attention mechanism.

5. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized in that: In step S6, the specific steps include: S61. Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation; S62, randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding according to the knowledge point final embedding obtained in step S5; S63, combining the student hidden state, combining the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron, and outputting a recommended item; S64, putting the recommended items into the DKT model pre-trained in step S3 for testing and giving a score; S65. Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete a project recommendation. The formula is as follows: LOSS=―E t ∑logP Among them, E t represents the score improvement, and P represents the probability of the project being recommended; S66. Repeat the above steps to generate recommendation items in sequence and generate a learning path.

6. A system for recommending learning paths based on graph attention reinforcement learning embedded in a large language model according to any one of claims 1 to 5, characterized in that: The system includes the following modules: Data preprocessing module (110); A large language model embedding representation module (120) performs embedding representation using a large language model according to the knowledge point name; DKT model pre-training module (130), pre-training the DKT model based on the pre-processed data; A knowledge graph construction module (140) constructs two knowledge graphs according to the embedding representation of the large language model and the prerequisite relationship of the knowledge points; A graph attention module (150), based on the two knowledge graphs, uses a graph attention algorithm to obtain and accumulate the embedded representations of updated knowledge points; The recommendation module (160) finally embeds the obtained knowledge points into representations, recommends them in sequence in combination with the reinforcement learning method, and generates a learning path.

7. An electronic device, characterized in that: The electronic device comprises: a processor (210); and a memory (220) for storing one or more programs; When the one or more programs are executed by the processor (210), the processor executes the learning path recommendation method based on graph attention reinforcement learning based on large language model embedding as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by the processor (210), the method for recommending a learning path based on graph attention reinforcement learning embedded in a large language model as described in any one of claims 1 to 5 is implemented.

Images