A learning path recommendation method, system, device and storage medium for graph attention reinforcement learning based on large language model embedding

Abstract

The present invention relates to the field of data mining recommendation technology, and specifically discloses a graph attention reinforcement learning learning path recommendation method, system, device and storage medium based on large language model embedding, including the following steps: data preprocessing; large language model embedding representation; DKT model pre-training; constructing a data knowledge graph based on the large language model embedding representation and the original prerequisite relationship of the data set; updating the graph content with a graph attention algorithm, stabilizing the learning process with multi-head attention, and updating the knowledge point embedding representation; and recommending a learning path to the student in sequence based on the obtained knowledge point embedding representation and the student's historical learning information. The present invention introduces a large language model to embed the knowledge points, adds a graph attention algorithm and a reinforcement learning method, mines the correlation between knowledge points, and can achieve fast and reasonable learning path recommendation for students.

Description

Technical Field

[0001] The present invention relates to the field of data mining recommendation technology, and in particular to a method, system, device and storage medium for recommending learning paths based on graph attention reinforcement learning embedded in a large language model. Background Art

[0002] With the rapid development of internet technology, more and more learning resources are being widely disseminated online, giving rise to online education systems. Online education provides learners with the opportunity to learn anytime and anywhere, and also provides personalized learning and training programs for different students, playing a vital role in the development of students.

[0003] However, with the explosive growth of knowledge and learning resources, how to recommend personalized learning resources to learners based on their differences in learning ability, knowledge background, learning goals, etc., and tailor a learning path for learners to achieve their learning goals is a new challenge at present. Summary of the Invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a learning path recommendation method, system, device and storage medium for graph attention reinforcement learning based on large language model embedding, which can quickly and reasonably recommend learning paths for students, solving the problems mentioned in the above background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a learning path recommendation method based on graph attention reinforcement learning embedded in a large language model, comprising the following steps:

[0006] S1, data preprocessing;

[0007] S2. Use a large language model to embed the knowledge point name;

[0008] S3. Pre-train the DKT model based on the pre-processed data;

[0009] S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0010] S5. Based on the two knowledge graphs, use the graph attention algorithm to obtain the updated embedding representations of the knowledge points and accumulate them;

[0011] S6. The knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0012] Preferably, in step S1, data preprocessing includes data segmentation, data cleaning and data merging; the data segmentation refers to dividing the data according to different types, extracting the knowledge point names in file 1, and extracting the user ID, knowledge point name and student answer status in file 2; the data cleaning includes name deduplication, illegal character removal, data missing and data digitization processing; the data merging refers to the splicing processing of the cleaned data to form input data that can be input into the model.

[0013] Preferably, in step S4, the following is specifically included:

[0014] S41. Calculate the cosine similarity of the embedding representations of each knowledge point based on the knowledge point embedding representation generated by the large language model. If the similarity exceeds 0.4, it is considered that there is a similar relationship. The IDs of the two knowledge points with similar relationships are recorded as a pair, which is recorded as a similar relationship pair.

[0015] S42. Based on the knowledge point prerequisite relationship provided by the data set, the IDs of two knowledge points with a prerequisite relationship are recorded as a pair, which is recorded as a prerequisite relationship pair;

[0016] S43. Call the machine learning algorithm library torch_geometric, use the knowledge point ID as the node, similarity relationship pairs and prerequisite relationship pairs as the edge, and construct the similarity relationship knowledge graph and the prerequisite relationship knowledge graph respectively.

[0017] Preferably, in step S5, the embedded representation of each knowledge point is passed into the graph attention model, linearly transformed, and then the attention scores between each pair of knowledge points are calculated and softmax operation is performed on them. After that, the neighbor node information is aggregated. At the same time, a multi-head attention mechanism is used to stabilize the attention learning process, and finally the final embedded representation of each knowledge point is output. Specifically, it includes the following:

[0018] S51, attention learning layer: According to the nodes in the two generated knowledge graphs, the embedding representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output of the i-th node is the embedded representation z after the linear transformation. i As shown below:

[0019] z i =Wh i

[0020] Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix;

[0021] S52. Calculate the raw attention score between each pair of nodes: Find the nodes at both ends of an edge based on the relationship between the edges in the two generated knowledge graphs. First, concatenate the embedding representations of the two nodes, then perform a dot product calculation on the concatenated embedding and a learnable weight, and finally activate it using the LeakyReLU function. The output is the raw attention score e between each pair of nodes. ij As shown below:

[0022] e ij =LeakyReLU(a T (z i ||z j ))

[0023] Among them, z i and z j Represents the embedded representation of two nodes after linear transformation, || here represents splicing, a T is a learnable weight vector;

[0024] S53. Based on the two generated knowledge graphs, perform Softmax operation on the original attention scores of all incoming edges of a node to obtain the attention weight a. ij As shown below:

[0025]

[0026] Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N(i) exp(e ik ) means summing the original attention scores of each pair of nodes;

[0027] S54. According to the attention weight, aggregate the features of all neighbor nodes to obtain the final embedded representation h′ of the aggregation i As shown below:

[0028]

[0029] Among them, σ() is the activation function;

[0030] S55. Using the multi-head attention mechanism to stabilize the learning process of attention:

[0031]

[0032] Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads of the multi-head attention mechanism.

[0033] Preferably, in step S6, the following is specifically included:

[0034] S61. Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation;

[0035] S62: Randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding based on the knowledge point final embedding obtained in step S5;

[0036] S63: Combine the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron, based on the student's hidden state, and output a recommended item.

[0037] S64: Test the recommended items using the DKT model pre-trained in step S3 and give a score.

[0038] S65. Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete an item recommendation. The formula is as follows:

[0039] LOSS=-E t ∑log P

[0040] Among them, E t represents the score improvement, and P represents the probability of the project being recommended;

[0041] S66. Repeat the above steps to generate recommended items in sequence and generate a learning path.

[0042] On the other hand, to achieve the above-mentioned purpose, the present invention also provides the following technical solution: a graph attention reinforcement learning learning path recommendation system based on large language model embedding, the system comprising the following modules:

[0043] Data preprocessing module;

[0044] Large language model embedding representation module, which uses the large language model to embed representation according to the knowledge point name;

[0045] DKT model pre-training module, pre-trains the DKT model based on pre-processed data;

[0046] The knowledge graph construction module builds two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0047] The graph attention module uses the graph attention algorithm to obtain the updated embedding representation of knowledge points based on the two knowledge graphs and accumulates them;

[0048] The recommendation module finally embeds the obtained knowledge points and combines them with reinforcement learning methods to recommend them in sequence and generate a learning path.

[0049] On the other hand, to achieve the above-mentioned purpose, the present invention further provides the following technical solution: an electronic device, comprising: a processor; and a memory for storing one or more programs;

[0050] When the one or more programs are executed by the processor, the processor executes the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0051] On the other hand, to achieve the above-mentioned purpose, the present invention also provides the following technical solution: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0052] The beneficial effects of the present invention are:

[0053] 1) Solution for obtaining knowledge point relevance: For different knowledge points in the dataset, in the absence of other data, this invention uses a large language model embedding representation generation method to obtain the implicit relevance information between knowledge points, thereby establishing a connection between the student's knowledge status and the knowledge points;

[0054] 2) A new method for learning path recommendation: Compared with previous learning path recommendation methods, this invention incorporates the similarity and prerequisite relationships of knowledge points into each recommendation item through a graph attention algorithm and reinforcement learning rewards, and uses a pre-trained DKT test model to evaluate the quality of the path, making learning path recommendations more accurate and reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flow chart of a learning path recommendation method for graph attention reinforcement learning based on large language model embedding in an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the architecture of a learning path recommendation method for graph attention reinforcement learning based on large language model embedding in an embodiment of the present invention;

[0057] Figure 3 Schematic diagram of a learning path recommendation system module based on graph attention reinforcement learning embedded in a large language model according to an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the structure of an electronic device in an embodiment of the present invention;

[0059] In the figure, 110-data preprocessing module; 120-large language model embedding representation module; 130-DKT model pre-training module; 140-knowledge graph construction module; 150-graph attention module; 160-recommendation module; 210-processor; 220-storage. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0061] See also Figure 1 and Figure 2 The present invention provides a technical solution: a learning path recommendation method based on graph attention reinforcement learning embedded in a large language model, comprising the following steps:

[0062] S1. Data preprocessing: The Junyi dataset contains 26 million answers from 191,874 students and 712 valid knowledge points. To make better recommendations, we first need to perform the following operations on the data:

[0063] The data is segmented into different types, and the knowledge point names are extracted in file 1, and the user ID, knowledge point name and student answer status are extracted in file 2;

[0064] Data cleaning includes name deduplication, illegal character removal, data loss, and data digitization;

[0065] Deduplication of knowledge point names: Deduplication of knowledge point names in file 1 is performed, and serial numbers are added to non-duplicate items.

[0066] Illegal character removal: Delete illegal characters that appear in the knowledge point names in file 1.

[0067] Data missing processing: Delete the data items with empty user ID and knowledge point name in file 2.

[0068] Data digitization processing: Using the character replacement method, the knowledge point names in file 2 are replaced with numbers in sequence according to the knowledge point names and serial numbers in file 1.

[0069] Data merging refers to the process of combining cleaned data to form input data that can be input into the model.

[0070] Group file 2 by user ID, extract the knowledge point number and student response for each user ID into two separate rows. Delete the data for users with fewer than two knowledge point numbers and save the remaining data.

[0071] S2. Use a large language model to embed the knowledge point name;

[0072] In order to extract the relevance between knowledge points, we use the open source large language model Llama-2-7B to embed each knowledge point and extract the hidden information of the knowledge point, including the environment deployment and embedding output. Specifically,

[0073] 2-1) Configure the operating environment required by the Llama-2-7B large language model on your local computer, including GPU driver, Python language environment, and run the large language model;

[0074] 2-2) Extract the knowledge point names from file 1 and input them sequentially into the Llama-2-7B model according to the following input format. Generate and save the embedded representation of each knowledge point in turn.

[0075] ###Instruction: Given knowledge concept in order, output the embedding of each concept.

[0076] ###Input:{knowledge concept}

[0077] ###Response:{Embedding Vector}.

[0078] S3. Pre-train the DKT model based on the pre-processed data;

[0079] In order to evaluate the rationality of the learning path, a pre-trained knowledge tracing (DKT) model is required to simulate the student learning process, including environment deployment, model training, and testing:

[0080] 3-1) Configure the operating environment required by the DKT model locally and deploy the DKT model;

[0081] 3-2) Input the pre-processed data of file 1 into the DKT model in sequence for training and testing;

[0082] 3-3) After training is completed, save the trained DKT model parameters.

[0083] S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0084] The details include:

[0085] 4-1) Based on the knowledge point embedding representation generated by the large language model, calculate the cosine similarity of the embedding representation of each knowledge point. If the similarity exceeds 0.4, it is considered to have a similar relationship. The IDs of two knowledge points with similar relationships are recorded as a pair, which is recorded as a similar relationship pair;

[0086] 4-2) Based on the prerequisite relationship of the knowledge points provided by the dataset, the IDs of two knowledge points with a prerequisite relationship are recorded as a pair, which is recorded as a prerequisite relationship pair;

[0087] 4-3) Call the machine learning algorithm library torch_geometric, use the above knowledge point IDs as nodes, similarity relationship pairs and prerequisite relationship pairs as edges, and build a similarity relationship knowledge graph and a prerequisite relationship knowledge graph respectively.

[0088] S5. Based on the two knowledge graphs, use the graph attention algorithm to obtain the updated embedding representations of the knowledge points and accumulate them;

[0089] 5-1) Attention learning layer: According to the nodes in the two generated knowledge graphs, the embedding representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output embedding representation z of the i-th node after linear transformation is i As shown below:

[0090] z i =Wh i

[0091] Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix;

[0092] 5-2) Calculate the raw attention score between each pair of nodes: According to the relationship between the edges in the two generated knowledge graph data, find the nodes at both ends of an edge. First, concatenate the embedding representations of the two nodes, then perform a dot product calculation on the concatenated embedding and a learnable weight, and finally activate it using the LeakyReLU function to output the raw attention score e between each pair of nodes. ij As shown below:

[0093] e ij =LeakyReLU(a T (z i ||z j ))

[0094] Among them, z i and z j Represents the embedded representation of two nodes after linear transformation, || here represents splicing, a T is a learnable weight vector;

[0095] 5-3) According to the two generated knowledge graphs, perform Softmax operation on the original attention scores of all incoming edges of a node to obtain the attention weight a ij As shown below:

[0096]

[0097] Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N (i)exp(e ik ) means summing the original attention scores of each pair of nodes;

[0098] 5-4) According to the attention weight, the features of all neighbor nodes are aggregated to obtain the final embedded representation h′ of the aggregation i As shown below:

[0099]

[0100] Among them, σ() is the activation function;

[0101] 5-5) Using the multi-head attention mechanism to stabilize the learning process of attention:

[0102]

[0103] Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads of the multi-head attention mechanism.

[0104] S6. The knowledge points are finally embedded and represented, and the reinforcement learning method is used to recommend them in sequence to generate a learning path.

[0105] The details include:

[0106] 6-1) Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation;

[0107] 6-2) Randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding based on the knowledge point final embedding obtained in step S5;

[0108] 6-3) Combine the student's hidden state and the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron to output a recommended item;

[0109] 6-4) Test the recommended items in the DKT model pre-trained in step S3 and give a score;

[0110] 6-5) Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete an item recommendation. The formula is as follows:

[0111] LOSS=-E t ∑logP

[0112] Among them, E t represents the score improvement, and P represents the probability of the project being recommended;

[0113] 6-6) Repeat the above steps to generate recommended items in sequence and generate a learning path.

[0114] The present invention introduces a large language model to embed knowledge points, adds a graph attention algorithm and a reinforcement learning method, mines the correlation between knowledge points, and can quickly and reasonably recommend learning paths for students.

[0115] Based on the same inventive concept as the above method embodiment, the embodiment of the present application also provides a learning path recommendation system based on graph attention reinforcement learning embedded in a large language model, which can implement the functions provided by the above method embodiment, such as Figure 3 As shown, the system includes the following modules:

[0116] Data preprocessing module 110;

[0117] A large language model embedding representation module 120 performs embedding representation based on the knowledge point name using a large language model;

[0118] DKT model pre-training module 130, pre-training the DKT model based on the pre-processed data;

[0119] The knowledge graph construction module 140 constructs two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of the knowledge points;

[0120] The graph attention module 150 uses the graph attention algorithm to obtain the updated embedding representations of the knowledge points based on the two knowledge graphs and accumulates them;

[0121] The recommendation module 160 uses the final embedded representation of the obtained knowledge points and combines them with the reinforcement learning method to recommend them in sequence and generate a learning path.

[0122] Based on the same inventive concept as the above method embodiment, the embodiment of the present application further provides an electronic device, such as Figure 4 As shown, the device includes: a processor 210; and a memory 220 for storing one or more programs;

[0123] When the one or more programs are executed by the processor 210, the processor executes the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0124] The learning path recommendation method based on graph attention reinforcement learning with large language model embedding specifically includes the following:

[0125] Data preprocessing;

[0126] Use a large language model to embed representations based on knowledge point names;

[0127] Pre-train the DKT model based on the pre-processed data;

[0128] Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0129] Based on the two knowledge graphs, the graph attention algorithm is used to obtain the embedded representations of the updated knowledge points and accumulate them;

[0130] The obtained knowledge points are finally embedded and represented, and the reinforcement learning method is combined to recommend them in sequence to generate a learning path.

[0131] Based on the same inventive concept as the above-mentioned method embodiment, the embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 210, it implements the graph attention reinforcement learning learning path recommendation method based on large language model embedding.

[0132] The learning path recommendation method based on graph attention reinforcement learning with large language model embedding specifically includes the following:

[0133] Data preprocessing;

[0134] Use a large language model to embed representations based on knowledge point names;

[0135] Pre-train the DKT model based on the pre-processed data;

[0136] Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points;

[0137] Based on the two knowledge graphs, the graph attention algorithm is used to obtain the embedded representations of the updated knowledge points and accumulate them;

[0138] The obtained knowledge points are finally embedded and represented, and the reinforcement learning method is combined to recommend them in sequence to generate a learning path.

[0139] In the several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0140] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0141] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, electronic device, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. It should be noted that, in this article, the terms "include", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0142] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "an", "the" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise.

[0143] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0144] The word "if," as used herein, may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0145] The references to "first" and "second" in the embodiments merely distinguish similar objects and do not represent a specific ordering of the objects. It is understood that the specific order or precedence of "first" and "second" can be interchanged where appropriate. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0146] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A learning path recommendation method based on graph attention reinforcement learning with large language model embedding, characterized by: The steps include: S1, data preprocessing; S2. Use a large language model to embed the knowledge point name; S3. Pre-train the DKT model based on the pre-processed data; S4. Construct two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationships of knowledge points; S5. Based on the two knowledge graphs, use the graph attention algorithm to obtain the updated embedding representations of the knowledge points and accumulate them; S6. The final embedding representation of the obtained knowledge points is combined with reinforcement learning methods to recommend them in sequence and generate a learning path; In step S5, the specific steps include: S51, attention learning layer: According to the nodes in the two generated knowledge graphs, the embedding representation of each knowledge point is passed into the graph attention model. The initial embedding of the node is first subjected to a simple linear transformation, and the output of the i-th node is the embedded representation z after the linear transformation. i As shown below: z i =Wh i Among them, h i represents the embedding representation of the i-th node, W is a trainable matrix; S52. Calculate the raw attention score between each pair of nodes: Find the nodes at both ends of an edge based on the relationship between the edges in the two generated knowledge graphs. First, concatenate the embedding representations of the two nodes, then perform a dot product calculation on the concatenated embedding and a learnable weight, and finally activate it using the LeakyReLU function. The output is the raw attention score e between each pair of nodes. ij As shown below: e ij =LeakyReLU(a T (With i ||From j )) Among them, z i and z j Respectively represent the embedded representations of node i and node j after linear transformation, || here represents splicing, a T is a learnable weight vector; S53. Based on the two generated knowledge graphs, perform Softmax operation on the original attention scores of all incoming edges of a node to obtain the attention weight a. ij As shown below: Among them, e ik represents the original attention score between the i-th node and the k-th node, ∑ k∈N(i) exp(e ik ) means summing the original attention scores of each pair of nodes; S54. According to the attention weight, the features of all neighbor nodes are aggregated to obtain the final embedding representation h′ of the node aggregation i As shown below: Among them, σ() is the activation function; S55. Using the multi-head attention mechanism to stabilize the learning process of attention: Finally, the final embedding representation of each knowledge point is output, where K represents the number of heads in the multi-head attention mechanism; In step S6, the specific steps include: S61. Build an LSTM neural network simulator, randomly generate 10 knowledge point IDs as the student's historical learning data, input them into the LSTM neural network simulator, and generate the student's hidden state representation; S62: Randomly generate 5 knowledge point IDs as student target concepts, and query the corresponding target concept embedding based on the knowledge point final embedding obtained in step S5; S63: Combine the final embedding representation of the knowledge point obtained in step S5 with the embedding representation of the target concept using a multi-layer perceptron, based on the student's hidden state, and output a recommended item. S64: Test the recommended items using the DKT model pre-trained in step S3 and give a score. S65. Consider the rating as a reward in reinforcement learning and incorporate it into the loss function to complete an item recommendation. The formula is as follows: LOSS=-E t ∑logP Among them, E t represents the score improvement, and P represents the probability of the project being recommended; S66. Repeat the above steps to generate recommended items in sequence and generate a learning path.

2. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized by: In step S1, data preprocessing includes data segmentation, data cleaning and data merging; the data segmentation refers to dividing the data according to different types, extracting the knowledge point names in file 1, and extracting the user ID, knowledge point name and student answer status in file 2; the data cleaning includes name deduplication, illegal character removal, data missing and data digitization; the data merging refers to the splicing of the cleaned data to form input data that can be input into the model.

3. The learning path recommendation method based on graph attention reinforcement learning with large language model embedding according to claim 1 is characterized by: In step S4, the specific steps include: S41. Calculate the cosine similarity of the embedding representations of each knowledge point based on the knowledge point embedding representation generated by the large language model. If the similarity exceeds 0.4, it is considered that there is a similar relationship. The IDs of the two knowledge points with similar relationships are recorded as a pair, which is recorded as a similar relationship pair. S42. Based on the knowledge point prerequisite relationship provided by the data set, the IDs of two knowledge points with a prerequisite relationship are recorded as a pair, which is recorded as a prerequisite relationship pair; S43. Call the machine learning algorithm library torch_geometric, use the knowledge point ID as the node, similarity relationship pairs and prerequisite relationship pairs as the edge, and construct the similarity relationship knowledge graph and the prerequisite relationship knowledge graph respectively.

4. A system for recommending learning paths using graph attention reinforcement learning based on large language model embedding according to any one of claims 1 to 3, characterized in that: The system includes the following modules: Data preprocessing module (110); A large language model embedding representation module (120) is used to embed the knowledge point name using the large language model; A DKT model pre-training module (130) pre-trains the DKT model based on the pre-processed data; The knowledge graph construction module (140) constructs two knowledge graphs based on the embedding representation of the large language model and the prerequisite relationship of the knowledge points; The graph attention module (150) obtains the updated embedding representations of the knowledge points based on the two knowledge graphs using the graph attention algorithm and accumulates them; The recommendation module (160) finally embeds the obtained knowledge points and recommends them in sequence in combination with the reinforcement learning method to generate a learning path.

5. An electronic device, characterized in that: The electronic device includes: a processor (210); and a memory (220) for storing one or more programs; When the one or more programs are executed by the processor (210), the processor is caused to execute the graph attention reinforcement learning learning path recommendation method based on large language model embedding as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by the processor (210), the method for recommending a learning path based on graph attention reinforcement learning embedded in a large language model as described in any one of claims 1 to 3 is implemented.

Images