Concept recommendation method based on large language model structure learning and knowledge representation

Abstract

The present invention discloses a concept recommendation method based on large language model structure learning and knowledge representation, which belongs to the field of concept recommendation. The method of the present invention first obtains the learning data and concept knowledge graph of the student account in the learning process and performs preprocessing; uses the pre-trained language model to encode the enhanced text generated by the large language model to obtain the final hidden vector of each concept, and then constructs the enhanced concept knowledge graph, pre-trains the GraphGPS concept encoder based on the enhanced concept knowledge graph and the dual-view cross-graph comparison learning method; synthesizes educational data by the TabDDPM model to pre-train the deep knowledge tracking model; splices the concept representation of question embedding, answer embedding, structure perception and student knowledge state representation to form the final representation, and then obtains the coding sequence, pre-trains the sequence coding model and fine-tunes the concept recommendation matrix, and finally forms a concept recommendation model for concept recommendation.

Description

Technical Field

[0001] The present invention belongs to the field of concept recommendation, and in particular relates to a concept recommendation method based on large language model structure learning and knowledge representation. Background Art

[0002] The continuous progress of Internet technology has greatly promoted the popularity of online education, making it a key way for individuals to acquire professional knowledge and skills. Online education platforms recommend appropriate learning content to students to serve their knowledge acquisition. Concept recommendation aims to recommend the next concept to be learned by students based on their knowledge status and human knowledge system. Although knowledge tracking models can predict knowledge status, previous methods have not effectively integrated the human knowledge system into the design process of education models. Many fields have begun to use large language models (LLMs) to generate and encode texts and introduce external human knowledge systems. Therefore, integrating LLMs into concept recommendation has become a major challenging problem that needs to be solved in the development of online learning. This urgently requires a reasonable and effective concept recommendation method to solve the problem that online education platforms fail to explore the potential deep semantic connections between concepts. The development of LLMs provides a way to introduce concept semantics, using it to generate and encode text information to help represent concepts for recommendation. Therefore, the core of integrating LLMs into concept recommendation is to represent concepts in a way that includes information from the human knowledge system and adapt the encoding of text by LLMs to concept recommendation.

[0003] At present, in order to improve students' mastery level, the strategy of planning concept learning paths should be based on two types of key information: 1) Students' knowledge status (student side), that is, students' mastery of concepts. By applying knowledge tracing models, students' knowledge status can be effectively predicted and evaluated. Many related models have been proposed in this regard, such as Deep Knowledge Tracing (DKT), Attentive Knowledge Tracing (AKT), etc., which use neural networks to dynamically track students' knowledge status, capture the relationship between knowledge points, and use historical learning data for prediction. However, they have difficulty understanding the semantics of knowledge concepts and questions, and there are problems of cold start and lack of interpretability. 2) Human knowledge system (concept side), that is, the semantic and structural relationship between concepts. Many existing methods introduce the relationship between concepts through knowledge graphs, but these methods ignore the intrinsic semantics of concepts, such as Graph-based Knowledge Tracing (GKT). Therefore, although they capture how students interact with concepts and their dependencies, they fail to integrate specific factual knowledge and external knowledge about concepts, and cannot reveal the deep semantic associations implicit between concepts.

[0004] Introducing the knowledge of LLMs to represent concepts faces two challenges: 1) How to represent concepts in a way that includes information from the human knowledge system? Avoid ambiguity of concepts in different learning contexts; 2) How to adapt the non-smooth and anisotropically distributed text encodings generated by LLMs to concept recommendations. Summary of the invention

[0005] The purpose of the present invention is to solve the problems existing in the prior art and provide a concept recommendation method based on large language model structure learning and knowledge representation.

[0006] The present invention uses a large language model to generate non-ambiguous concept explanations, and introduces the GraphGPS concept encoder, which is pre-trained in a self-supervised manner through a dual-view cross-graph contrast learning method to migrate concept representations from the text space to a knowledge space consistent with the knowledge structure, accurately capturing the complex relationships between concepts. Finally, concept recommendations are made based on the student's knowledge status and the enhanced concept representation.

[0007] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0008] A concept recommendation method based on large language model structure learning and knowledge representation, comprising the following steps:

[0009] S1: Obtain the learning data and related concept knowledge graphs of student accounts during the learning process from the online education platform, and preprocess the learning data and concept knowledge graphs;

[0010] S2: Generate enhanced text for each concept in the concept knowledge graph using a large language model, encode the enhanced text using a pre-trained language model, and obtain the final hidden vector for each concept. Update the nodes in the concept knowledge graph using the final hidden vector to obtain an enhanced concept knowledge graph. Input the enhanced concept knowledge graph into the GraphGPS concept encoder, and pre-train the GraphGPS concept encoder using a dual-view cross-graph contrast learning method to obtain a pre-trained GraphGPS concept encoder.

[0011] S3: Randomly sample the learning data of student accounts as the input of the TabDDPM model, generate synthetic education data through the back-diffusion process, merge the synthetic education data with the learning data of student accounts to form a knowledge tracking dataset, and pre-train the deep knowledge tracking model on the knowledge tracking dataset;

[0012] S4: Input the new concept knowledge graph into the pre-trained GraphGPS concept encoder, output the structure-aware concept representation of each time step, input the new learning data into the pre-trained deep knowledge tracking model, output the student knowledge state representation of each time step, embed the new student learning data, obtain the question embedding and answer embedding of each time step, concatenate the question embedding, answer embedding, structure-aware concept representation and student knowledge state representation of each time step, obtain the final representation of each time step, and the final representation of each time step corresponding to each learning data constitutes a coding sequence of the learning data;

[0013] S5: Input the coding sequence of the learning data into the sequence encoding model for pre-training, fine-tune the learnable concept recommendation matrix based on the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model, and construct a concept recommendation model by the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model as well as the fine-tuned concept recommendation matrix. Input the student learning data for concept recommendation into the concept recommendation model, calculate the score of each concept, generate the concept recommendation results for the online education platform for the student account, and push them to the user interface of the online education platform in a preset manner.

[0014] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0015] Preferably, the specific process of step S1 includes:

[0016] S11: Obtain the learning data and concept knowledge graph of the student account during the learning process from the database of the online education platform. The learning data of the student account includes the student's answer record, learning behavior data and learning time. The concept knowledge graph includes the name, definition, preceding concept and succeeding concept of the concept.

[0017] S12: Clean the learning data, remove irrelevant or redundant learning behavior data, retain the answer records related to the learning process, then interpolate the missing learning data, and finally remove outliers and noise in the learning data to obtain the preprocessed learning data;

[0018] S13: Standardize the representation of each concept in the concept knowledge graph, process the relationship between concepts in the concept knowledge graph, correct and enhance incomplete or inaccurate relationships, use each concept after the standardized representation as a node in the preprocessed concept knowledge graph, use the association information between the processed concepts as the edges in the preprocessed concept knowledge graph, and obtain the preprocessed concept knowledge graph.

[0019] Preferably, in step S2, in the process of pre-training the GraphGPS concept encoder using the dual-view cross-graph contrast learning method, the enhanced concept knowledge graph is processed by the data enhancement method and the model enhancement technique based on graph contrast learning, and the data enhancement graph and the model enhancement graph are generated accordingly. The GraphGPS concept encoder is perturbed by the model enhancement method based on pruning to obtain a perturbation encoder. The data enhancement graph and the model enhancement graph are processed by the GraphGPS concept encoder respectively to obtain the first encoding representation and the third encoding representation respectively. The data enhancement graph and the model enhancement graph are processed by the perturbation encoder respectively to obtain the second encoding representation and the fourth encoding representation respectively. The four encoding representations are each passed through a projection layer consisting of a two-layer MLP, and each encoding representation obtains a corresponding encoding result. Two encoding results from the same enhanced concept knowledge graph and processed by different encoders using the same data enhancement method are used as positive sample pairs, and two encoding results from different enhanced concept knowledge graphs and processed by the same data enhancement method are used as negative sample pairs. The NT-Xent loss is calculated based on the constructed positive and negative sample pairs to update the GraphGPS concept encoder parameters.

[0020] Preferably, the pruning-based model enhancement method is one or more of a weight pruning-based enhancement method, a layer pruning-based enhancement method or an attention head pruning-based enhancement method, wherein the weight pruning-based enhancement method adopts the L1 unstructured pruning method.

[0021] Preferably, the layer pruning based enhancement method is implemented using the first Bernoulli random variable and residual connections, and the first Bernoulli random variable determines whether each layer of the Transformer architecture is retained.

[0022] Furthermore, the enhancement method based on layer pruning can be expressed as follows:

[0023]

[0024] in, and For the Transformer architecture and Tier Output feature representation of the attention head; ReLU activation function. Represents the first Bernoulli random variable, which takes the value of 0 or 1 and is used to determine the first Whether the layer is retained; Represents the Transformer architecture The GraphTransformer-based encoder within the layer is used to combine local and global information for feature extraction; Indicates constant change.

[0025] Preferably, the enhancement method based on attention head pruning is implemented using the Bernoulli random variable of each attention head of the Transformer architecture, and the Bernoulli random variable of each attention head is used to determine whether each attention head of the Transformer architecture is retained.

[0026] Furthermore, the enhancement method based on attention head pruning can be expressed as follows:

[0027]

[0028] in, Represents the Transformer architecture The output representation of the layer; Represents a splicing operation; Represents the Transformer architecture Tier The Bernoulli random variable of each attention head takes the value of 0 or 1, which is used to determine the Transformer architecture. Layer Whether the attention head is retained; Represents the Transformer architecture Tier The representation of an attention head; is the linear layer weight.

[0029] Preferably, in step S3, during the pre-training of the deep knowledge tracking model, the data in the knowledge tracking dataset is input into the deep knowledge tracking model, first encoded by a recurrent neural network to obtain the learning history and input it into a gated recurrent unit to obtain a representation of the student's knowledge state.

[0030] Preferably, in step S4, the question serial numbers of the answer records in the new learning data are sequentially passed through two MLPs to obtain question embedding, and the answers of the answer records in the new student learning data are passed through two MLPs to obtain answer embedding.

[0031] Preferably, in step S5, during the pre-training process of the sequence coding model, multi-head attention calculation is first performed on the coding sequence of the learning data, and the multi-head attention calculation results are input into the feedforward network for sequence coding to obtain the sequence coding results of each time step, and the sequence coding model parameters are updated based on minimizing the sequence coding loss.

[0032] Preferably, in step S5, during the fine-tuning of the concept recommendation matrix, the pre-trained GraphGPS concept encoder regenerates the structure-aware concept representation, the pre-trained deep knowledge tracking model regenerates the student knowledge state representation, and after the question embedding and the answer embedding are regenerated, the regenerated question embedding, answer embedding, structure-aware concept representation and student knowledge state representation are concatenated to obtain the final representation and constitute a new encoding sequence, the pre-trained sequence encoding model performs sequence encoding on the new encoding sequence, and the sequence encoding result is regenerated, the regenerated sequence encoding result is multiplied by the learnable concept recommendation matrix, the scores of all concepts are calculated, and the concept recommendation matrix is ​​updated using the cross entropy loss.

[0033] On the other hand, the present invention also provides a method for constructing a concept recommendation model based on large language model structure learning and knowledge representation, which comprises the following steps:

[0034] S1: Obtain the learning data and related concept knowledge graphs of student accounts during the learning process from the online education platform, and preprocess the learning data and concept knowledge graphs;

[0035] S2: Generate enhanced text for each concept in the concept knowledge graph using a large language model, encode the enhanced text using a pre-trained language model, and obtain the final hidden vector for each concept. Update the nodes in the concept knowledge graph using the final hidden vector to obtain an enhanced concept knowledge graph. Input the enhanced concept knowledge graph into the GraphGPS concept encoder, and pre-train the GraphGPS concept encoder using a dual-view cross-graph contrast learning method to obtain a pre-trained GraphGPS concept encoder.

[0036] S3: Randomly sample the learning data of student accounts as the input of the TabDDPM model, generate synthetic education data through the back-diffusion process, merge the synthetic education data with the learning data of student accounts to form a knowledge tracking dataset, and pre-train the deep knowledge tracking model on the knowledge tracking dataset;

[0037] S4: Input the new concept knowledge graph into the pre-trained GraphGPS concept encoder, output the structure-aware concept representation of each time step, input the new learning data into the pre-trained deep knowledge tracking model, output the student knowledge state representation of each time step, embed the new student learning data, obtain the question embedding and answer embedding of each time step, concatenate the question embedding, answer embedding, structure-aware concept representation and student knowledge state representation of each time step, obtain the final representation of each time step, and the final representation of each time step corresponding to each learning data constitutes a coding sequence of the learning data;

[0038] S5: Input the encoded sequence of the learning data into the sequence encoding model for pre-training, fine-tune the learnable concept recommendation matrix based on the pre-trained GraphGPS concept encoder, deep knowledge tracking model and sequence encoding model, and construct a concept recommendation model by the pre-trained GraphGPS concept encoder, deep knowledge tracking model, sequence encoding model and the fine-tuned concept recommendation matrix.

[0039] It should be noted that the specific implementation process of the concept recommendation model construction method S1~S5 based on large language model structure learning and knowledge representation in the present invention can refer to the specific implementation process of the concept recommendation method based on large language model structure learning and knowledge representation in the present invention, which will not be repeated here.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] Compared with traditional knowledge tracking methods, this invention effectively eliminates concept ambiguity and improves the accuracy and interpretability of concept recommendation by combining a large language model with a concept knowledge graph, using structure-aware text representation and GraphGPS concept encoder. Through a dual-view cross-graph comparison learning method and a generative model based on TabDDPM, this invention enhances the robustness of the concept recommendation model under the conditions of student cold start and data scarcity, can accurately infer students' knowledge status, realize personalized and accurate concept recommendation, and meet the high-precision requirements of online education platforms for concept recommendation models. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of the steps of the present invention;

[0043] Figure 2 A schematic diagram of a conceptually recommended model structure of the present invention;

[0044] Figure 3 It is a schematic diagram of the dual-view cross-graph comparison learning method of the present invention. DETAILED DESCRIPTION

[0045] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0046] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0047] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned concept recommendation method based on large language model structure learning and knowledge representation includes the following steps S1 to S5. The specific implementation process is described in detail below.

[0048] S1: Obtain the learning data and related concept knowledge graphs of student accounts during the learning process from the online education platform, and preprocess the learning data and concept knowledge graphs.

[0049] It should be noted that in step S1 of the present invention, the learning data and related concept knowledge graphs of the student accounts during the learning process are obtained from the online education platform, and the learning data are cleaned and processed to ensure the quality and integrity of the learning data. The concept knowledge graph is then structured to ensure that the representation of each concept is accurate and unambiguous and to optimize the association between concepts. Finally, the preprocessed learning data and concept knowledge graph are saved in a standard format.

[0050] As a specific implementation form of the embodiment of the present invention, the specific process of step S1 includes:

[0051] S11: Obtain the learning data and concept knowledge graph of the student account during the learning process from the database of the online education platform. The learning data of the student account includes the student's answer record, learning behavior data and learning time. The concept knowledge graph includes the name, definition, predecessor concept and successor concept of the concept.

[0052] In this embodiment, learning data can be obtained through an online education platform, a learning platform, or a learning management system (LMS); the concept knowledge graph describes the dependencies between concepts and can be constructed by experts in the education field or extracted from public education resources.

[0053] It should be noted that in the present invention, the key feature of the concept recommendation problem is that students' choices are largely based on the human knowledge system, which is constructed by the semantics of concepts and the relationship between concepts. Teachers also develop structured teaching plans based on a full understanding of each concept. Therefore, integrating the semantics of each concept is crucial to designing a concept recommendation method.

[0054] Generating explanations for each concept directly using concept names or using conversational models such as GPT-4o can lead to ambiguity issues, where a concept has different meanings in a research field or context. For example, "table" can refer to a data structure in data science, or a watch in daily conversation. If LLMs is asked to explain "table", it may incorrectly interpret it as an item such as a "watch". To eliminate this ambiguity, the present invention improves the prompts for LLMs by including the preceding and succeeding nodes of the concept from the knowledge graph. This approach helps LLMs provide explanations tailored to specific educational contexts, greatly reducing the risk of ambiguity.

[0055] In order to ensure that the text of a concept contains sufficient information, the present invention designs a text representation of each concept, which includes the following four parts:

[0056] 1) Concept name: the name of the concept, such as "table" in mathematics, rather than "watch" in Chinese;

[0057] 2) Concept explanation: that is, the definition of the concept, using GPT-4o to infer the explanation of the concept based on the predecessor and successor nodes on the knowledge graph, such as "the concept of a table refers to the structured arrangement of data or information in rows and columns";

[0058] 3) Prerequisite concept: the name of the prerequisite concept of the concept in the knowledge graph, such as "histogram";

[0059] 4) Successor concept: The name of the post-condition concept of the concept in the knowledge graph, such as "Venn diagram, mean", "Venn diagram".

[0060] S12: Clean the learning data, remove irrelevant or redundant learning behavior data, retain the answer records related to the learning process, then interpolate the missing learning data, and finally remove outliers and noise in the learning data to obtain the preprocessed learning data.

[0061] In this embodiment, the learning data is cleaned and processed to remove irrelevant or redundant learning behavior data, retain the answer records related to the learning process, and then interpolate the missing learning data (such as unanswered records) to ensure data integrity. Finally, a denoising algorithm is used to remove outliers and noise in the learning data to improve the quality of the learning data.

[0062] S13: Standardize the representation of each concept in the concept knowledge graph, process the relationship between concepts in the concept knowledge graph, correct and enhance incomplete or inaccurate relationships, use each concept after the standardized representation as a node in the preprocessed concept knowledge graph, use the association information between the processed concepts as the edges in the preprocessed concept knowledge graph, and obtain the preprocessed concept knowledge graph.

[0063] In this embodiment, each concept in the concept knowledge graph is first represented in a standardized manner to ensure that the concept name, definition, predecessor concept, and successor concept can clearly reflect its position in the knowledge system. Then, the relationship between concepts in the concept knowledge graph is processed, and incomplete or inaccurate relationships are corrected and enhanced, so that the relationship between each concept and its associated concepts is clear and unambiguous. Finally, each concept after the standardized representation is used as a node in the preprocessed concept knowledge graph, and the association information between the processed concepts is used as an edge in the preprocessed concept knowledge graph to obtain the preprocessed concept knowledge graph.

[0064] In this embodiment, the processed student learning data and concept knowledge graph data are saved in a standard format (such as JSON, CSV, etc.) and managed in a unified manner. These data will be used for subsequent model training, concept recommendation, and performance evaluation, and metadata will be added to each piece of data to facilitate subsequent rapid access and management.

[0065] S2: Use the large language model to generate enhanced text for each concept in the concept knowledge graph, use the pre-trained language model to encode the enhanced text, and obtain the final hidden vector of each concept. Use the final hidden vector to update the nodes in the concept knowledge graph to obtain the enhanced concept knowledge graph, and input the enhanced concept knowledge graph into the GraphGPS concept encoder. Use the dual-view cross-graph contrast learning method to pre-train the GraphGPS concept encoder to obtain the pre-trained GraphGPS concept encoder.

[0066] It should be noted that in step S2 of the present invention, a large language model (such as GPT-4) is used to generate enhanced text for each concept in the concept knowledge graph to ensure that the concept definition matches the specific educational background. Enhanced text for each concept Then add a marker at the beginning Considering the directness of the language model architecture and its impressive text understanding capabilities, the pre-trained language model is used to encode the augmented text to obtain the final hidden vector of each concept. Take the concept as an example, and its corresponding final hidden vector can be recorded as:

[0067]

[0068] in, Indicates the first words; represents the pre-trained language model; Indicates the length of the augmented text.

[0069] It should be noted that in step S2 of the present invention, the GraphGPS concept encoder is used as its core encoding mechanism. Taking into account the limitations in processing long-distance dependencies, the present invention enhances the concept node representation capability by combining global information and local information, and accurately captures the complex semantic relationship between concepts. The GraphGPS concept encoder includes two modules, an embedding module and a processing module. The embedding module is used to effectively fuse the features of position encoding (PE), structure encoding (SE), nodes, edges, and graphs; the processing module is used to combine local message passing and global attention mechanisms. In addition, the GraphGPS concept encoder uses a graph convolutional network GCN to capture local graph information, and introduces a Transformer architecture to capture global information. By combining local graph information and global information, a richer and more comprehensive node representation can be generated, and finally a structure-aware concept representation is generated, which can accurately reflect the complex relationship between concepts.

[0070] In this embodiment, the first part of the Transformer architecture in the GraphGPS concept encoder is Taking the layer as an example, the corresponding processing process is:

[0071]

[0072] In the formula, For concept In the Layer, Node representation of the layer; Representation concept The previous layer node represents the combination function with the aggregation result; It is multi-head attention calculation; Represents concepts In the The query matrix, key matrix, and value matrix of the layer; To acquire concepts Aggregation function for neighbor embeddings; is a nonlinear transformation function; Representation concept The neighbor set.

[0073] It should be noted that in step S2 of the present invention, in the process of pre-training the GraphGPS concept encoder using the dual-view cross-graph contrastive learning method, the data enhancement method and the graph contrastive learning (Graph ContrastiveLearning, The enhanced concept knowledge graph is processed by the model enhancement technique of GCL, and the data enhancement graph and the model enhancement graph are generated accordingly. The GraphGPS concept encoder is perturbed by the model enhancement method based on pruning to obtain the perturbation encoder. The data enhancement graph and the model enhancement graph are processed by the GraphGPS concept encoder respectively, and the first encoding representation and the third encoding representation are obtained accordingly. The perturbation encoder processes the data enhancement graph and the model enhancement graph respectively, and the second encoding representation and the fourth encoding representation are obtained accordingly. The four encoding representations are respectively passed through a projection layer composed of a two-layer MLP, and each encoding representation obtains a corresponding encoding result. The two encoding results from the same enhanced concept knowledge graph and processed by different encoders using the same data enhancement method are used as positive sample pairs, and the two encoding results from different enhanced concept knowledge graphs and processed by the same data enhancement method are used as negative sample pairs. The NT-Xent loss is calculated based on the constructed positive and negative sample pairs to update the GraphGPS concept encoder parameters.

[0074] In this embodiment, in order to obtain a smooth, structure-aware concept representation, this embodiment combines the enhanced concept knowledge graph for learning. The purpose of using the enhanced concept knowledge graph is to improve the applicability of ordinary text embeddings for concept recommendation tasks. This is achieved by converting the embedding of concepts from a text representation space to a representation space (knowledge space) that is consistent with the knowledge structure. In this knowledge space, the representation of each concept enriches the relational features in the concept knowledge graph, ensuring that nodes that are structurally similar to each other also show similarity in the embedding space. In addition, considering that existing graph contrastive learning methods focus on single-view enhancement, two positive samples are generated and their contrast loss is calculated, however, this single-view enhancement limits the diversity of positive samples, thereby limiting the model's ability to effectively capture feature consistency within related views. Based on the above problems, the present invention proposes a comprehensive method for data and model enhancement for contrastive learning, namely a dual-view cross-graph contrastive learning method, which is an innovative method that can generate twice as many positive samples as traditional methods. A dual-view cross-graph contrastive learning method is used to pre-train the GraphGPS concept encoder, including pruning-based model enhancement (weight pruning-based enhancement, layer pruning-based enhancement, and attention head pruning-based enhancement) and multi-view contrastive learning. When the structure of the enhanced concept knowledge graph is disturbed, a stable node representation is learned to reduce redundant information, improve the learning efficiency of the GraphGPS concept encoder, and enhance the GraphGPS concept encoder's ability to represent concepts in the knowledge graph, thereby capturing the deep structural information of the graph. Through these optimizations, a GraphGPS concept encoder that can accurately reflect the deep relationship between concepts is finally generated.

[0075] Specifically, during the pre-training process of the GraphGPS concept encoder, the initial encoding of each node is first set to the final hidden vector of each concept. Figure 3As shown, the enhanced concept knowledge graph is used as the input graph, and the data enhancement method is first used to process the input graph to obtain a data enhancement graph, wherein the data enhancement graph is composed of an adjacency matrix and a node feature matrix. The data enhancement method used in this embodiment is specifically to generate a variety of positive samples by perturbing the input graph structure (such as random pruning, changing node connections, etc.), enriching the training data, and finally outputting a data enhancement graph. Then, the input graph is processed using a model enhancement technique based on graph contrast learning (such as a shuffling strategy, etc.) to obtain a model enhancement graph. Due to the complexity and redundancy of the Graph Transformer architecture, the present invention proposes a pruning-based model enhancement method to process the GraphGPS concept encoder with controllable and slight perturbations to obtain a perturbation encoder. Then the data enhancement graph is input into the GraphGPS concept encoder to obtain a first encoding representation, the data enhancement graph is input into the perturbation encoder to obtain a second encoding representation, the model enhancement graph is input into the GraphGPS concept encoder to obtain a third encoding representation, and the model enhancement graph is input into the perturbation encoder to obtain a fourth encoding representation. After the dual-view enhancement operation of the present invention, a total of four encoding representations will be obtained for each enhanced concept knowledge graph. Each of the four coded representations passes through a projection layer consisting of two layers of MLP, and four corresponding coding results are generated. Therefore, the sources of these coding results can be divided into two categories: the first category is the coding results generated by the same data enhancement method but using different encoders, and the second category is the coding results generated by the same encoder but using different data enhancement methods.

[0076] First Take an enhanced concept knowledge graph as an example, and its corresponding four encoding results can be recorded as , and It is the encoding result generated by the same data enhancement method but using different encoders. and It is the encoding result generated by the same encoder but using different data enhancement methods. Therefore, two encoding results from the same enhanced concept knowledge graph and processed by different encoders using the same data enhancement method can be used as positive sample pairs, that is, and ; Two encoding results from different enhanced concept knowledge graphs and processed by the same data augmentation method are used as negative sample pairs, for example and , and That is the The two encoding results corresponding to the enhanced concept knowledge graph. It can be seen that this method has twice as many positive samples as the previous enhancement method, which enhances the model’s ability to capture different views. Figure 1and significantly improves its performance.

[0077] Furthermore, following the architecture of the previous graph contrast learning method, the present invention calculates the NT-Xent loss for each positive sample pair, which is used to maximize the similarity of similar sample pairs and minimize the similarity of different sample pairs during the training process. The core of the NTXent loss is to use a temperature-based parameter The softmax function is used to strengthen the similarity of the positive sample pairs. A positive sample pair of an enhanced concept knowledge graph is denoted as , then the NTXent loss of a positive sample pair is The functional form is:

[0078]

[0079]

[0080] In the formula, represents the natural exponential function; Representation calculation and The cosine similarity between ; is the temperature parameter; Indicates the number of enhanced concept knowledge graphs; Representation calculation and The cosine similarity between ; For the Enhanced concept knowledge graph and The encoding results processed using the same data enhancement method; Represents matrix transpose; represents the L2 norm.

[0081] It should be noted that in step S2 of the present invention, the pruning-based model enhancement methods can be roughly divided into two categories: modification of model weights (i.e., enhancement methods based on weight pruning) and modification of model structure (i.e., enhancement methods based on layer pruning and enhancement methods based on attention head pruning). These methods can reduce the complexity of the GraphGPS concept encoder by eliminating redundant components and parameters and generate more reliable positive samples. The above three model enhancement methods will be described in detail below.

[0082] 1) Enhancement method based on weight pruning

[0083] Existing methods (such as SimGRACE) perturb the original encoder by adding Gaussian noise to the weights. However, this method is relatively challenging to control because random modification of weights can lead to the loss of key information. To address this problem, the present invention adopts the L1 unstructured pruning method as an enhancement method based on weight pruning. The focus of this method is to selectively prune the weights with the smallest L1 norm during model training, that is, to eliminate the least important parameters while retaining information that enhances model performance. This process produces a perturbed encoder. The implementation of the above-mentioned L1 unstructured pruning method belongs to the prior art and will not be repeated here.

[0084] 2) Enhancement method based on layer pruning

[0085] Since Graph Transformer introduces various complex components and parameters, dropping layers can prevent overfitting and is an effective strategy for generating positive sample pairs. It allows the model to learn multiple effective layers during training instead of over-relying on the output of a single layer. The enhancement method based on layer pruning can be expressed as follows:

[0086]

[0087] in, and For the Transformer architecture and Tier Output feature representation of the attention head; ReLU activation function. Represents the first Bernoulli random variable, which takes the value of 0 or 1 and is used to determine the first Whether the layer is retained; Represents the Transformer architecture The GraphTransformer-based encoder within the layer is used to combine local and global information for feature extraction; Indicates constant change.

[0088] 3) Enhancement method based on attention head pruning

[0089] The multi-head mechanism (MHA) is crucial for obtaining graph representation in Transformer, and each independent attention head can extract feature information from different dimensions of the graph. However, in the MHA mechanism, the information extracted by many heads has no significant effect on the final result, resulting in redundancy.

[0090] The present invention randomly discards the output of the attention head with a specific probability during the training process. The enhancement method based on attention head pruning can be expressed as follows:

[0091]

[0092] in, Represents the Transformer architecture The output representation of the layer; Represents a splicing operation; Represents the Transformer architecture Tier The Bernoulli random variable of each attention head takes the value of 0 or 1, which is used to determine the Transformer architecture. Layer Whether the attention head is retained; Represents the Transformer architecture Tier The representation of an attention head; is the linear layer weight.

[0093] S3: First, the learning data of student accounts are randomly sampled as the input of the TabDDPM model, and synthetic educational data are generated through the back diffusion process to enhance the diversity of the training data set and solve the problem of data scarcity. Then, the synthetic educational data is merged with the learning data of student accounts to form a knowledge tracing data set, and a deep knowledge tracing model (DKT) is pre-trained on the knowledge tracing data set to improve the deep knowledge tracing model's ability to predict changes in students' knowledge status.

[0094] It should be noted that in step S3 of the present invention, personalized concept recommendation must be consistent with the human knowledge system and based on the knowledge status of individual students. To this end, the present invention adopts a widely recognized deep knowledge tracking model to monitor and evaluate the knowledge status of students in real time. In the process of pre-training the deep knowledge tracking model, the data in the knowledge tracking data set is input into the deep knowledge tracking model, and first encoded by a recurrent neural network (RNN) to predict the student's mastery level of all concepts and obtain a learning history. Due to the simplicity and effectiveness of the gated recurrent unit (GRU), it is selected as a sequence encoder to optimize the effect of the model in knowledge mastery prediction, thereby enhancing the accuracy of concept recommendation, that is: the learning history is input into the gated recurrent unit (GRU) to obtain the student's knowledge status representation. It should be noted that the student's knowledge status representation changes dynamically, so it is necessary to predict the knowledge mastery at each time step.

[0095] Based on minimizing the following knowledge tracking loss Update deep knowledge tracking model parameters:

[0096]

[0097] in, Represents the deep knowledge tracking model for students The concept of time step The predicted results of mastery status; Indicates that students The concept of time step The true grasp of the situation.

[0098] It should be noted that, in step S3 of the present invention, the implementation of the TabDDPM model belongs to the prior art. In the embodiment of the present invention, the TabDDPM model is basically introduced to facilitate the understanding of those skilled in the art.

[0099] Denoising diffusion probability models (DDPM) have attracted much attention in the field of generative modeling, often outperforming other methods in generating realistic and diverse samples. The TabDDPM model provides a streamlined DDPM variant tailored for tabular data, mixed data types, including numerical and categorical features. The structure of a tabular data example is , Represents the numerical characteristics of the answer history (i.e. the numerical attributes of the table data), Respectively represent The TabDDPM model uses multinomial diffusion to represent categorical and binary features, while Gaussian diffusion is used for numerical features. The back-diffusion process in the TabDDPM model features a multi-layer neural network, and the output produced is equivalent to the original data sample without noise added in the diffusion model in terms of dimension. , which specifically represents the original numerical and categorical features of the tabular data in the TabDDPM model.

[0100] During the training process of the TabDDPM model, the goal is to minimize the following combined loss function :

[0101]

[0102] In the formula, represents the mean squared error loss of Gaussian diffusion for handling numerical attributes of tabular data; Represents the individual KL divergence loss generated by each multinomial diffusion for processing the categorical attributes of the tabular data. The sum of the KL divergence losses is divided by the number of categorical attributes. To adjust the cumulative loss of multiple diffusions to ensure comprehensive training dynamics.

[0103] S4: Input the new concept knowledge graph into the pre-trained GraphGPS concept encoder, output the structure-aware concept representation of each time step, input the new learning data into the pre-trained deep knowledge tracking model, output the student knowledge state representation of each time step, embed the new student learning data to obtain the question embedding and answer embedding of each time step, concatenate the question embedding, answer embedding, structure-aware concept representation and student knowledge state representation of each time step to obtain the final representation of each time step, and the final representation of each time step corresponding to each learning data constitutes an encoding sequence of learning data.

[0104] It should be noted that in step S4 of the present invention, the question serial numbers of the answer records in the new student learning data are sequentially passed through two MLPs to obtain question embedding, and the answers to the answer records in the new learning data are passed through two MLPs to obtain answer embedding. Further, the encoding of each learning data can be divided into four parts: structure-aware concept representation, student knowledge state representation, question embedding, and answer embedding. The above four parts are spliced ​​to obtain the final representation.

[0105] In this embodiment, Final representation of the time step For example, its form can be written as:

[0106]

[0107] in, yes Time step concept Problem embedding; yes Time step concept The answer is embedded; yes The student's knowledge state representation at time step; yes Time step concept tectonic awareness of the concept representation; Indicates the concatenation operation. Thus, a coding sequence is formed. .

[0108] S5: Input the coding sequence of the learning data into the sequence encoding model for pre-training, fine-tune the learnable concept recommendation matrix based on the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model, and construct a concept recommendation model by the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model as well as the fine-tuned concept recommendation matrix. Input the student learning data for concept recommendation into the concept recommendation model, calculate the score of each concept, generate the concept recommendation results for the online education platform for the student account, and push them to the user interface of the online education platform in a preset manner.

[0109] It should be noted that in step S5, during the pre-training process of the sequence coding model, multi-head attention calculation is first performed on the coding sequence of the learning data, and the multi-head attention calculation results are input into the feedforward network for sequence coding to obtain the sequence coding results of each time step, and the sequence coding model parameters are updated based on minimizing the sequence coding loss.

[0110] In the present invention, for the pre-training of the sequence encoding model, the same loss function as the self-supervised sequence recommendation method is used as the sequence encoding loss , this loss function covers most of the sequence-based self-supervised learning objectives, namely, the mask item prediction loss , mask segment prediction loss , mask attribute prediction loss and the associated attribute prediction loss , thus, the functional form of the above sequence encoding loss is:

[0111]

[0112] It should be noted that in step S5, during the fine-tuning process of the concept recommendation matrix, the pre-trained GraphGPS concept encoder regenerates the structure-aware concept representation, the pre-trained deep knowledge tracking model regenerates the student knowledge state representation, and after the question embedding and the answer embedding are regenerated, the regenerated question embedding, answer embedding, structure-aware concept representation and student knowledge state representation are concatenated to obtain the final representation and form a new encoding sequence, the pre-trained sequence encoding model performs sequence encoding on the new encoding sequence, regenerates the sequence encoding result, and compares the regenerated sequence encoding result with the learnable concept recommendation matrix. Multiply and calculate the scores of all concepts , and the cross entropy loss is used to update the concept recommendation matrix.

[0113] In this embodiment, the cross entropy loss The functional form is:

[0114]

[0115] in, is the amount of learning data; Is a binary indicator, indicating the concept Is it learning data? The next concept; It is learning data Medium Concept The predicted probability of Indicates the number of concepts.

[0116] It should be noted that in step S5, after the GraphGPS concept encoder, deep knowledge tracking model and sequence encoding model are pre-trained and the concept recommendation matrix is ​​fine-tuned, the pre-trained GraphGPS concept encoder, deep knowledge tracking model and sequence encoding model and the fine-tuned concept recommendation matrix form a concept recommendation model to perform the concept recommendation task. Figure 2 As shown, in the process of concept recommendation, the structure-aware concept representation output by the GraphGPS concept encoder remains unchanged, and the student learning data to be recommended for concept is passed through the deep knowledge tracking model to output the student knowledge state representation, and the student learning data to be recommended for concept is embedded. After the corresponding question embedding and answer embedding are obtained, these two embeddings and the structure-aware concept representation and the student knowledge state representation are spliced ​​to obtain the final representation of the concept recommendation and form the encoding sequence. The pre-trained sequence encoding model outputs the sequence encoding result, and then the sequence encoding result is multiplied by the fine-tuned concept recommendation matrix to calculate the score of each concept. The concepts with scores exceeding the preset score threshold are used as recommendation results, and displayed in the user interface of the online education platform in a preset recommendation mode. Learners can log in to their accounts to view these recommendation results. The recommendation mode in this embodiment can be to graphically display the push results in a pre-set functional interface module, or to push them in the form of system messages, etc.

[0117] Furthermore, when a new subject needs to be recommended, the knowledge graph corresponding to the new subject needs to be text-enhanced again by the large language model, language model and GraphGPS concept encoder to generate a new structure-aware concept representation.

[0118] The present invention will now use a specific example to demonstrate the application effect of the concept recommendation method based on large language model structure learning and knowledge representation described in S1 to S5 in the above embodiments on a specific data set, so as to facilitate understanding of the essence of the present invention.

[0119] Example

[0120] The specific implementation process of the concept recommendation method based on large language model structure learning and knowledge representation adopted in this embodiment is as described above and will not be repeated here.

[0121] In order to verify the effectiveness of the method of the present invention, this embodiment selects an actual online education data set for experimentation. The data set is derived from a real large-scale online learning platform and contains multi-dimensional data, specifically including the learning records of student accounts, concept mastery, course progress, and interactive behaviors between students. The initial learning state of each student is initialized through historical learning data and a concept recommendation model, and the performance in the subsequent learning process will continuously update its knowledge state. At the same time, the semantic associations between concepts are dynamically captured and modeled by combining the concept knowledge graph proposed by the present invention with a large language model. Through the above process, the method proposed by the present invention is used to recommend the next learning concept. The evaluation index of the experiment is AUC (i.e., the area under the ROC curve). The AUC value ranges from 0 to 1. The larger the value, the better the effect of the model.

[0122] Table 1. Concept recommendation results of the proposed method on the education dataset

[0123] Recommended Method AUC(%) DKT 65.89 AKT 77.43 GKT 72.35 The present invention 93.18

[0124] The experimental results are shown in Table 1. The proposed method has achieved significant improvement in the AUC evaluation index. Compared with the existing benchmark methods, the concept recommendation model of the present invention can more effectively capture the deep semantic relationship between concepts and make accurate recommendations based on the students' knowledge status.

[0125] The above-mentioned comparison methods include: the attention graph convolution knowledge concept recommendation model ACKRec under heterogeneous perspective, the attention knowledge tracking model AKT, and the graph-based knowledge tracking model GKT. The implementation methods of these models all belong to the existing technology.

[0126] In addition, an ablation experiment is performed to better verify the contribution of each module of the present invention. The experimental results are shown in Table 2. The specific details of the ablation experiment are as follows:

[0127] W / KG: Remove the hints of LLM from the preceding and succeeding concepts in the concept knowledge graph, directly use the concept name to generate explanations, and examine the role of the knowledge graph in eliminating concept ambiguity.

[0128] W / DC: Remove the dual-view cross-graph contrast learning method and only use single-view augmentation for concept embedding learning. This will explore whether dual-view learning plays a key role in improving the stability and performance of concept embedding.

[0129] W / TabDDPM: The TabDDPM model is not used to generate synthetic educational data, and only the original students’ learning records are used for deep knowledge tracking. This helps to understand the contribution of TabDDPM in data-scarce and cold-start scenarios.

[0130] W / Prune: Cancel the weight pruning-based enhancement method, layer pruning-based enhancement method, or attention head pruning-based enhancement method in the concept recommendation model and restore the complete Transformer structure. This will evaluate the effectiveness of the pruning strategy in reducing redundancy and computational complexity while maintaining the performance of the concept recommendation model.

[0131] Table 2. Ablation experimental results of the method of the present invention

[0132] Recommended Method AUC(%) W / KG 76.68 W / DC 81.34 W / TabDDPM 72.63 W / Prune 88.26

[0133] The results of ablation experiments show that the modules proposed in the present invention have significant effects. The main reason is that the TabDDPM model contributes the most in data scarcity and cold start scenarios. After removing this module, the generalization ability of the concept recommendation model is significantly reduced, especially when dealing with limited learning records. The process of enhancing the concept knowledge graph is crucial in eliminating concept ambiguity. The lack of preceding and succeeding concept prompts will lead to ambiguity in the concept interpretation generated by the concept recommendation model, thereby reducing the recommendation accuracy. The dual-view cross-graph contrast learning method plays an important role in enhancing the embedding stability. After removing this module, although the concept recommendation model is still available, the embedding performance is not robust enough, resulting in performance degradation. Although the removal of the pruning strategy does not significantly affect the core performance of the concept recommendation model, the computational complexity and resource consumption are significantly increased, indicating that pruning is effective in improving computational efficiency. In summary, the various modules of the present invention are crucial to the performance improvement of the concept recommendation model, and the dual-view cross-graph contrast learning method and the pruning-based model enhancement method further optimize the embedding stability and computational efficiency of the concept recommendation model.

[0134] Compared with the existing concept recommendation method, the advantages of the present invention are:

[0135] 1) Recommendation tasks usually require an adapter to transfer text embeddings to adapt to the task. Traditional adapters often use multiple multi-layer perceptron (MLP) units. The training objectives are not customized for a specific field, but are trained in combination with recommendation tasks, and the adaptation effect is not obvious. In the context of the education field, text information is closely linked to the human knowledge system. This invention adapts educational text according to the structure of human knowledge, rather than just using a simple MLP and training it in the recommendation task.

[0136] In the traditional graph contrastive learning (GCL) method, the contrastive learning object is the positive sample obtained by single-view enhancement and passed through an encoder based on a graph neural network (GNN). The present invention adopts the GraphGPS concept encoder, first uses the pre-trained language model BART to encode the text, and uses the GraphGPS concept encoder as its core encoding mechanism to deal with the limitations of long-distance dependencies and enhance the concept node representation ability through global information. Pre-training is carried out through a dual-view cross-graph contrastive learning method (data enhancement and model enhancement), and stable node representation is learned when the structure of the enhanced concept knowledge graph is disturbed, thereby capturing the deep structural information of the graph. In addition, in view of the redundant characteristics of the graph Transformer model (GT), the present invention proposes a pruning-based model enhancement method to obtain positive samples based on concept weights and knowledge structures. Compared with adding random noise in the GraphGPS concept encoder, the pruning-based model enhancement method has less impact on the GraphGPS concept encoder and can generate more reliable positive samples.

[0137] 2) Deep Knowledge Tracking (DKT) aims to use advanced deep learning methods to understand the temporal patterns in student interaction sequences and predict students' responses to new questions. The evolution of DKT is influenced by a variety of methods, including memory architecture, attention mechanism, graph-based learning, text features and forgetting, and there are also studies to improve the efficiency of DKT through data enhancement strategies. Generative Artificial Intelligence (GAI) models use deep learning technology to create synthetic data. The diffusion model, as the most advanced GAI model, breaks the long-term dominance of generative adversarial networks (GANs) in complex tasks.

[0138] Students' learning data often suffer from data scarcity and cold start problems, which will reduce the performance of DKT in concept recommendation systems. This paper introduces the most advanced GAI diffusion model, TabDDPM, which is a diffusion model that can generate synthetic educational data. It is applicable to any tabular dataset and can handle various feature types, which is better than alternative methods designed for tabular data, such as GAN-based and VAE-based methods. The educational data generated by TabDDPM effectively increases the training dataset used to train the DKT model, significantly improving the performance of DKT, especially in scenarios with little training data and more test data.

[0139] In summary, the concept recommendation method proposed in the present invention can effectively combine the students' knowledge status and the semantic relationship of concepts, and provide an efficient personalized learning path recommendation solution for the online education platform.

[0140] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A concept recommendation method based on large language model structure learning and knowledge representation, characterized in that: The following steps are involved: S1: Obtain the learning data and related concept knowledge graphs of student accounts during the learning process from the online education platform, and preprocess the learning data and concept knowledge graphs; S2: Generate enhanced text for each concept in the concept knowledge graph using a large language model, encode the enhanced text using a pre-trained language model, and obtain the final hidden vector for each concept. Update the nodes in the concept knowledge graph using the final hidden vector to obtain an enhanced concept knowledge graph. Input the enhanced concept knowledge graph into the GraphGPS concept encoder, and pre-train the GraphGPS concept encoder using a dual-view cross-graph contrast learning method to obtain a pre-trained GraphGPS concept encoder. S3: Randomly sample the learning data of student accounts as the input of the TabDDPM model, generate synthetic education data through the back-diffusion process, merge the synthetic education data with the learning data of student accounts to form a knowledge tracking dataset, and pre-train the deep knowledge tracking model on the knowledge tracking dataset; S4: Input the new concept knowledge graph into the pre-trained GraphGPS concept encoder, output the structure-aware concept representation of each time step, input the new learning data into the pre-trained deep knowledge tracking model, output the student knowledge state representation of each time step, embed the new student learning data, obtain the question embedding and answer embedding of each time step, concatenate the question embedding, answer embedding, structure-aware concept representation and student knowledge state representation of each time step, obtain the final representation of each time step, and the final representation of each time step corresponding to each learning data constitutes a coding sequence of the learning data; S5: Input the coding sequence of the learning data into the sequence encoding model for pre-training, fine-tune the learnable concept recommendation matrix based on the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model, and construct a concept recommendation model by the pre-trained GraphGPS concept encoder, deep knowledge tracing model and sequence encoding model as well as the fine-tuned concept recommendation matrix. Input the student learning data for concept recommendation into the concept recommendation model, calculate the score of each concept, generate the concept recommendation results for the online education platform for the student account, and push them to the user interface of the online education platform in a preset manner.

2. A concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: The specific process of step S1 includes: S11: Obtain the learning data and concept knowledge graph of the student account during the learning process from the database of the online education platform. The learning data of the student account includes the student's answer record, learning behavior data and learning time. The concept knowledge graph includes the name, definition, preceding concept and succeeding concept of the concept. S12: Clean the learning data, remove irrelevant or redundant learning behavior data, retain the answer records related to the learning process, then interpolate the missing learning data, and finally remove outliers and noise in the learning data to obtain the preprocessed learning data; S13: Standardize the representation of each concept in the concept knowledge graph, process the relationship between concepts in the concept knowledge graph, correct and enhance incomplete or inaccurate relationships, use each concept after the standardized representation as a node in the preprocessed concept knowledge graph, use the association information between the processed concepts as the edges in the preprocessed concept knowledge graph, and obtain the preprocessed concept knowledge graph.

3. A concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: In step S2, in the process of pre-training the GraphGPS concept encoder using the dual-view cross-graph contrast learning method, the enhanced concept knowledge graph is processed by the data enhancement method and the model enhancement technique based on graph contrast learning, and the data enhancement graph and the model enhancement graph are generated accordingly. The GraphGPS concept encoder is perturbed by the model enhancement method based on pruning to obtain a perturbation encoder. The data enhancement graph and the model enhancement graph are processed by the GraphGPS concept encoder respectively to obtain the first encoding representation and the third encoding representation respectively. The data enhancement graph and the model enhancement graph are processed by the perturbation encoder respectively to obtain the second encoding representation and the fourth encoding representation respectively. The four encoding representations are each passed through a projection layer consisting of a two-layer MLP, and each encoding representation corresponds to an encoding result. Two encoding results from the same enhanced concept knowledge graph and processed by different encoders using the same data enhancement method are used as positive sample pairs, and two encoding results from different enhanced concept knowledge graphs and processed by the same data enhancement method are used as negative sample pairs. The NT-Xent loss is calculated based on the constructed positive and negative sample pairs to update the GraphGPS concept encoder parameters.

4. A concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 3, characterized in that: The pruning-based model enhancement method is specifically one or more of a weight pruning-based enhancement method, a layer pruning-based enhancement method, or an attention head pruning-based enhancement method, among which the weight pruning-based enhancement method adopts the L1 unstructured pruning method.

5. A concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 4, characterized in that: The layer pruning based enhancement method is implemented using the first Bernoulli random variable and residual connections. The first Bernoulli random variable determines whether each layer of the Transformer architecture is retained.

6. A concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 4, characterized in that: The enhancement method based on attention head pruning is implemented using the Bernoulli random variable of each attention head of the Transformer architecture. The Bernoulli random variable of each attention head is used to determine whether each attention head of the Transformer architecture is retained.

7. The concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: In step S3, during the pre-training process of the deep knowledge tracking model, the data in the knowledge tracking dataset is input into the deep knowledge tracking model, first encoded by the recurrent neural network to obtain the learning history and input it into the gated recurrent unit to obtain the student knowledge state representation.

8. The concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: In step S4, the question serial numbers of the answer records in the new learning data are sequentially passed through two MLPs to obtain question embedding, and the answers of the answer records in the new student learning data are passed through two MLPs to obtain answer embedding.

9. The concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: In step S5, during the pre-training process of the sequence coding model, multi-head attention calculation is first performed on the coding sequence of the learning data, and the multi-head attention calculation results are input into the feedforward network for sequence coding to obtain the sequence coding results of each time step, and the sequence coding model parameters are updated based on minimizing the sequence coding loss.

10. The concept recommendation method based on large language model structure learning and knowledge representation as claimed in claim 1, characterized in that: In step S5, during the fine-tuning process of the concept recommendation matrix, the pre-trained GraphGPS concept encoder regenerates the structure-aware concept representation, the pre-trained deep knowledge tracking model regenerates the student knowledge state representation, and after the question embedding and the answer embedding are regenerated, the regenerated question embedding, answer embedding, structure-aware concept representation and student knowledge state representation are concatenated to obtain the final representation and form a new encoding sequence. The pre-trained sequence encoding model performs sequence encoding on the new encoding sequence to regenerate the sequence encoding result, which is multiplied by the learnable concept recommendation matrix, the scores of all concepts are calculated, and the concept recommendation matrix is ​​updated using the cross entropy loss.

Images