A graph collaborative filtering course recommendation method quantifying preference and hierarchical penalty

Abstract

This invention discloses a graph collaborative filtering course recommendation method based on quantified preference and hierarchical penalty, designed to improve the course recommendation performance of online learning platforms and reduce course dropout rates. The method first collects user learning log data and calculates user preference for registered courses by quantifying user learning behavior and learning duration. Next, a user-course bipartite graph is constructed, and the preference is applied as an attention weight in the neighborhood feature aggregation process of the graph convolutional network. To alleviate the problem of node oversmoothing, this invention introduces a new loss function term to measure and optimize the preservation of initial features in the embedding representation, and applies different penalty strengths to embedding representations at different levels, effectively mitigating the oversmoothing phenomenon that occurs in deep iterations of graph neural networks.

Description

Technical Field

[0001] This invention relates to the field of recommendation algorithm technology, and in particular to recommendation methods in the context of course recommendation, specifically a graph collaborative filtering course recommendation method that quantifies preference and hierarchical penalty. Background Technology

[0002] With the rapid development of internet technology, more and more learners are opting to access educational resources through online platforms. This model breaks down the time and space limitations of traditional education, enabling high-quality educational content to reach a wider audience. However, while online learning offers unprecedented convenience and flexibility, it also faces a significant challenge—high dropout rates. The completion rates for many online courses are far lower than those in traditional classroom settings, which not only impacts learners' individual development but also reduces the overall effectiveness of online education platforms.

[0003] To address the high dropout rate in online learning, existing research has employed graph neural networks (GNNs) to improve collaborative filtering models, thereby optimizing course recommendation performance. GNNs construct user-item bipartite graphs and utilize GNNs to learn embedded representations of users and items. However, current implicit feedback-based GNN methods have several limitations. Firstly, lacking explicit user ratings for registered courses, existing models rely solely on user registration behavior to construct the user-item graph. In this process, all interaction edges have uniform weights, ignoring differences in edge weights and failing to fully reflect learners' personalized preferences for different courses. Furthermore, this homogenization of edge weights simplifies complex user-course interactions, limiting the model's ability to learn deeper user and course features, thus hindering the development of better user and course representations and impacting recommendation accuracy. Secondly, during GNN training, as the number of layers increases, node feature representations tend to converge, leading to similar feature representations within the same connected component. This reduces the model's ability to capture personalized information from different nodes, affecting the performance and accuracy of the recommendation system. Summary of the Invention

[0004] To address the high dropout rate of online platform courses and the problems existing in current graph collaborative filtering recommendation models, the present invention aims to provide a graph collaborative filtering course recommendation method that quantifies preference and hierarchical penalty.

[0005] The objective of this invention is achieved through the following technical solution: a graph collaborative filtering course recommendation method that quantifies preference and hierarchical penalty, comprising the following steps:

[0006] 1. Collect learning log data of users on registered courses on the online course platform, wherein the learning log data specifically refers to when a user performs what kind of interaction on a certain course;

[0007] 2. Based on the learning log data, quantify the user's preference for registered courses;

[0008] 3. Construct a user-course node bipartite graph. If a user has registered for a course, establish an edge between the user node and the course node, and assign aggregate attention weight to the edge based on the preference degree obtained in step 2.

[0009] 4. Based on the existing simplified graph convolutional network model LightGCN, and according to the weighted bipartite graph established in step 3, the adjacency matrix of the aggregation update formula in the model is constructed, and a hierarchical penalty is added to the training loss function, so as to learn the final embedding representation of each user node and course node.

[0010] 5. Based on the final embedding representations of user nodes and course nodes obtained in step 4, calculate the inner product of user embedding and course embedding as the predicted score; for each user, sort all courses in descending order according to the predicted score, and select the top 5% of courses to form a recommendation list.

[0011] Furthermore, step 2 includes the following sub-steps:

[0012] 2.1 Define user u i In course c j During the learning process, different types of interactive actions are performed on the learning modules of the registered courses; wherein, the learning modules include instructional videos, Q&A sessions, and courseware resources; the different types of interactive action sets are as follows:

[0013]

[0014] Among them, |A(u i ,c j )| represents the user u i In course c j The total number of different interactive action types on the screen; They represent user u respectively i In course c j The 1st, 2nd, pth, |A(u) i ,c j )| Types of interaction;

[0015] The user u i In course c j The total number of different types of interactive actions is:

[0016]

[0017] in, Each represents the user u i In course c j No.1,2,p,|A(u i ,c j The total number of interactive actions;

[0018] Learning time is represented by user u i In course c j The total time spent learning is calculated by summing the timestamp differences of adjacent action types (such as playing and pausing video). i In the already registered course c j The learning time on the platform is expressed as follows:

[0019]

[0020] 2.2 Definition For user u i For course c j behavioral preference For user u i For course c j Time preference, defined as X ij For user u i For course c j Preference level;

[0021] Calculate the user u i In course c j The number of times the p-th action is performed relative to the total number of users in course c. j The ratio of the average number of the top 20% of the number of times of the p-th action.

[0022]

[0023] in, Indicates user u i In course c j The number of times the p-th action is performed. This indicates that all users are in course c j Number of times the p-th action is performed;

[0024] Calculate the user u i In course c j The proportion of the number of times the p-th type of movement is relative to the total number of times all movements are performed in the course.

[0025]

[0026] Vectorized representation as Finally, the user u is obtained i For course c j behavioral preferences

[0027]

[0028] Calculate using the following formula MeanTop 20％ (t j For all users in the same course c j The average of the top 20% in terms of online learning time;

[0029]

[0030] Among them, t ij Indicates user u i In course c j The learning time on the t j This indicates that all users are in course c j The collection of learning time on the platform;

[0031] right and Weighted summation to obtain user u i For course c j Preference X ij :

[0032]

[0033] Where β is a hyperparameter;

[0034] Following the above method, the preference for all users and all courses is calculated, and finally, a user-course preference matrix X is constructed.

[0035] Furthermore, the user-course preference matrix is ​​represented as X∈R |U|×|I| Where |U| represents the total number of users, |i| represents the total number of courses, R represents the set of real numbers, and X represents the total number of courses. ij Indicates user u i For course c j Preference level.

[0036] Furthermore, step 4 includes the following sub-steps:

[0037] 4.1. Perform row normalization on the user-course preference matrix X using the softmax function:

[0038]

[0039] in, This represents the row-normalized user-course preference matrix;

[0040] Construct the adjacency matrix A of the user-course node bipartite graph:

[0041]

[0042] The user embedding layer and the course embedding layer are randomly initialized using one-hot encoding, and the 0th layer embedding representation is: E (0) ∈R (|U|+|I|)*T T is the dimension of the embedding layer; then, the embedding representation of each layer is updated by simplifying GCN aggregation as follows:

[0043] E (k+1) =AE (k)

[0044] Where k represents the number of layers in the graph convolutional network; the embedding representation of the last layer node is directly used as the final embedding of the user-course. and

[0045] 4.2 In network training, BPR loss is used as part of the total loss, and the objective function is expressed as:

[0046]

[0047] in, X represents paired training data. + X represents the interaction pair between a user and a registered course. - σ represents the interaction pair between the user and the remaining unregistered courses, Θ represents all trainable model parameters, and λ controls the strength of L2 regularization to prevent overfitting.

[0048] Define a user set U and a course set I, containing the number of users |U| and the number of courses |I|, respectively, and let e represent the initial feature embedding vector of a node. The final embedding vector, after being learned by the network, is measured by the following formula to determine the degree to which the initial features are preserved in the final embedding representation:

[0049]

[0050] in:

[0051]

[0052] From the above formula, we can see that Save avg The smaller the value, the better the initial features are preserved in the final embedded representation; among which, This represents the average degree of retention of the initial features in the final embedding vector of the user set. e represents the average retention of initial features in the final embedding vector of the course set. i e j Let represent the initial feature vectors of user i and user j, or course i and course j, respectively. Diff represents the final embedding vectors obtained after the network learns from user i and user j, or course i and course j, respectively. cos (E i E j The cosine similarity difference between two sets of embedding vectors is used to quantify the degree of change between the final embedding representation and the initial embedding representation.

[0053] Therefore, the total loss function is expressed as:

[0054] L = L BPR +αL save

[0055] Where α is a hyperparameter, and L save The initial feature retention rate of each layer of user and course nodes, calculated using GCN during network training, is incorporated into the loss, as shown below:

[0056]

[0057] in, Let represent the average retention of initial features in the embedding vectors of the user set and course set in the l-th layer of the network, respectively. These represent the penalty coefficients for the user and course item at level l, respectively. and All values ​​are This means that the penalty coefficient decreases as the layer depth increases.

[0058] The beneficial effects of this invention are as follows: User preferences are indirectly quantified by analyzing user behavior and learning duration in learning logs, thereby optimizing the aggregation weights in the user-course graph and enhancing the model's ability to capture personalized preferences. Simultaneously, by introducing a metric for the retention of initial features and incorporating it into the loss function, the problem of overly smooth deep nodes is effectively alleviated, thus improving the accuracy and personalization of the recommendation system and helping to reduce dropout rates in online courses. Attached Figure Description

[0059] Figure 1 This is a flowchart of a graph-based collaborative filtering course recommendation method that quantifies preference and hierarchical penalty.

[0060] Figure 2 It is a graph convolutional network model with preference weights and hierarchical penalties. Detailed Implementation

[0061] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0062] like Figure 1 As shown in the figure, the graph collaborative filtering course recommendation method based on quantified preference degree and hierarchical penalty provided by the embodiment of the present invention includes the following steps:

[0063] Step 1: Collect learning log data of users on the online course platform regarding their registered courses, such as when a user performed what kind of interaction with a course.

[0064] Step 2: Based on the learning log data recorded in Step 1, statistically analyze and quantify the user's preference for the courses they have registered for.

[0065] (2.1) Online courses generally consist of multiple modules such as instructional videos, Q&A sessions, and courseware resources. (User u) i In the already registered course c j During the learning process, students will interact with the relevant learning modules in different ways.

[0066] Define user u i In course c j A collection of different types of interactive actions:

[0067]

[0068] Among them, |A(u i ,c j )| represents the user u i In course c j The total number of different interactive action types on the screen; They represent user u respectively i In course c j The 1st, 2nd, pth, |A(u) i ,c j )| Types of interaction.

[0069] User u i In course c j The total number of interactions of different types is represented as follows:

[0070]

[0071] in, Each represents the user u i In course c j No.1,2,p,|A(u i ,c j The total number of interactive actions.

[0072] Learning time is represented by user u i In course c j The total time spent learning is calculated by summing the timestamp differences of adjacent action types (such as playing and pausing video). i In the already registered course c j The learning time on the platform is expressed as follows:

[0073]

[0074] (2.2) Definition For user u i For course c j behavioral preference For user u i For course c j Time preference, X ij For user u i For course c j Preference level.

[0075] Calculate using the following formula Indicates user u i In course c j The ratio of the number of times the p-th action is performed to the average number of times the p-th action is performed by the top 20% of all users in the same course, and this ratio is minimized by taking the minimum value of 1. Between 0 and 1, and ultimately ensure It is also between 0 and 1.

[0076]

[0077] in, Indicates user u i In course c j The number of times the p-th action is performed. This indicates that all users are in course c j The number of times the p-th action is performed.

[0078] Calculate user u i In course c j The proportion of the number of times the p-th type of movement is relative to the total number of times all movements are performed in the course.

[0079]

[0080] Vectorized representation User u can be obtained i For course c j behavioral preferences

[0081]

[0082] Calculate using the following formula MeanTop 20％ (t j For all users in the same course c j The average study time is in the top 20%.

[0083]

[0084] Finally, and Weighted summation, as shown below, where β is a hyperparameter, yields the user u. i For course c j Preference X ij :

[0085]

[0086] Finally, calculate the preference degree of all users for all courses and represent it as a user-course preference matrix X.

[0087] Step 3: Construct a user-course node bipartite graph. If a user has registered for a course, an edge is established between the user node and the course node. The preference degree calculated in Step 2 is used to assign aggregate attention weights to this edge. The user-course preference degree matrix is ​​described as X∈R. |U|×|I| Where |U| is the total number of users, |I| is the total number of courses, R is the set of real numbers, and X is the total number of courses. ij For user u i For course c j Preference level.

[0088] Step 4: Using the simplified graph convolutional network model with preference weights and hierarchical penalties proposed in this invention, learn the final embedding representation of each user node and course node. The network model is as follows: Figure 2 As shown, the network includes a user / course embedding layer, an aggregation update layer, and a prediction layer. The network training details are as follows:

[0089] (4.1) First, the user-course preference matrix X is row-normalized using the softmax function:

[0090]

[0091] Therefore, the adjacency matrix A of the user-course node bipartite graph is constructed as follows:

[0092]

[0093] Depend on Figure 2The user embedding layer and course embedding layer, represented in the figure, are randomly initialized using one-hot encoding. The 0th layer embedding representation is: E (0) ∈R (|U|+|I|)*T T represents the dimension of the embedding layer. Then, a simplified GCN with three layers (removing the feature transformation matrix and non-linear activation function) is used to aggregate and update the embeddings of each layer. Unlike traditional simplified GCNs, which typically use averaging or summing when aggregating features of neighboring nodes, the model in this invention weights and aggregates the features of neighboring nodes based on the user's preference for the course. In other words, when updating its embedding representation, a node pays more attention to neighboring nodes with stronger connections, resulting in more accurate and personalized information delivery and enhanced model performance. The specific aggregation and update formula is as follows:

[0094] E (k+1) =AE (k)

[0095] Where k represents the number of layers in the graph convolutional network; the embedding representation of the 3rd layer node is used as the final embedding representation of the user-course. and

[0096] (4.2) In network training, BPR loss was used as part of the total loss. Its objective function aims to maximize the difference between the predicted values ​​of positive samples (observed interactions) and negative samples (unobserved interactions). The objective function is:

[0097]

[0098] in X represents paired training data. + X represents the interaction pair between a user and a registered course. - σ represents the interaction pair between the user and the remaining unregistered courses, Θ represents all trainable model parameters, and λ controls the strength of L2 regularization to prevent overfitting.

[0099] In the research and practice of recommender systems, although graph convolutional networks (GCNs) are widely used due to their advantages in processing graph-structured data, as the number of network layers increases, GCNs may encounter a problem called "over-smoothing" when processing node embeddings. Over-smoothing refers to the phenomenon where, as the number of network layers increases, the embedding representations of different nodes tend to be consistent, making it impossible to effectively distinguish the characteristics of different nodes, thus impairing the accuracy and personalized recommendation capabilities of the recommender system. To address this problem, this invention proposes a novel method that alleviates over-smoothing by introducing an index to measure the degree of initial feature preservation in the final embedding. The core idea of ​​this method is to focus not only on the smoothness of node embeddings during network training but also on preserving the initial feature information of nodes. By preserving the initial feature information, it ensures that nodes do not become over-smooth. Furthermore, given the significant differences in the distribution of the final embedding representations of users and items, the user and item representation matrices are calculated separately, such as... Figure 2 As shown, the retention loss is calculated for the user and item representation matrices of each layer. Specifically, the user set U and course set I are defined to contain the number of users |U| and the number of courses |I|, respectively. Let e ​​denote the initial feature embedding vector of a node. Let represent the final embedding vector after network learning. The following formula is used to calculate the degree of preservation of the initial features in the final embedding representation:

[0100]

[0101] in:

[0102]

[0103]

[0104] From the above formula, we can see that Save avg The smaller the value, the better the initial features are preserved in the final embedded representation; among which, This represents the average degree of retention of the initial features in the final embedding vector of the user set. e represents the average retention of initial features in the final embedding vector of the course set. i e j Let represent the initial feature vectors of user i and user j, or course i and course j, respectively. Diff represents the final embedding vectors obtained after the network learns from user i and user j, or course i and course j, respectively. cos (E i E j The cosine similarity difference between the two sets of embedding vectors is used to quantify the degree of change between the final embedding representation and the initial embedding representation.

[0105] To mitigate the oversmoothing of user and item representations in simplified GCN collaborative filtering models based on the removal of feature transformation matrices and nonlinear activation functions, this invention directly incorporates the aforementioned metrics for retention into the loss function as part of model training. By adjusting the loss function, the model not only pursues smoothness in node embeddings during training but also retains as much of the initial feature information of nodes as possible. This ensures that even in deep networks, the model maintains sensitivity to the characteristics of different nodes, thereby improving the performance of the recommendation system. Furthermore, in graph collaborative filtering models, the smoothness of deep representations is a double-edged sword. On one hand, smoothness helps the model capture global relationships between nodes in the graph, thus improving the overall performance of the recommendation system. On the other hand, excessive smoothness leads to homogenization of node features, making it difficult for the model to distinguish the unique attributes of different nodes, which is detrimental to personalized recommendations. Therefore, considering the contribution of embedding smoothness to the effectiveness of graph collaborative filtering models, while limiting the oversmoothing of the final model representation, it is also necessary to ensure appropriate smoothness in deeper representations. This invention introduces a hierarchical penalty mechanism in the retention loss function. The core idea is to assign different penalty coefficients to network layers of different depths to control the smoothness of each layer. Specifically, deeper network layers receive lower penalty coefficients. This method supplements the initial embeddings into the embedding representation of each layer, thus mitigating the oversmoothing problem. The total loss function can be expressed as:

[0106] L = L BPR +αL save

[0107] Where α is a hyperparameter, and L save The initial feature retention rate of each layer of user and course nodes, calculated using GCN during network training, is incorporated into the loss, as shown below:

[0108]

[0109] in, Let represent the average retention of initial features in the embedding vectors of the user set and course set in the l-th layer of the network, respectively. These represent the penalty coefficients for the user and course item at level l, respectively. and All values ​​are This means that the penalty coefficient decreases as the layer depth increases.

[0110] Step 5: Based on the user embeddings and course embeddings learned from the network in Step 4, calculate the inner product of the user embeddings and course embeddings as the predicted score. For each user, sort all courses in descending order according to the predicted score, and select the top 5% of courses to form a recommendation list.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A graph collaborative filtering course recommendation method quantifying preference degrees and hierarchical penalties, characterized in that, Includes the following steps: (1) Collect learning log data of users on registered courses on the online course platform, wherein the learning log data specifically refers to when a user performs what kind of interaction on a course; (2) Based on the learning log data, quantify the user's preference for the registered courses; (3) Construct a user-course node bipartite graph. If a user has registered for a course, establish an edge between the user node and the course node, and assign aggregate attention weight to the edge according to the preference degree obtained in step (2). (4) Based on the existing simplified graph convolutional network model LightGCN, and according to the weighted bipartite graph established in step (3), the adjacency matrix of the aggregation update formula in the model is constructed, and a hierarchical penalty is added to the training loss function, so as to learn the final embedding representation of each user node and course node. (5) Based on the final embedding representation of the user node and course node obtained in step (4), calculate the inner product of the user embedding and the course embedding as the predicted score; for each user, sort all courses in descending order according to the predicted score, and select the top 5% of courses to form a recommendation list. Step (2) includes the following sub-steps: (2.1) defining a user in the course learning process, a set of different types of behavior interaction actions are performed on the learning blocks of the registered course; wherein the learning blocks include teaching videos, question answering, and courseware resources; the set of different types of interaction actions are as follows: ； in, Indicates the user In the course The total number of different interactive action types on the screen; Representing users respectively In the course The first Types of interaction; The user In the course The total number of different types of interactive actions is: ； in, Each represents the user In the course No. The total number of interactive actions; The user Registered courses The learning time on the platform is expressed as follows: ； (2.2) Definition For users For the course behavioral preference For users For the course Time preference, defined For users For the course Preference level; Calculate the user In the course The first The number of actions relative to all users in the course Upper The ratio of the average number of the top 20% of the number of times a certain action is performed. : ； in, Indicates user In the course The first Number of times of each action This indicates that all users are in the course Upper Number of times of each action; This indicates that all users are in the course Upper The average number of times the top 20% of the action count set; Calculate the user In the course The first The proportion of this type of movement to the total number of movements in the course. : ； Vectorized representation as , Ultimately, the user is satisfied. For the course behavioral preferences : ； Calculate using the following formula ,in For all users in the same course The average of the top 20% in terms of online learning time; ； in, Indicates user In the course Study time on This indicates that all users are in the course The collection of learning time on the platform; right and Weighted summation to obtain the user's result For the course Preference : ； in, For hyperparameters; Following the above method, the preference scores of all users and all courses are calculated, and a user-course preference matrix is ​​finally constructed. .

2. The graph collaborative filtering course recommendation method based on quantified preference and hierarchical penalty according to claim 1, characterized in that, The user-course preference matrix is ​​represented as follows: ;in, Indicates the total number of users. This indicates the total number of courses. Represents the set of real numbers. Indicates user For the course Preference level.

3. The graph collaborative filtering course recommendation method based on quantified preference and hierarchical penalty according to claim 1, characterized in that, Step (4) includes the following sub-steps: (4.1) User-course preference matrix Use the softmax function for row normalization: ； in, This represents the row-normalized user-course preference matrix; Construct the adjacency matrix of the user-course node bipartite graph : ； The zeroth layer embedding is randomly initialized using one-hot ID encoding by the user embedding layer and the course embedding layer, and is represented as follows: T' is the dimension of the embedding layer; then, the embedding representation of each layer is updated by simplifying GCN aggregation as follows: ； Where k represents the number of layers in the graph convolutional network; the embedding representation of the last layer node is used as the final embedding of the user-course. and ; (4.2) In network training, BPR loss is used as part of the total loss, and the objective function is expressed as: ； in, Represents paired training data. This represents the interaction between the user and the registered course. This represents the interaction pairs between the user and the other unregistered courses. For the sigmoid function, This represents all trainable model parameters. Control the strength of L2 regularization to prevent overfitting; Define a user set U and a course set I, each containing the number of users. and number of courses , This represents the initial feature embedding vector of the node. The final embedding vector, after being learned by the network, is measured by the following formula to determine the degree to which the initial features are preserved in the final embedding representation: ； in: ； ； ； ； From the above formula, it can be seen that, The smaller the value, the better the initial features are preserved in the final embedded representation; among which, This represents the average degree of retention of the initial features in the final embedding vector of the user set. This represents the average retention rate of the initial features in the final embedding vector of the course set. , Representing users respectively and users The initial feature vector, , Each represents a course and courses The initial feature vector, , Representing users respectively and users The final embedding vector obtained after network learning , Representing courses and courses The final embedding vector obtained after network learning and These represent the cosine similarity difference between the two sets of embedding vectors in the user set and the course set, respectively, and are used to quantify the degree of change between the final embedding representation and the initial embedding representation in the user set and the course set. Therefore, the total loss function is expressed as: ； in, For hyperparameters, and The initial feature retention rate of each layer of user and course nodes, calculated using GCN during network training, is incorporated into the loss, as shown below: ； in, , They represent the network's first and second generations respectively. The average retention rate of initial features in the embedding vectors of the layered user set and course set. , They represent the first The penalty coefficients for the corresponding users and course items are layered. and All values ​​are This indicates that the penalty coefficient decreases as the layer depth increases.

Images