Knowledge transfer-based long-tail graph comparative learning enhancement method

Abstract

The invention belongs to the technical field of comparative learning, and particularly relates to a knowledge transfer-based long-tail graph comparative learning enhancement method, which comprises the steps of long-tail enhancement, comparative learning and multi-task training. The long-tail enhancement, the comparative learning and the multi-task training are in data transmission connection in sequence, and the long-tail enhancement further comprises an automatic discarding module, a knowledge migration module and a generative adversarial learning module. According to the mode, representation of a tail node is enhanced by supplementing predicted neighbor information, an automatic deletion strategy is introduced, neighborhood information needed by the model is generated through the process of generating a pseudo tail node by a real head node, and a knowledge migration module is designed. The method is used for enabling a tail node to learn enough knowledge so as to enhance neighborhood information of the tail node, and finally a graph structure is enriched and perfected.

Description

Technical Field

[0001] The present invention belongs to the technical field of contrastive learning, and in particular relates to a long-tail graph contrastive learning enhancement method based on knowledge transfer. Background Art

[0002] In the real world, objects are often connected to each other in complex network structures, also known as graphs. For example, the interactions between users and products (such as clicks or purchases) form an e-commerce network, while research papers cite each other to form a citation graph. The popularity of graph structures has attracted many active studies on graph-based tasks, such as product recommendations using link prediction in e-commerce networks and node classification of papers into different topics in citation graphs. CF, as a classic method, represents the embedding of users and items through observed user-item interactions, which plays an important role in personalized preference prediction.

[0003] Graph convolutional networks have shown good results in recommendation systems because they can effectively utilize high-order relationships in graph data. However, graphs in many fields follow a long-tail distribution in terms of node degree, that is, a considerable number of nodes are tail nodes with smaller degrees, which will lead to the problem of sparse graph data. In order to solve the problem of sparse tail node data, GCN-based recommendation methods use contrastive learning to introduce self-supervision signals. Although some existing methods are effective, they treat all nodes uniformly and lack consideration of the significant differences between the head and tail nodes, that is, they do not distinguish between the head hot nodes and the tail unpopular nodes in the long-tail distribution. This may cause the model to over-focus on the head nodes when recommending and ignore the uniqueness and potential value of the tail nodes, which may lead to uneven representation distribution, which is a key factor affecting the performance of the contrastive learning method. In response to the data sparsity problem caused by the long tail, the present invention proposes a long-tail graph contrastive learning enhancement method based on knowledge transfer. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a long-tail graph contrast learning enhancement method based on knowledge transfer, including: long-tail enhancement, contrast learning and multi-task training;

[0005] The long-tail enhancement, contrastive learning and multi-task training are sequentially connected in data transmission, the long-tail enhancement further includes an automatic discarding module, a knowledge transfer module and a generative adversarial learning module, and the automatic discarding module, the knowledge transfer module and the generative adversarial learning module are sequentially connected in data transmission;

[0006] The generative adversarial learning module includes a graph convolutional network encoder with skip-connected graph convolutional layers and a graph autoencoder based on contrastive learning.

[0007] Beneficial effects of the present invention:

[0008] The present invention designs a learnable long-tail enhancement mode, which enhances the representation of the tail node by supplementing the predicted neighbor information. In order to make this enhancement mode learnable, an automatic deletion strategy is introduced. The neighborhood information required by the model is generated by the process of generating a pseudo tail node from the real head node, and a knowledge transfer module is designed to enable the tail node to learn enough knowledge to enhance the neighborhood information of the tail node, ultimately enriching and improving the graph structure.

[0009] In order to maximize the consistency of representation between different views, contrast loss is used as a discriminator, and it is innovatively combined with InfoNce to further improve the consistency between views. Because through research, it is found that in the use of InfoNce loss, the positive samples selected are usually two enhanced views of a data point, and the negative samples are other samples from the data set. The goal of loss calculation is to distinguish positive sample pairs from negative samples in the noise distribution, which may cause the representation learned by the model to be not robust enough or the discrimination is not high. In the contrast loss of the proposed GRACE module, the selected positive samples are representations of two different views of the same node, and the negative samples are representations of different nodes. The purpose of loss calculation is to make the similarity between positive samples higher than the similarity between any negative samples, which has better generalization ability. Therefore, by combining the two methods, we can draw on their advantages and complement each other's deficiencies, thereby enhancing the model's learning of positive samples and further improving the quality of representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is the overall flow chart of the present invention;

[0011] Figure 2 This is the long-tail enhancement flow chart of the present invention;

[0012] Figure 3 It is the overall framework diagram of the model of the present invention;

[0013] Figure 4 This is a performance comparison data diagram of different τ of the present invention;

[0014] Figure 5 This is a performance comparison data diagram of different τ of the present invention;

[0015] Figure 6 This is a performance diagram of the present invention on the yelp2018 dataset under different degree thresholds k;

[0016] Figure 7 This is a performance diagram of the present invention on the yelp2018 dataset under different degree thresholds k;

[0017] Figure 8 is the data sparsity data graph of the present invention;

[0018] Fig. 9 The figure is a performance comparison data diagram of different layers of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] like Figure 1-Figure 3 As shown, an embodiment of the present invention provides a long-tail graph contrast learning enhancement method based on knowledge transfer, including: long-tail enhancement, contrast learning and multi-task training;

[0021] The long-tail enhancement, contrastive learning and multi-task training are sequentially connected in data transmission, the long-tail enhancement further includes an automatic discarding module, a knowledge transfer module and a generative adversarial learning module, and the automatic discarding module, the knowledge transfer module and the generative adversarial learning module are sequentially connected in data transmission;

[0022] The generative adversarial learning module includes a graph convolutional network encoder with skip-connected graph convolutional layers and a graph autoencoder based on contrastive learning.

[0023] Specifically: For recommendation methods based on graph convolutional networks (GCNs), they model the interaction data between users and items as a bipartite graph, aiming to capture higher-order information through graph neural networks. For example, GC-MC applies GCN to user-item graphs and uses convolutional layers to explore direct connections between users and items. PinSage is an industrial solution that generates embeddings of items in Pinterest using effective random walks and graph convolution techniques. NGCF effectively models higher-order connectivity by explicitly injecting collaborative signals into the embedding process. In contrast, LightGCN simplifies the design of GCN, removes feature transformations and nonlinear activations, and improves its applicability in recommendation tasks. However, all of these methods rely on supervised learning paradigms for training and rely heavily on labeled data. Since the observed interaction data is very sparse relative to the entire interaction space, this may not be enough to obtain an effective representation. Therefore, when using these models for recommendations, it is necessary to consider how to deal with this sparsity and how to better utilize limited observed data;

[0024] Contrastive learning, as a self-supervised learning strategy, is often used to solve the problem of data sparsity. Recently, in the research field of recommendation systems, many studies have begun to introduce contrastive learning techniques. For example, MixCon3D combines the information of 2D images and 3D point clouds to improve the effect of contrastive learning. LightGCL is a simple and effective graph contrastive learning method used to improve the performance of recommendation systems. In related work, the SGL model was proposed, which performs data enhancement in the convolutional layer by randomly deleting nodes and edges and random walks, and then uses contrastive learning for node embedding. In addition, the proposed CL4CTR is a contrastive learning framework for CTR (click-through rate) prediction. It mainly solves the problem of neglecting feature representation learning in traditional CTR prediction work, especially when dealing with low-frequency features. CL4CTR directly generates high-quality feature representations by introducing self-supervised learning, thereby improving the performance of CTR prediction;

[0025] GCN-based recommendation Let G = (V, A) represent the user-item bipartite graph of the GCN-based recommendation method, where V = U ∪ I, U = {u} represents the user set, I = {i} represents the item set, and the observed user-item interaction can be expressed as the interaction matrix R ∈ R |U|×|I| In the interaction matrix, if there is an interaction between user u and item i, the element r in the matrix u,i =1, otherwise r u,i = 0, A∈R n×n is the adjacency matrix, defined as:

[0026]

[0027] For each node i∈I, let N i represents the set of neighbor nodes of node i, N i Also called the neighborhood of i, its absolute value |N i | is also called the degree of i. The GCN-based recommendation method uses formula (2) to aggregate users and items in the bipartite graph G and generate the embedding representation of the next layer.

[0028]

[0029] Where D∈R n×n is a diagonal matrix of G. Each entry in the diagonal matrix D ii represents the number of non-zero values ​​in the i-th row of matrix A, H (l) ∈R n×d represents the d-dimensional node embedding matrix after the l-th graph convolutional layer, H (0) represents the initial node embedding matrix. Finally, the node embedding matrices output by the {0,1,…,L} layers are combined, that is, H = f pooling([H (0) ;H (1) ;…;H (L) ]) to generate preference scores between users and items for recommendation, where f pooling(·) represents the average pooling operation;

[0030] Long-tail distribution GCN-based recommendations hope to improve performance through the rich structural information in graph data. However, for nodes with small degrees, their number of neighbors is usually very small, which makes them unable to provide sufficient information, so the performance of these nodes is unsatisfactory. In this paper, in order to study the long-tail distribution in the graph, let V head and V tail They represent the head node and the tail node respectively. For a certain threshold K, the tail node is defined as the node whose degree does not exceed K, that is, V tail ={v:|N v ≤K|}, and the head node is the complement of the tail node, that is, V head ={v:|N v >K|}. According to the above description, V=V head ∪V tail and

[0031] The automatic discarding module is equipped with a trainable deletion strategy to reduce the distribution deviation between the enhanced data samples and the actual tail node samples.

[0032] Self-supervised learning is an emerging technology that has attracted much attention. It has achieved remarkable success in many fields, including computer vision, natural language processing, and graph learning, by leveraging unlabeled data. Inspired by the successful applications in these fields, self-supervised learning has also begun to show great potential in the field of recommender systems. For example, S3-Rec adopts four self-supervised objectives to learn the correlation between attributes, items, subsequences, and sequences by maximizing mutual information. Similarly, CLCRec further adopts a contrastive learning method to alleviate the cold start problem by maximizing the mutual dependence between item content and collaborative signals

[30] . In addition, a promising research direction is to integrate contrastive learning into graph-based recommender systems to deal with the label sparsity problem of self-supervised signals. For example, SGL uses node dropping, edge dropping, and random walks to generate different views on the user-item interaction graph and maximize the consistency between them. NCL combines the information of structure and semantic neighbors to enhance graph-based collaborative filtering. SimGCL generates contrastive views by adding uniform noise in the embedding space instead of relying on augmentation methods based on graph data. LAGCL uses a local adversarial graph convolutional layer and uses node-level adversarial training to enhance the representation ability of graph neural networks. These studies demonstrate the potential of self-supervised learning in recommendation systems and can effectively learn in the absence of labeled data;

[0033] The operation method of the automatic discard module is as follows:

[0034] Neighbors are sorted according to edge weights, and then the first T neighbors are selected according to importance weights. The importance weight S is calculated as follows:

[0035]

[0036] Where: S∈R nxn , Ws∈R dxd represents the trainable parameter, ⊙ represents the element product, H (l) ∈R n×d represents the d-dimensional node embedding matrix after the graph convolution layer of the l-th graph convolutional network encoder, H (0) represents the initial node embedding matrix, A∈R nxn Represents the adjacency matrix.

[0037] Specifically: In order to simulate the real tail node, a parameter k is set i To divide:

[0038]

[0039] Finally, the inspiration for the top-rank selection method is to select the top-K neighbors according to their importance weights to construct a new adjacency matrix. This matrix will be used for graph aggregation of the tail nodes. The new adjacency matrix The definition is as follows:

[0040]

[0041] Where δ is the control matrix The hyperparameter of smoothness, N i is the neighborhood corresponding to node i

[0042] The knowledge transfer module uses the neighbor information discarded when generating the pseudo tail node to predict the missing neighbor information of the real tail node. The knowledge transfer module reconstructs the tail node based on the learning of the discarded neighbor information to generate a pseudo head node.

[0043] The principle of the knowledge transfer module is as follows:

[0044] Assume h i represents the embedding vector of head node i, let h Ni represents the neighborhood N of i i By performing a pooling operation on the embedding vectors of all neighboring nodes of node i, the neighborhood migration on node i is expressed as:

[0045] h i +r i ≈h Ni

[0046] Where: r i represents the transfer vector that the learnable model can "predict";

[0047] Learn a model to predict the migration vector of the head node in the neighborhood migration on node i, and for the tail node i∈V tail , you can use m i To represent the missing information, through its ideal neighborhood The embedding vector and the observed neighborhood N i The difference of the embedding vector is formed, that is:

[0048]

[0049] Among them: Ideal neighborhood represents a potential theoretical neighborhood, h Ni represents the neighborhood N of i i The embedding vector of .

[0050] Specifically, it includes all nodes that may be connected to node i, not just the observed neighbor nodes. This means that the ideal neighborhood It has a wider scope and may include unobserved nodes or potential connection relationships. By considering the ideal neighborhood, the relationship between nodes, information transfer, and network structure can be more comprehensively understood and modeled.

[0051] First, we need to predict the unknown ideal neighborhood Specifically, a shared migration model can be used to process the head node and the tail node relative to their ideal neighborhood. For the head node, its observed neighborhood is considered as the ideal neighborhood, and learn how to predict the migration vector r i For the tail nodes, the migration model learned from the head nodes is used to construct their ideal neighborhood representations Among them, the migration vector ri is predicted on the head node by applying the migration model.

[0052] Next, we can use the predicted ideal neighborhood representation To infer the missing neighborhood information of the tail node i. Specifically, the neighbor representation of the tail node can be calculated based on the given neighbor information, thereby filling in the missing neighborhood information. This realizes the process of transferring knowledge from the head node to the tail node.

[0053]

[0054] Therefore, by sharing the migration model and predicting the ideal neighborhood representation, the missing neighborhood information between nodes can be effectively predicted and completed, thereby improving the accuracy and completeness of relationship modeling and information transfer.

[0055] Through the above discussion, we will start further specific work. We use and learn the neighborhood information obtained from the head node and pass it to the tail node through the knowledge transfer module, hoping to predict the missing neighborhood information in the aggregation process and then supplement the observed neighborhood information. Specifically, in each layer of the model, the concept of migration is adopted, so the embedding vector of the head node i in the lth layer should satisfy:

[0056]

[0057] in, is the migration vector of node i at layer l, It represents the ideal neighborhood of the lth layer.

[0058] In particular, for a given head node i, It can be approximated as the neighborhood N observed in the same layer i Representation therefore.

[0059]

[0060] Here It can be represented by the average pooling of all nodes in the observed domain, namely:

[0061]

[0062] On the other hand, through the knowledge transfer module, the learning of the discarded domain information reconstructs the tail node and generates pseudo head nodes, which are compared with the corresponding head nodes to enhance the prediction and utilization of the missing neighborhood information. In order to achieve robustness for the tail node embedding, the key idea is to reveal the missing neighborhood information. The transferable migration concept is applied in each layer. Specifically, in the lth layer, the missing domain information of the tail node i (whether it is a real tail node or a pseudo tail node) can be predicted as:

[0063]

[0064] In this way, the missing neighborhood information of the tail node can be predicted and filled, which enhances the model's ability to model the relationship between nodes and improves its robustness to unobserved neighborhood information.

[0065] In order to constrain the missing information, the loss of missing information constraint is introduced. Since it is assumed that there is no missing information on the head node, it is hoped that the missing information on the head node is close to zero during prediction. To achieve this goal, the following loss function is designed to constrain the missing information:

[0066]

[0067] Where V tr Represents a given set of training nodes, if P i =1 is the head node, P i = 0 is the tail node. Such a constraint loss can help improve the model's modeling ability for head nodes and missing information, strengthen the completion and prediction of missing information, and further improve the effect of relationship modeling and information transmission.

[0068] The generative adversarial learning module enables long-tail enhancement to be learned effectively. The representation distribution of the real tail node should match the representation distribution of the pseudo tail node produced by the automatic discarding module, which is calculated as follows:

[0069]

[0070] in, represents the representation after the lth layer is automatically discarded, and Represent the sparse matrices of nodes i and j respectively, represents the neighborhood corresponding to node i, It represents the representation after the l-1th layer is automatically discarded;

[0071] For the adversarial constraint of the tail node, the following loss function is adopted:

[0072]

[0073] The loss function for the adversarial constraint of the head node is as follows:

[0074]

[0075] Among them, L tail-disc represents the adversarial constraint loss function of the tail node, V represents the node set, V tail represents the set of tail nodes, CROSSENT(.) represents the cross entropy function, ζ(.) represents the indicator function, f d (:θ d ) indicates the use of θ d Parameterized discriminant function; L head-disc represents the adversarial constraint loss function of the head node, Represents the representation after which the representation is automatically discarded.

[0076] Specifically: It is necessary to ensure that the distribution representation of the pseudo-head node enhanced by the knowledge transfer module matches the distribution representation of the real head node. To achieve this goal, a generative adversarial network is chosen. The discriminator distinguishes pseudo-head / tail nodes from real head / tail nodes based on node representation, while the goal of the generator is to provide information so that the discriminator consistently classifies it as a real node. Here, the output layer is regarded as a generator and is allowed to compete with the discriminator in the learning process.

[0077] The contrastive learning uses the contrastive loss to obtain a loss function which is jointly optimized to obtain the final task objective function.

[0078] The final task objective function is:

[0079] L zcl =αL cl +βJ cl

[0080] Among them: α, β represent hyper parameters, L cl , J cl Respectively represent the objective functions for InfoNce and GCN self-supervised tasks.

[0081] Next, we need to learn the SimGC way to create different perspectives by slightly rotating the refined node embeddings in space. This approach not only preserves the original information, but also enhances the robustness of the model by introducing InfoNCE loss as an additional self-supervised learning task. The specific method is as follows:

[0082]

[0083] in and represents the noise vector added to the lth layer, which needs to satisfy ||Δ||2=ε and You can use ε to control and Relative to The rotation angle of and The noise is always in the same super-octant, so the introduced noise does not cause significant deviation. Noise is added in each convolutional layer and the output of all layers is averaged to obtain the final embedding of the node. To simplify the representation, h i represents the final node embedding after L layers, and and represents the two views generated by this method. Next, InfoNce and the designed GRACE are combined as additional self-supervised tasks to improve robustness;

[0084] For InfoNce, after obtaining the generated view of each node, the contrast loss is used to maximize the consistency of positive pairs and minimize the consistency of negative pairs:

[0085]

[0086] Where s(·) represents the cosine similarity function, which calculates the similarity between two vectors; τ is a hyperparameter. Similarly, the contrast loss of the project can be obtained Combining these two losses, we get the objective function of the self-supervised task, expressed as:

[0087]

[0088] Then, the GRACE module is used to calculate the contrast loss between the two views again. At each iteration, noise is added to the view, and then two views are generated, represented as G1 = t(h′ i ) and G2=(h″ i ), and the nodes in the two generated views are represented as:

[0089] and

[0090] in and They are the feature matrix and adjacency matrix of the view respectively.

[0091] First, we use the designed graph convolutional neural network (GCN) encoder model f(X,A)∈RN×F′ , used to learn the representation of nodes in the graph;

[0092] Specifically, its function is to process the input graph data and generate a low-dimensional node embedding representation, that is, F<<F′, and represent H=f(X, A) as the learned representation of the node. After that, contrast loss is used as a discriminator, that is, a loss function used to distinguish the embedding representation of the same node in two different views from the embedding representation of other nodes. For any node i, its embedding representation a in one view i is regarded as an anchor point, and the embedding b generated in another view i is considered as a positive sample, while the other node embeddings in the two views are naturally considered as negative samples. In the multi-view graph contrastive learning setting, we borrow the idea of ​​InfoNCE objective and i ,b i ) defines a pair of optimization objectives.

[0093]

[0094] Where τ is a hyperparameter, and θ(a,b)=s(g(a),g(b)) is defined, where s(·) represents the cosine similarity function, which calculates the similarity between two vectors, and g(·) represents a nonlinear mapping function, the purpose of which is to enhance the expressive power of the function θ(·). A two-layer perceptron is used to construct g(·).

[0095] For a given positive sample pair, all nodes in the other two views are naturally defined as negative samples. Therefore, negative samples come from two sources, namely, cross-view and intra-view nodes, corresponding to the second and third terms in the denominator in formula (21), respectively. Since the two views are symmetric, the loss of the other view is similarly defined as l(b i ,a i ). The overall objective to be maximized is formally defined as the average of all positive sample pairs, namely:

[0096]

[0097] Similarly, the contrast loss of the project can be obtained Combining these two losses, we get the objective function of the self-supervised task, expressed as:

[0098]

[0099] Then, the obtained loss function is jointly optimized to obtain the final task objective function:

[0100] L zcl =αLcl +βJ cl

[0101] Among them, α and β are hyper parameters.

[0102] The multi-task training adopts a multi-task training method to simultaneously optimize the main recommendation task and auxiliary tasks, including a knowledge transfer task, a discrimination task, and a contrastive learning task.

[0103] Specifically: a multi-task training method is used to simultaneously optimize the main recommendation task and auxiliary tasks, including knowledge transfer tasks, discrimination tasks, and contrastive learning tasks.

[0104] L=L rec +λ1L m +λ2L disc +λ3L zcl +λ4||Θ|| 2

[0105] Among them, Θ represents the parameter set of the model, while λ1, λ2, λ3, λ4 respectively represent the parameters that control the loss strength of the above calculation, and L represents the total loss. In this work, the Bayesian personalized ranking (BPR) loss function is used:

[0106]

[0107] This is a loss function used in collaborative filtering recommendation systems. Compared with the traditional mean square error loss function, the BPR loss function takes into account the user's personalized preferences and plays a very important role in the recommendation algorithm.

[0108] in, represents the preference score, σ(·) represents the sigmoid function, O={(u,i,j)|(u,i+O + ,(u,i)∈O -} represents training data, O - represents an unobserved interaction.

[0109] In order to verify the effectiveness and advantages of the proposed model, a large number of experiments are carried out, and the results are summarized and analyzed in detail.

[0110] Datasets,Three widely used public benchmark datasets are used to evaluate our model: Yelp2018, Amazon-Book, and Movielens-25M, as shown in Table 1.

[0111] The statistics of these datasets are as follows:

[0112] Table 1: Contents of the three datasets

[0113]

[0114] Evaluation Metrics,Considering the focus on top-N recommendations, we,follow the convention of previous studies, remove data with a score less than 4 from the,Movielens-25M dataset, and set the remaining score to 1.,The interaction actions are divided into training, validation, and test sets in a ratio of 7:1:2.,On the test set, the relevance-based metric Recall@20 and the,ranking-based metric NDCG@20 are used to evaluate the performance of each,model.

[0115] Baselines, where the proposed model is compared with other GCN-based recommendation methods, including:

[0116] -LightGCN This method designs lightweight graph convolution to improve training efficiency and representation ability.

[0117] -SG: By perturbing the graph structure (e.g., edge dropping), an auxiliary task is designed to achieve greater benefits in long-tail recommendations.

[0118] -NCL: Improves recommendation performance by clustering similar nodes to provide semantic neighbors and structural neighbors.

[0119] -RGCF: This approach integrates a graph denoising module and a diversity preservation module to enhance the robustness of GCN-based recommendations.

[0120] -SimGCL: demonstrates a positive correlation between representation uniformity and the ability to debias via feature-level perturbations and contrastive learning, and achieves better long-tail performance than SGL.

[0121] -LAGCL: Through long-tail enhanced graph comparative learning, it enriches the graph data structure and improves the recommendation performance.

[0122] Settings and Hyperparameters. The open-source LAGCL was used for model development. For fair comparison, the hyperparameters of each baseline model reported in the original paper were preferred. In cases where this information was not provided, a grid search was performed to tune the hyperparameters. For the common setting for all baseline models, all embeddings were initialized using Xavier initialization. The embedding size was 64 and the L2 regularization parameter was 10. -4 , the batch size is 2048. Adam will be used to optimize all models, and the learning rate is set to 0.001.

[0123] Performance comparison

[0124] Table 2 shows the comparison results of the proposed model with other baseline models on three datasets. The following observations are made: 1) The proposed model consistently outperforms all baseline models on different datasets and metrics. Specifically, on the Yelp2018, Amazon Books, and Movielens-25M datasets, LAGCL improves the Recall@20 metric by 26.1%, 64.7%, and 10.1% respectively compared to the LightGCN model. The model also achieves better performance than the strongest baseline model, LAGCL. For example, on the same dataset, the Recall@20 metric performance is improved by approximately 0.41%, 1.91%, and 0.53%.

[0125] All graph contrastive learning based methods, such as SGL, NCL, RGCF, SimGCL, and LAGCL, have achieved significant improvements over LightGCN on the three datasets, which confirms the effectiveness of contrastive learning for collaborative filtering tasks. SimGCL outperforms other baseline models except LACGL, indicating that feature-level enhancement is more suitable for collaborative filtering tasks than structure-level enhancement. The method predicts neighborhood information for the tail node while retaining the original graph structure. The model combines the advantages of feature-level enhancement, avoids the possibility of drastic changes in tail node information due to structure-level enhancement, and maximizes the consistency of node embeddings between views generated by graph enhancement to learn representations. Therefore, method LAGCL achieves state-of-the-art performance, as shown in Table 2.

[0126] Table 2: Overall performance comparison

[0127]

[0128] Ablation experiment

[0129] To empirically discuss the impact of each proposed component, ablation studies are performed on the model to compare several ablation variants, including a “w / o KT” variant that does not use the knowledge transfer module to enhance the tail node. We also consider a variant “w / o GRACE” that does not use the designed contrastive learning framework “GRACE” to calculate the contrastive loss, and a “w / o CP” variant that uses the two contrastive learning calculations in proportion to each other as proposed in Section 3.3. Figure 3The following observations were made: (1) Removing any component will lead to a drop in performance, with the two graph comparisons calculated proportionally contributing the most. This proves that in the proposed innovative method, the reasonable combination of the two graph comparisons can play the greatest role. Without the reasonable use of the two comparison loss calculation methods, we will not be able to achieve meaningful data enhancement. (2) In the GCN-based recommendation scenario, it is crucial to use the knowledge transfer module to increase the importance of the tail nodes. (3) Removing GRACE will significantly reduce the performance. This shows that the knowledge transfer module and the GRACE model complement each other.

[0130] In order to achieve the best performance of the model, we need to adjust important hyperparameters. This paper experiments on the temperature parameter τ and the degree threshold k on the Yelp2018 dataset.

[0131] Sensitivity analysis of temperature parameter τ( Figure 4 and Figure 5 As shown in Figure 2, different temperature parameters τ will directly affect the effect of the model. The temperature coefficient can be used to control the distribution shape of logits. For a given logits distribution shape, when the τ value increases, 1 / τ decreases, s(h′ i ,h″ i ) / τ will make the values ​​in the original logits distribution smaller, and after the exponential operation, it will become even smaller, causing the original logits distribution to become smoother. On the contrary, if τ is small, 1 / τ will become larger, and the values ​​in the original logits distribution will become larger accordingly. After the exponential operation, it will become even larger, making the distribution more concentrated and more peaky. In order to select the best τ, in the experiment, our τ increases from 0.15 to 0.50 every 0.05 units. The experimental results are as follows Figure 5 As shown in Figure 2, it can be observed that the performance of our designed model changes with the change of τ. When τ is adjusted to a suitable value, the performance of the model reaches the best. Over-adjusting τ may weaken the auxiliary ability of contrastive learning in improving recommendation performance.

[0132] The degree threshold k is based on Figure 6 , Figure 7As shown in the figure, we conducted a threshold parameter sensitivity experiment on the Yelp2018 dataset to distinguish between head nodes and tail nodes. We performed experimental verification by testing a threshold of every 5 units. The results show that our model performs best when k=20. We draw the following conclusions: (1) Our model is able to learn transfer strategies from head users, thereby benefiting tail users. (2) A larger k will reduce the number of head users and may lead to insufficient learning of the knowledge transfer module. On the contrary, a smaller k will lead to obvious quality differences between head users, resulting in inaccurate learning of the knowledge transfer module. Therefore, choosing an appropriate segmentation threshold can maximize the overall benefit.

[0133] In order to demonstrate the advantages of our proposed model in solving data sparsity, we ran the model multiple times (10 times) on the three datasets of Yelp2018, Movielens-25M and Amazon-Book, averaged the results, and then compared them with the results obtained by the current best model to obtain the same proportion of improvement of our model. As can be seen from Table 1, the densities of the three datasets are: After processing, the data densities of the two datasets are 0.1296%, 0.0859% and 0.0619% respectively, that is, the sparsity of the datasets is ranked from large to small as Yelp2018, Movielens-25M and Amazon-Book. After model processing, the three datasets are compared with the best baseline model LAGCL in terms of improvement and improvement ratio, for example Figure 8 As shown in the figure, for the dataset with the most sparse data, Amazon-Book, the improvement ratio is the highest, followed by Movielens-25M. It can be concluded that the model we designed has obvious advantages in solving data sparsity.

[0134] In order to explore the impact of the number of convolutional layers in the encoder based on the graph convolutional network (GCN) we designed and the number of propagation layers in the GNN on the overall model, we conducted experiments with the number of layers ranging from 1 to 4. The experimental results are shown in Figure 2. Fig. 9 As shown. Fig. 9 Figures a, b, and c show that the number of layers in the encoder based on the graph convolutional network (GCN) we designed has a significant impact on the performance of the model. Figure d is an experiment on the propagation layer in the GNN conducted on the yelp2018 dataset. It can be seen that when the number of layers is set to 2, the performance of the recommendation algorithm is the best.

[0135] The design of the multi-layer structure helps the model to be more efficient in information transmission, so that it can deeply capture rich neighboring node information and deep semantic features. However, it is worth noting that when the model level reaches four layers, we observe that the performance of the recommendation system does not continue to improve, but instead shows a downward trend. This indicates that too many layers will lead to excessive smoothing of the embedding in the model, which will have a negative impact on the recommendation performance.

[0136] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A long-tail graph contrastive learning enhancement method based on knowledge transfer, characterized in that: include: Long-tail enhancement, contrastive learning, and multi-task training; The long-tail enhancement, contrastive learning and multi-task training are sequentially connected in data transmission, the long-tail enhancement further includes an automatic discarding module, a knowledge transfer module and a generative adversarial learning module, and the automatic discarding module, the knowledge transfer module and the generative adversarial learning module are sequentially connected in data transmission; The generative adversarial learning module includes a graph convolutional network encoder with skip-connected graph convolutional layers and a graph autoencoder based on contrastive learning.

2. According to the knowledge transfer-based long-tail graph contrast learning enhancement method of claim 1, it is characterized in that: The automatic discarding module is equipped with a trainable deletion strategy to reduce the distribution deviation between the enhanced data samples and the actual tail node samples.

3. According to the knowledge transfer-based long-tail graph contrast learning enhancement method of claim 2, it is characterized in that: The operation method of the automatic discard module is as follows: Neighbors are sorted according to edge weights, and then the first T neighbors are selected according to importance weights, which are calculated as follows: Among them, S represents the importance weight, W s represents the trainable parameter, ⊙ represents the element product, H (l) represents the d-dimensional node embedding matrix after the graph convolution layer of the l-th graph convolutional network encoder, H (0) represents the initial node embedding matrix, A represents the adjacency matrix, and T represents the matrix transpose.

4. According to the knowledge transfer-based long-tail graph contrastive learning enhancement method of claim 1, it is characterized in that: The knowledge transfer module uses the neighbor information discarded when generating the pseudo tail node to predict the missing neighbor information of the real tail node. The knowledge transfer module reconstructs the tail node based on the learning of the discarded neighbor information to generate a pseudo head node.

5. The long-tail graph contrast learning enhancement method based on knowledge transfer according to claim 4 is characterized in that: The principle of the knowledge transfer module is as follows: Assume h i represents the embedding vector of head node i, let h Ni represents the neighborhood N of i i The embedding vector of node i is obtained by pooling the embedding vectors of all neighboring nodes of node i. The learnable model predicts the migration vector of the head node in the neighborhood migration on node i, and for the tail node i, uses m i To represent the missing information, through its ideal neighborhood The embedding vector and the observed neighborhood N i The difference of the embedding vectors is composed of: Among them, the ideal neighborhood It represents a potential theoretical neighborhood; The neighborhood migration on node i is expressed as: Among them, r i represents the transfer vector that the learnable model can predict.

6. The long-tail graph contrast learning enhancement method based on knowledge transfer according to claim 1, characterized in that: The generative adversarial learning module enables long-tail enhancement to be effectively learned. The representation distribution of the real tail node should match the representation distribution of the pseudo tail node produced by the automatic discarding module, which is calculated as follows: in, represents the representation after the lth layer is automatically discarded, and Represent the sparse matrices of nodes i and j respectively, represents the neighborhood corresponding to node i, It represents the representation after the l-1th layer is automatically discarded; For the adversarial constraint of the tail node, the following loss function is adopted: The loss function for the adversarial constraint of the head node is as follows: Among them, L tail-disc represents the adversarial constraint loss function of the tail node, V represents the node set, V tail represents the set of tail nodes, CROSSENT(.) represents the cross entropy function, ζ(.) represents the indicator function, f d (:θ d ) indicates the use of θ d Parameterized discriminant function; L head-disc represents the adversarial constraint loss function of the head node, Represents the representation after which the representation is automatically discarded.

7. The long-tail graph contrast learning enhancement method based on knowledge transfer according to claim 1 is characterized in that: The loss function obtained by contrastive learning using contrastive loss is jointly optimized to obtain the final task objective function, which is: L zcl =αL cl +βJ cl Among them, L zcl represents the final task objective function, α, β represent the first and second hyperparameters respectively, L cl , J cl Respectively represent the objective functions for InfoNce and GCN self-supervised tasks.

8. The long-tail graph contrast learning enhancement method based on knowledge transfer according to claim 1 is characterized in that: The multi-task training adopts a multi-task training method to simultaneously optimize the main recommendation task and auxiliary tasks, including a knowledge transfer task, a discrimination task, and a contrastive learning task.

Images