A recommendation method, device and equipment based on contrastive learning and joint optimization

Abstract

The present invention provides a recommendation method, device and equipment based on contrastive learning joint optimization. The recommendation method includes: obtaining interaction data between users and items and constructing a bipartite graph; constructing a recommendation model including an embedding layer and a graph encoder, inputting the bipartite graph into the recommendation model for contrastive learning joint optimization training until the training stop condition is met; obtaining the item view representation and user view representation output by the graph encoder after training is completed, calculating the similarity between the user view representation of each user and the item view representation of all items, sorting the similarities from high to low, and recommending the top N items in the sorting to the user. The data augmentation operation is canceled, eliminating the need for data augmentation, and the recommendation task and contrastive learning task, which were originally two independent tasks, are combined into a joint learning task, which improves training efficiency and stability. Under the contrastive learning joint optimization framework, the recommendation method has the ability to suppress the recommendation of popular items, thereby improving recommendation performance.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a recommendation method, device and equipment based on contrastive learning and joint optimization. Background Art

[0002] Recommender systems have been widely used to alleviate information overload in real-world applications such as social media, news, videos, and e-commerce. They aim to estimate whether a user will show preference for an item based on their historical interactions.

[0003] Graph Contrastive Learning (GCL) has become a prominent research area in deep representation learning due to its simple model design and effectiveness in data-sparse environments. Therefore, more and more research focuses on using GCL to improve recommendation performance. A typical GCL-based recommendation model is Figure 2 As shown in sub-figure (a), the recommendation model generates two different views by applying structural perturbations (e.g., random edge dropout) or embedding perturbations (e.g., random uniform noise). Finally, the contrastive learning (CL) loss function is used to increase the similarity between the target node and its positive samples while reducing the similarity with negative samples. The recommendation loss function is used to correct the recommendation bias. The contrastive learning loss and the recommendation loss are both calculated and applied independently.

[0004] In the prior art, Chinese patent publication number CN 115659059 A discloses a recommendation model training method based on graph contrastive learning. This patent performs data augmentation on the initial graph to generate different enhanced views. It then extracts more information from these enhanced views through contrastive learning, thereby improving the accuracy of the recommendation model's vector representation of users and objects. This allows the recommendation model to more accurately explore users' preferences for objects, thereby improving the accuracy of the recommendation model in recommending objects to target users.

[0005] Although data augmentation techniques (such as structural perturbations and embedding perturbations) have been used as important tools for GCL, those skilled in the art have always been skeptical about their role in the field of recommendation. For example, in existing papers (such as Junliang Yu, Hongzhi Yin, Xin Xia, Tong Chen, Lizhen Cui, and Quoc Viet Hung Nguyen. 2022. Are graph augmentations necessary simple graph contrastive learning for recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1294-1303), it is believed that the enhancement technology of the structural perturbation method has no effect on the recommendation performance, but lacks consideration of the effectiveness of feature enhancement. On this basis, the inventors of the present application further conducted detailed experimental verification of the effects of structural perturbation methods and embedding methods on recommendation performance, and found that both structural perturbations and embedding perturbations have little effect on recommendation performance. In addition, the use of data augmentation also has the problem of increasing perturbation operations, which will reduce the training speed of the recommendation model and reduce the stability of the recommendation model after training. Therefore, after discarding the influence of data augmentation, the contrastive learning task will become simpler. How to combine the simplified contrastive learning task with the recommendation system to improve the recommendation performance, training speed and stability of the recommendation model is the technical problem to be solved in this application. Summary of the Invention

[0006] The present invention aims to solve the technical problems existing in the prior art and provides a recommendation method, device and equipment based on contrastive learning and joint optimization.

[0007] To achieve the above-mentioned object of the present invention, according to a first aspect of the present invention, the present invention provides a recommendation method based on contrastive learning joint optimization, comprising: obtaining interaction data of users and items and constructing a bipartite graph based on the interaction data; constructing a recommendation model, wherein the recommendation model comprises an embedding layer and a graph encoder, the embedding layer embeds the users and items in the bipartite graph respectively, and the graph encoder encodes the embedded representations of the users and items respectively to obtain view representations of the users and items; inputting the bipartite graph into the recommendation model for contrastive learning joint optimization training until the training stop condition is met; obtaining the item view representation and user view representation output by the graph encoder after the training is completed, calculating the similarity between the user view representation of each user and the item view representation of all items, sorting the similarities from high to low, and recommending the top N items in the sorting to the user, where N is a positive integer.

[0008] The above technical solution: constructs a streamlined recommendation model structure, eliminates the data augmentation part in traditional graph contrastive learning (GCL), eliminates the need for data augmentation, reduces training computational costs, and improves training efficiency. Through contrastive learning, the contrastive learning loss and recommendation loss, which are key factors affecting recommendation performance, are integrated into a unified loss, improving the recommendation accuracy and stability of the recommendation model and having the ability to suppress the recommendation of popular items.

[0009] In a preferred embodiment of the present invention, in each training of the recommendation model, the loss is calculated according to a pre-constructed model loss function, and the parameters of the recommendation model are adjusted based on the loss; the model loss function is positively correlated with the cosine similarity of the negative sample pairs and negatively correlated with the cosine similarity of the positive sample pairs.

[0010] This technical solution eliminates the need for two independent contrastive learning and recommendation loss functions, as in traditional GCL. Instead, it streamlines and integrates them into a single loss function, the model loss function. This reduces computational complexity while increasing the similarity of representations for positive pairs and decreasing that for negative pairs, thereby improving recommendation accuracy. The indices of the two samples in a negative pair contain both the user and item indices of the positive pair. Compared to traditional contrastive learning, which processes user and item sets separately, this framework treats nodes within batch B as negative samples, regardless of node type. Therefore, this learning framework can more concisely jointly address the relationship between users and items in recommendation tasks, as well as the relationship between users and items in contrastive learning tasks.

[0011] In a preferred embodiment of the present invention, the model loss function is:

[0012] Where B represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, zi represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, λ represents the loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′ Denotes the view representation of user n′.

[0013] The above technical solution adopts a streamlined and unified optimization method, integrating the contrastive learning loss and recommendation loss, key factors affecting recommendation performance. The loss of negative sample pairs (user-user pairs, user-item pairs, and item-item pairs) is incorporated into the model loss function, greatly accelerating the training efficiency of the recommendation model. Furthermore, the cosine similarity metric is used in the model loss function. The cosine similarity metric has significant advantages over the model loss function, as it can maintain the centralized distribution of node embedding moduli in bipartite graphs and achieve the effect of de-popularity. n can be the index u and i in the positive sample pair. Compared with traditional contrastive learning that processes user sets and item sets separately, this method treats nodes within the training batch B as negative samples without distinguishing between node types. Therefore, this learning framework can jointly handle the relationship between users and items in recommendation tasks and the relationship between users and items in contrastive learning tasks in a more concise manner.

[0014] In a preferred embodiment of the present invention, the model loss function is also related to the modulus distribution of the user view representation and the modulus distribution of the item view representation.

[0015] The above technical solution: Through experiments and analysis, the inventors discovered that the user and item embeddings learned by the model loss function have a concentrated embedding modulus length distribution, that is, the embedding lengths are mostly concentrated around a certain value, resulting in good de-popularity ability. After further exploring the correlation between embedding modulus length distribution and the de-popularity ability of the recommendation model, the inventors incorporated the node modulus length distribution into the model loss function to further promote the concentrated distribution of node models, thereby improving the de-popularity ability and stability of the recommendation model.

[0016] In a preferred embodiment of the present invention, the model loss function is:

[0017]

[0018] Where B represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, z i represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, α represents the first loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′represents the view representation of user n′, β represents the second loss weight coefficient, ||Z u ||2 represents the modulus distribution of all user view representations, ||Z i ||2 represents the modulus distribution of all item views, Var(||Z u ||2) means to find ||Z u Variance of ||2, Var(||Z i ||2) means to find ||Z i Variance of ||2.

[0019] The above technical solution: adds the node modulus distribution to the model loss function, directly and explicitly controls the distribution concentration of the node modulus, strengthens the modulus distribution advantage of the recommendation model of this application, and greatly reduces the frequency of recommending popular items.

[0020] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides an item recommendation device for implementing the recommendation method based on contrastive learning joint optimization described in the first aspect of the present invention, including: an acquisition module for acquiring interaction data between users and items; a mapping module for constructing a bipartite graph based on the interaction data; a recommendation model construction module for constructing a recommendation model, wherein the recommendation model includes an embedding layer and a graph encoder, the embedding layer embeds the users and items in the bipartite graph respectively, and the graph encoder encodes the embedded representations of the users and items respectively to obtain view representations of the users and items; a contrastive learning joint optimization training module for inputting the bipartite graph into the recommendation model for contrastive learning joint optimization training until the training stop condition is met; a recommendation module for obtaining the item view representation and user view representation output by the graph encoder after the training is completed, calculating the similarity between the user view representation of each user and the item view representation of all items, sorting the similarities from high to low, and recommending the top N items in the sorting to the user, where N is a positive integer.

[0021] The above technical solution has the beneficial technical effects of the recommendation method based on contrastive learning and joint optimization of the first aspect of the present invention.

[0022] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the recommendation method based on contrastive learning and joint optimization described in the first aspect of the present invention.

[0023] The above technical solution has the beneficial technical effects of the recommendation method based on contrastive learning and joint optimization of the first aspect of the present invention.

[0024] In order to achieve the above-mentioned purpose of the present invention, according to the fourth aspect of the present invention, the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the recommendation method based on contrastive learning and joint optimization described in the first aspect of the present invention.

[0025] The above technical solution has the beneficial technical effects of the recommendation method based on contrastive learning and joint optimization of the first aspect of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 1 is a flow chart of a recommendation method based on contrastive learning and joint optimization in one embodiment of the present invention;

[0027] Figure 2 2. It is a schematic diagram showing a structural comparison between a recommendation model and a traditional graph contrast learning model GCL in another embodiment of the present invention;

[0028] Figure 3 This is a structural block diagram of an item recommendation device according to another embodiment of the present invention;

[0029] Figure 4 is a schematic structural diagram of an electronic device in another embodiment of the present invention;

[0030] Figure 5 This is a comparison diagram of the node embedding distribution visualization of the existing model and the CGCL model recommended by this application in another embodiment of the present invention;

[0031] Figure 6 3. This is a comparison chart of the changes in PPR and modulus length ratio of the existing model and the recommended model of this application in the first 50 training rounds in another embodiment of the present invention;

[0032] Figure 7 This is a performance comparison of CGCL, SGL, and SimGCL in the first 50 epochs of training in another embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0034] In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0035] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal communication between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0036] The inventors of this application conducted an experimental verification to verify whether data augmentation technology has an effect on recommendation models. They selected the SGL model (adding structural perturbation data augmentation) and SimGCL model (adding embedding perturbation data augmentation), the most widely used in the Graph Contrastive Learning (GCL) model series, for experiments. The SGL model was modified to obtain the variant models SGL-CO (cropped SGL, including only contrastive learning) and SGL-AO (cropped SGL, including only data augmentation). The SimGCL model was modified to obtain the variant models SimGCL-AO (cropped SimGCL, including only data augmentation) and SGL-CO (cropped SimGCL, including only contrastive learning). The performance of these models was evaluated on two benchmark datasets, Yelp2018 and Amazon-Book. The performance comparison results are shown in Table 1. LightGCN is used as an existing graph encoder. It can be seen that the performance of SGL and SimGCL both exceeds LightGCN, thus proving the effectiveness of contrastive learning tasks. SGL-AO and SimGCL-AO do not use contrastive learning loss (CL) for auxiliary training. They are slightly better than LightGCN, but worse than SimGCL, SGL, SimGCL-CO, and SGL-CO because they introduce some randomness to make the model more generalizable. It is worth noting that SimGCL and SGL only show a slight advantage over their respective SimGCL-CO and SGL-CO methods. This shows that structural perturbations and embedding perturbations do not play their due role and have limited effect on improving recommendation performance, while contrastive learning loss has a greater effect on improving recommendation performance.

[0037] Recall@20 represents the recall rate of the top 20 recommendations in a given recommendation list. NDCG stands for Normalized Discounted Cumulative Gain, and NDCG@20 represents the NDCG value of the top 20 recommendations in a given ranked list. It can be seen that data augmentation through structural perturbation and embedding perturbation has a relatively small contribution to the recommendation performance of the recommendation model. The inventors believe that the introduction of contrastive learning loss has a greater impact on recommendation performance, so more attention should be paid to the role of contrastive learning loss.

[0038] The inventors further studied and found that Figure 2 As shown in sub-figure b, simply adding contrastive learning loss to the recommendation loss can produce considerable improvements. In other words, in GCL-based recommendation systems, contrastive learning loss itself is the key factor driving performance improvements. Therefore, for GCL-based recommendation models, the contrastive learning loss function is truly crucial. In response to this discovery, the inventors have made improvements and proposed a recommendation method, device, and apparatus based on contrastive learning combined with optimization.

[0039] Table 1 Performance comparison of different models

[0040]

[0041]

[0042] In one embodiment, the present invention discloses a recommendation method based on contrastive learning and joint optimization, the flow chart of the method is as follows: Figure 1 Shown, including:

[0043] Step S1: Obtain interaction data between users and items and construct a bipartite graph based on the interaction data.

[0044] The interaction data includes the user set U, the item set I, and the interaction information between users and items. The node set V = U∪I of the bipartite graph, and both users and items are nodes of the bipartite graph.

[0045] The edge set of the bipartite graph is E = {(u, i) | u∈U, i∈I}, which represents the interaction information between users and items. It should be noted that the items here are preferably, but not limited to, physical goods, service goods, or software goods. Figure 2 The original graph in is a bipartite graph.

[0046] Step S2: construct a recommendation model. The recommendation model structure of the present invention is as follows: Figure 2As shown in subgraph c in Figure 1, this is represented by Compact Graph Contrastive Learning (CGCL), which consists of an embedding layer and a graph encoder. The embedding layer embeds users and items in the bipartite graph, obtaining user embeddings and item embeddings, respectively. The graph encoder encodes the user and item embeddings to obtain user and item view representations, respectively. The graph encoder encodes the user embeddings to obtain user view representations, and the graph encoder encodes the item embeddings to obtain item view representations.

[0047] The embedding layer is preferably, but not limited to, a node attribute embedding network. Node attribute features are propagated and transformed through a fully connected layer (or multiple fully connected layers) to obtain an embedded representation of the node. When the recommendation model begins training, it randomly initializes the user embedding representation and item embedding representation. The graph encoder encodes the node embedding representation and edge information of the bipartite graph into a fixed-length vector representation to obtain a view representation of the node. The graph encoder is preferably, but not limited to, a graph convolutional network (GCN), which updates the node representation by performing convolution operations on neighboring nodes, capturing the node's neighbor information and local topology.

[0048] In step S3, the bipartite graph is input into the recommendation model for comparative learning and joint optimization training until the training stop condition is met.

[0049] The inventors proposed a fast and concise contrastive learning strategy called compact graph contrastive learning, such as Figure 2 As shown in subfigure c in [ ], this strategy unifies the contributions of the recommendation loss and contrastive learning loss into a simple loss function. This significantly reduces training costs by eliminating the computational overhead associated with data perturbations and loss function calculations. Preferably, during each training session of the recommendation model, the loss is calculated using a pre-built model loss function, and the model parameters are adjusted based on the loss. The model loss function is positively correlated with the cosine similarity of negative sample pairs and negatively correlated with the cosine similarity of positive sample pairs.

[0050] In this embodiment, the separate loss functions are combined into one, which simplifies the loss calculation process and reduces the computational cost of loss calculation. Therefore, it is further preferred that the model loss function is:

[0051]

[0052] Where B represents the training batch, which represents the part of the dataset selected from the original complete dataset for training; u and i represent the user index and item index in the positive sample pair, respectively, and u and i have interaction; z u represents the view representation of user u, zi represents the view representation of user i; n and n′ respectively represent the indexes of the two samples in the negative sample pair, and the combination of n and n′ needs to exclude the positive sample pair (u, i); α represents the first loss weight coefficient, τ represents the temperature coefficient, and z n represents the view representation of user n, z n′ represents the view representation of user n′. Here, n includes u and i, which treat nodes in batch B as negative samples without distinguishing between node types. Therefore, the latter term of the above model loss function can simultaneously obtain negative sample pairs consisting of user-user pairs, user-item pairs, and item-item pairs. The above model loss function uses cosine similarity to maintain a concentrated distribution of the module length of node embeddings (i.e., the view representation of the node) and ultimately eliminate the popularity problem, avoiding the frequent recommendation of popular items. exp represents the indicator function, and log represents the logarithmic function.

[0053] In this embodiment, the training stopping condition is preferably, but not limited to, the early stopping method. When the recommendation indicator (such as the Recall@20 value) does not improve after a preset Q rounds of training, the training is stopped. Q is a positive integer, preferably, but not limited to, 10.

[0054] In this embodiment, the specific training process of the recommendation model in step S3 is as follows:

[0055] Step S31: Initialize the node embedding representation randomly. Set hyperparameters: embedding dimension is set to 64, first loss weight coefficient α is selected from the interval {0.2, 0.5, 1.0, 2.0}, and temperature coefficient τ is selected from the interval {3, 4, 5, 6, 7}. Set the learning rate to 0.001 and the batch size B to 2048.

[0056] Step S32: Contrastive learning and joint optimization training:

[0057] The node embedding representation output by the embedding layer is input to the graph encoder. The graph encoder outputs the node view representation. Based on the view representations of all nodes, a loss value is calculated according to the model loss function. The network parameters of the embedding layer and the graph encoder are preferably optimized based on the obtained loss value using, but not limited to, an Adam optimizer. After the embedding layer parameters are optimized and updated, a new node embedding representation is generated. The network parameters of the embedding layer preferably include, but are not limited to, an embedding matrix. The network parameters of the graph encoder preferably include, but are not limited to, a convolution kernel size.

[0058] Step S33, determine whether the training stop condition is met, if so, complete the recommendation model training, if not, return to execute steps S32 and S33.

[0059] Step S4: Obtain the item view representations and user view representations output by the trained graph encoder. Calculate the similarity between each user's user view representation and the item view representations of all items. Ranking the similarities from high to low, recommend the top N items in the ranking to the user, where N is a positive integer, preferably, but not limited to, 20, 10, 15, or 5. The similarity is preferably, but not limited to, cosine similarity or inner product similarity. For each user, calculate the similarity between their user view representation and the item view representations of all items, then sort the similarities from high to low to obtain a similarity sequence. Select the top N items in the similarity sequence as the recommended items for the user.

[0060] A comparative analysis is conducted on the feature distribution uniformity and modulus length of the recommendation model disclosed in this embodiment and the existing model in terms of node embedding. Figure 5 The figure shows the visualization of the node embedding distribution of the existing model and the recommended model CGCL of this embodiment, considering the two perspectives of uniformity and modulus. The uniformity of the normalized feature distribution on the unit hypersphere can be used as an important representation for evaluating the performance of contrastive learning. Specifically, the representation that shows the best performance is retained, and then the representation distribution is plotted using non-parametric Gaussian kernel density estimation, as shown in Figure 1. Figure 5 As shown. For ease of comparison, the representations on the hypersphere are not displayed directly. Instead, they are mapped to two-dimensional normalized vectors using t-SNE nonlinear dimensionality reduction and visualization technology (t-Distributed Stochastic Neighbor Embedding). Since all representations are normalized, they will be distributed on a circle with a radius of 1, and the density estimate of the angle of each point on the circle is directly displayed. Therefore, it can be seen that the existing SimGCL model has a more dispersed representation distribution at each angle than the existing LightGCN and SGL, ultimately producing superior recommendation performance, which is more obvious on the Yelp2018 dataset. In addition, the model without the enhancement method only shows a small difference in uniformity from the corresponding method. CGCL can be considered a more general alignment and uniformity optimization model than DirectAU. It uses temperature coefficient and cosine similarity to improve the model's recommendation performance and training efficiency. CGCL treats nodes within a mini-batch as negative samples without distinguishing between node types, which is beneficial to the overall uniformity of nodes.

[0061] analyze Figure 5, we found that, except for CGCL proposed in this application, all existing models exhibit an unbalanced distribution in terms of embedding modulus. We attribute this phenomenon to the use of BPR loss with inner product in these existing models, which promotes prediction by assigning higher scores to observed interactions than unobserved interactions. Therefore, in the process of minimizing the loss during model training, users and items with more interaction data tend to have larger modulus. In contrast, the unified use of cosine similarity in CGCL alleviates this problem and produces a more concentrated distribution.

[0062] Furthermore, we focus on the relationship between the modulus distribution and the popularity of items. In order to evaluate the effectiveness of each method in removing popularity, we define the top 5% items with the most interaction data as “popular items”. pop And track the frequency with which these items are recommended to users in each training round. We propose the PPR (Percentage of Popular Recommended Items) metric to quantify this frequency:

[0063]

[0064] in, Represents the recommendation list of user u The length of |u| represents the number of users. express and I pop The length of the intersection.

[0065] In addition, the change in modulus distribution is observed by calculating the ratio of the average modulus of popular items to that of other items, namely the "Modulus Ratio". Figure 6 The specific results of each training round (Epoch) are shown in . The changes in PPR and modulus ratio indicators are basically consistent in all models. An increase in modulus ratio indicates a higher frequency of recommending popular items, and vice versa. This shows that modulus ratio is a reliable indicator of the ability to measure popularity. Then, LightGCN tends to recommend popular items in the first 50 rounds because these items provide huge gradient benefits for BPR loss. The SGL and SimGCL models claim to solve the problem of popularity bias by using InfoNCE loss. However, experimental results show that GCL models, especially SimGCL, have a relatively large tendency to recommend popular items, which shows that InfoNCE alone is not enough to alleviate this problem. Their ability to suppress the recommendation of popular items mainly lies in their fast convergence speed. When the PPR value has not increased significantly, they will produce the best recommendation results. In contrast, the recommendation model proposed in this application always maintains a concentrated distribution of modulus, resulting in a significant reduction in the frequency of recommending popular items.

[0066] The following is an experimental verification of the recommendation performance of the recommendation model and recommendation method provided in this embodiment:

[0067] 1. Experimental Setup

[0068] Dataset Selection. Experiments were conducted on three benchmark datasets: Douban-Book, Yelp2018, and Amazon-Book. Detailed statistics are given in Table 2. To ensure experimental fairness, we directly adopted the processed datasets used in previous work. The interaction data was split into training, validation, and test sets in a ratio of 7:1:2. For the test setting, we compared the advantages and disadvantages of different models by evaluating the top-K recommendation performance and reported the average Recall@K and NDCG@K, where K = 20. Each experiment was performed 5 times, and the average results were reported.

[0069] Table 2. Statistics of the dataset.

[0070]

[0071] Comparison Method. The proposed CGCL is compared with the following existing recommendation system CF models (Collaborative Filtering Model):

[0072] -LightGCN: A graph-based CF method that designs a light graph convolution by removing feature transformation and nonlinear activation.

[0073] -DGCF: A graph-based model that decouples user intent from user behavior and encourages the independence of different intents.

[0074] -SGL: An enhanced GCL-based model that generates different views by perturbing the graph structure. We only use the most effective edge-dropping method.

[0075] -dnn+ssl: A DNN-based recommendation model that generates different opinions by perturbing item features. In our experiments, due to the lack of feature information, we use ID as feature information.

[0076] -MixGCF: A mixture-based hard negative sample mining model that uses mixed domain information to generate new negative samples and dynamically selects the hardest samples for model training.

[0077] -SimGCL: An enhanced GCL model that generates diverse views by adding directional random noise to the embedding.

[0078] -NCL: A non-incremental GCL model that enriches the contrastive learning task by considering both structural and semantic information.

[0079] Hyperparameter Settings. For all methods, embeddings were initialized using the Xavier method with an embedding dimension of 64. The model was optimized with the Adam optimizer, a learning rate of 0.001, and a batch size of 2,048. For CGCL, α and τ were adjusted in the ranges of {0.2, 0.5, 1.0, 2.0} and {3, 4, 5, 6, 7}, respectively.

[0080] 2. Performance comparison

[0081] Table 3. Overall performance comparison of different GCL methods at different layers

[0082]

[0083] Comparison with GCL. Table 3 shows the comparison results of CGCL, SGL, and SimGCL. The bold numbers represent the best performance, and the underlined numbers represent the second best performance. Findings:

[0084] - The SGL and SimGCL implementations outperform LightGCN in most cases, demonstrating the superiority of using contrastive learning tasks to assist recommendation tasks.

[0085] -SGL-CO and SimGCL-CO show comparable performance to the original methods. Especially in the case of Amazon-Book and 1-layer GCN encoder, SimGCL-CO seems to outperform SimGCL. This suggests that existing data augmentation methods may not be as effective as expected.

[0086] -CGCL shows the best performance in all cases. In particular, the CGCL model achieves the largest performance improvement on Amazon-Book, which is due to the CGCL model’s ability to maintain a concentrated distribution of module lengths.

[0087] - Increasing the number of model layers from 1 to 3 can significantly improve the performance of SGL and SimGCL. However, the CGCL model is less affected by the change in the number of layers, which indicates that CGCL can learn most of the useful information with only one hop of neighborhood.

[0088] Comparison with other models. To further confirm the performance of CGCL, we compare it with other recently proposed methods without data augmentation, as shown in Table 4. As can be seen, CGCL outperforms all baselines on the three datasets. The next best method is MixGCF, which achieves the same recommendation performance as the existing GCL. This is because MixGCF directly selects the most similar samples as negative samples, and these similar negative samples have a significant improvement in performance. Finally, the remaining methods do not show a clear advantage.

[0089] Table 4. Overall performance comparison with other models.

[0090]

[0091] 3. Advantages of CGCL

[0092] Convergence speed. Figure 7 The performance comparison of CGCL with SGL and SimGCL in the first 50 epochs of training is shown in Figure 2. All experiments are performed with a 3-layer GCN.

[0093] from Figure 7 As can be seen from the figure, the convergence speed of SGL and SimGCL has been greatly improved compared with the LightGCN model. However, two new problems also emerged during the training process: (1) The model is unstable in the early stage. For example, the recommendation results of SimGCL on Yelp2018 and Amazon-Book dropped sharply in the first 10 rounds. We believe that this is because the excessive random increment seriously undermines the learning of node representation. (2) The learned representation cannot remain effective for a long time. SGL and SimGCL drop rapidly after reaching the best performance. This is very obvious on Amazon-Book, where the performance drops by about 10% after every 10 rounds of training. This may hinder further optimization of the model because the representations of some nodes fail to converge to the optimal state, while the representations of other nodes may begin to become invalid.

[0094] For CGCL, it takes slightly more time to reach optimal performance than SGL and SimGCL, but it still consistently outperforms existing models in recommendation performance. More importantly, CGCL successfully alleviates the two problems mentioned above. CGCL is always advantageous in early rounds of training because the representation of nodes is learned entirely from complete interaction data without perturbations. In addition, the performance of CGCL is always stable. Even after reaching the peak, the recommendation performance of CGCL only decreases slightly, because the CGCL model always maintains a low proportion of popular items in the recommendation list.

[0095] Runtime. As shown in Table 5, collected on an A6000-48G GPU and AMD EPYC 7262, SGL and SimGCL increase the time by about two times compared to LightGCN. At the same time, the training time of SGL and SimGCL is roughly equal, which means that the time of the existing GCL model is mainly spent in the graph convolution process of multiple different views. In contrast, CGCL only applies graph convolution on one view. In addition, it cancels the negative sampling operation, resulting in less training time than LightGCN. Most notably, the training time of CGCL for one round on Douban-Book is close to half that of LightGCN.

[0096] Table 5. Comparison of GCL model running times (in seconds).

[0097]

[0098] Depopularity. The ability of SGL and SimGCL to mitigate popularity is primarily due to their rapid convergence. To further validate CGCL's depopularity capabilities, all items were divided into four groups of popular items based on popularity. Specifically, the first group contained the 40% least popular items; the second group contained the 40% of items adjacent to the first group; similarly, the third group included the 15% of items adjacent to the second group; and the 5% most popular items belonged to the fourth group. Experiments were then conducted to observe the recall value contributed by each group when the model achieved optimal performance. A higher recall value in a group does not necessarily mean that the model is more inclined to recommend items in that group, as higher recommendation precision also helps improve recall. Furthermore, among existing models, SGL exhibits poor depopularity, recommending a higher proportion of the top 5% "hot items" (in group 4) than LightGCN across all three datasets. SimGCL performed between LightGCN and CGCL in exploring long-tail items and significantly reduced the recommendation frequency of popular items on the Yelp 2018 and Amazon-Book datasets. The fact that the CGCL model achieves better overall performance in all datasets while recommending fewer “popular items” demonstrates the popularity-reducing capability of CGCL.

[0099] In another embodiment of the present invention, based on the above embodiment, the following distinction is set: the model loss function is also related to the modulus length distribution of the user view representation and the modulus length distribution of the item view representation.

[0100] The model loss function is:

[0101]

[0102] Where n represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, z i represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, α represents the first loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′ represents the view representation of user n′, β represents the first loss weight coefficient, ||Z u ||2 represents the modulus distribution of all user view representations, ||Z i ||2 represents the modulus distribution of all item views, Var(||Z u ||2) means to find ||Z u Variance of ||2, Var(||Z i ||2) means to find ||Z i Variance of ||2.

[0103] In this embodiment, the advantages of modulus distribution are continuously utilized. Based on the previous embodiment, a new auxiliary function is used to further promote the centralized distribution of node modulus. Variance is an important indicator to measure the difference between each sample value and the average value of all samples, that is, in represents the average value of all samples x. In this embodiment, we hope that the node modulus is concentrated at a certain value, which also means that we hope that the variance of the node modulus is smaller. Therefore, in this embodiment, the variance of the node modulus is explicitly added to the loss function to obtain a new CGCL model.

[0104] In the model loss function of this embodiment, the processing of node modulus is somewhat similar to the embedded regularization, but the purpose is different. Generally, the model often adds an embedded L2 regular expression in the loss function to prevent the model from overfitting, that is, However, the purpose of this embodiment is to make the embedding distribution of nodes more concentrated, and the specific value is not important.

[0105] The model loss function of the CGCL model in this embodiment is divided into three parts:

[0106] (1) This part is a variation of the BPR loss, temporarily eliminating the negative sampling component of the BPR loss and focusing solely on learning the similarity between interacting users and items. In addition, we use cosine similarity instead of inner product, which effectively suppresses the recommendation frequency of popular items.

[0107] (2) This part combines the negative sample part of the BPR loss with the original loss function of contrastive learning. It uses a unified form to optimize the relationships between user-user pairs, user-item pairs, and item-item pairs, greatly accelerating the training efficiency of the model.

[0108] (3)(Var(||Z u ||2)+Var(||Z i The ||2)) part directly and explicitly controls the concentration of node embedding modulus lengths, continuing to leverage the advantages of modulus length distribution.

[0109] A performance comparison and verification experiment was conducted on the CGCL model provided in this embodiment and the similar recommendation model DirectAU in the existing non-contrastive learning field, which further enhanced the feasibility of the model in actual operation (from: Chenyang Wang, Yuanqing Yu, Weizhi Ma, Min Zhang, Chong Chen, Yiqun Liu, and Shaoping Ma. 2022. Towards Representation Alignment and Uniformity in Collaborative Filtering. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 1816-1825).

[0110] Performance Comparison. To demonstrate the difference between the CGCL model of this embodiment and DirectAU, Table 6 compares the results of CGCL and DirectAU. The results show that the CGCL model of this embodiment significantly outperforms DirectAU in all cases, indicating that the CGCL model of this embodiment is more reasonable in its choice of temperature coefficient and GCL object.

[0111] Table 6. Overall performance comparison of CGCL and DirectAU at different layers

[0112]

[0113]

[0114] Runtime comparison. Cosine similarity outperforms the Gaussian kernel function in terms of time and space cost. This is a very important advantage for the CGCL model in this embodiment. GCL usually benefits from larger batch sizes because larger batch sizes are more conducive to the uniformity of representation. This is also empirically applicable to recommendations. The time and space costs of the two similarity metrics are evaluated under different batch size settings. Please note that in order to make a fair comparison, CUDA cache clearing is used during training, which also affects the training efficiency of the model.

[0115] Table 7. Comparison of cosine similarity and Gaussian kernel function in terms of time cost (seconds) and space cost (GB)

[0116]

[0117] The results are shown in Table 7. We can see that the Gaussian kernel incurs a higher cost in all settings. In particular, for a batch size of 4,096, the Gaussian kernel takes 10 times longer than cosine similarity and requires 30GB of memory to train, exceeding the capacity of most GPUs. However, for cosine similarity, larger batch sizes lead to an exponential decrease in training time. The results demonstrate that cosine similarity outperforms the Gaussian kernel in both runtime and memory usage.

[0118] The present invention also discloses an item recommendation device for implementing the recommendation method based on contrastive learning and joint optimization in the above embodiment. The system block diagram of the device is as follows: Figure 3 Shown, including:

[0119] The acquisition module is used to obtain the interaction data of users and items; the graph construction module constructs a bipartite graph based on the interaction data; the recommendation model construction module constructs a recommendation model, which includes an embedding layer and a graph encoder. The embedding layer embeds the users and items in the bipartite graph respectively, and the graph encoder encodes the embedded representations of users and items respectively to obtain the view representations of users and items; the contrastive learning joint optimization training module inputs the bipartite graph into the recommendation model for contrastive learning joint optimization training until the training stop condition is met; the recommendation module obtains the item view representation and user view representation output by the graph encoder after training is completed, calculates the similarity between the user view representation of each user and the item view representation of all items, sorts the similarities from high to low, and recommends the top N items in the sorting to the user, where N is a positive integer.

[0120] In this embodiment, preferably, the contrastive learning joint optimization training module calculates the loss according to a pre-constructed model loss function in each training of the recommendation model, and adjusts the parameters of the recommendation model based on the loss; the model loss function is positively correlated with the cosine similarity of the negative sample pairs and negatively correlated with the cosine similarity of the positive sample pairs.

[0121] In this embodiment, preferably, the loss function is:

[0122]

[0123] Alternatively, the model loss function is:

[0124]

[0125] Where B represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, z i represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, α represents the first loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′ represents the view representation of user n′, β represents the second loss weight coefficient, ||Z u ||2 represents the modulus distribution of all user view representations, ||Z i ||2 represents the modulus distribution of all item views, Var(||Z u ||2) means to find ||Z u Variance of ||2, Var(||Z i ||2) means to find ||Z i Variance of ||2.

[0126] In another embodiment of the present invention, the present invention further discloses a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, it implements the recommendation method based on contrastive learning and joint optimization provided in the above embodiment of the present invention.

[0127] In another embodiment of the present invention, the present invention also discloses an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the recommendation method based on contrastive learning and joint optimization provided in the above embodiment of the present invention.

[0128] The present invention also discloses an electronic device, such as Figure 4As shown, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the recommendation method based on contrastive learning and joint optimization provided in the above embodiment of the present invention.

[0129] like Figure 4 FIG2 is a schematic diagram of the structure of an electronic device that implements a recommendation method based on contrastive learning and joint optimization according to an embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a recommendation method program based on contrastive learning and joint optimization.

[0130] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the memory 11 (for example, executing a recommendation method program based on comparative learning joint optimization, etc.), as well as calling data stored in the memory 11, to perform various functions of the electronic device and process data.

[0131] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, an SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a recommendation method program based on comparative learning and joint optimization, but can also be used to temporarily store data that has been output or is to be output.

[0132] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0133] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, for displaying information processed in the electronic device and for displaying a visual user interface.

[0134] Figure 4 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 4 The structure shown does not limit the electronic device and may include fewer or more components than shown in the figure, or combine some components, or arrange the components differently.

[0135] For example, although not shown, the electronic device may further include a power source (such as a battery) to power various components. Preferably, the power source may be logically connected to at least one processor 10 via a power management device, thereby implementing functions such as charge management, discharge management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0136] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0137] Furthermore, if the modules / units integrated into the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0138] In the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention.

[0139] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0140] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

Claims

1. A recommendation method based on contrastive learning and joint optimization, characterized in that: include: Obtaining interaction data between users and items and constructing a bipartite graph based on the interaction data; Constructing a recommendation model, wherein the recommendation model includes an embedding layer and a graph encoder. The embedding layer embeds users and items in a bipartite graph, and the graph encoder encodes the embedded representations of users and items to obtain view representations of users and items. The bipartite graph is input into the recommendation model for comparative learning and joint optimization training until a training stop condition is met. During each training of the recommendation model, the loss is calculated according to a pre-established model loss function, and the parameters of the recommendation model are adjusted based on the loss. The model loss function is related to the modulus length distribution of the user view representation and the modulus length distribution of the item view representation. Obtain the item view representation and user view representation output by the trained graph encoder, calculate the similarity between each user's user view representation and the item view representation of all items, sort the similarities from high to low, and recommend the top N items in the sort to the user, where N is a positive integer.

2. The recommendation method based on contrastive learning and joint optimization according to claim 1, characterized in that: The model loss function is also positively correlated with the cosine similarity of negative sample pairs and negatively correlated with the cosine similarity of positive sample pairs.

3. The recommendation method based on contrastive learning and joint optimization according to claim 2, characterized in that: The model loss function is: Where B represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, z i represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, α represents the first loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′ represents the view representation of user n′, β represents the second loss weight coefficient, ||Z u ||2 represents the modulus distribution of all user view representations, ||Z i ||2 represents the modulus distribution of all item views, Var(||Z u ||2) means to find ||Z u Variance of ||2, Var(||Z i ||2) means to find ||Z i Variance of ||2.

4. An item recommendation device, used to implement the recommendation method based on contrastive learning and joint optimization according to any one of claims 1 to 3, characterized in that: include: Acquisition module, used to obtain interaction data between users and items; A graph building module, constructing a bipartite graph based on the interaction data; A recommendation model construction module is used to construct a recommendation model. The recommendation model includes an embedding layer and a graph encoder. The embedding layer embeds users and items in a bipartite graph, and the graph encoder encodes the embedded representations of users and items to obtain view representations of users and items. A contrastive learning and joint optimization training module feeds the bipartite graph into the recommendation model for contrastive learning and joint optimization training until a training stop condition is met. During each training session of the recommendation model, the loss is calculated according to a pre-built model loss function, which is related to the modulus length distribution of the user view representation and the modulus length distribution of the item view representation, and the parameters of the recommendation model are adjusted based on the loss. The recommendation module obtains the item view representation and user view representation output by the trained graph encoder, calculates the similarity between each user's user view representation and the item view representation of all items, sorts the similarities from high to low, and recommends the top N items in the sort to the user, where N is a positive integer.

5. The item recommendation device according to claim 4, wherein: The model loss function is also positively correlated with the cosine similarity of negative sample pairs and negatively correlated with the cosine similarity of positive sample pairs.

6. The item recommendation device according to claim 5, wherein: The model loss function is: Where B represents the training batch, u and i represent the user index and item index in the positive sample pair respectively, and z u represents the view representation of user u, z i represents the view representation of user i, n and n′ respectively represent the indexes of the two samples in the negative sample pair, α represents the first loss weight coefficient, τ represents the temperature coefficient, z n represents the view representation of user n, z n′ represents the view representation of user n′, β represents the second loss weight coefficient, ||Z u ||2 represents the modulus distribution of all user view representations, ||Z i ||2 represents the modulus distribution of all item views, Var(||Z u ||2) means to find ||Z u Variance of ||2, Var(||Z i ||2) means to find ||Z i Variance of ||2.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the recommendation method based on contrastive learning and joint optimization according to any one of claims 1 to 3 is implemented.

8. An electronic device, comprising: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the recommendation method based on contrastive learning and joint optimization as described in any one of claims 1 to 3.

Images