A commodity recommendation method and system based on hierarchical contrastive learning

Abstract

The present invention discloses a product recommendation method and system based on hierarchical contrastive learning, which includes: based on the Auto-Encoder architecture, introducing global contrast and self-contrast learning strategies, optimizing and learning the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence, fusing them, and optimizing them through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector; performing dot product calculation on the reconstructed user-product interaction sequence and the user's comprehensive representation vector; and selecting the product with the largest probability of the user interacting with the candidate product at the next moment as the next recommended product. By using the present invention, the accuracy of product sequence recommendation is improved by improving the learning of user behavior representation and the recommendation algorithm process. As a product recommendation method and system based on hierarchical contrastive learning, the present invention can be widely used in the field of user product recommendation technology.

Description

Technical Field

[0001] The present invention relates to the technical field of user product recommendation, and in particular to a product recommendation method and system based on hierarchical contrastive learning. Background Art

[0002] With the explosive growth of information, a wide variety of service platforms are being developed online, including shopping, video, news, music, dating, and social networking. To improve the efficiency of information acquisition and distribution, sequential recommendation algorithms analyze historical interactions between users and items to identify their long-term and short-term interests, enabling more accurate recommendations. These algorithms have been widely researched and applied in both industry and academia.

[0003] When user behavior data is sufficient, models such as RNNs, Transformers, and GNNs are often well-suited for sequential recommendation tasks. However, in real-world applications, these approaches often face data sparsity and cold-start issues (new users join and insufficient data on their interactions with items / services), making them less than ideal for learning user profiles. Therefore, a more straightforward solution is to leverage additional auxiliary information, such as item information, reviews, and ratings, to build finer-grained connections between users or items and enrich the information contained within. However, the effectiveness of such approaches is highly dependent on auxiliary information, and the acquisition of auxiliary information is often limited to specific scenarios and platforms, making data acquisition expensive. A more effective solution is to leverage existing data through data augmentation or multi-faceted data processing to maximize the usefulness of each piece of data, thereby improving the learning of user interest representations (user profiles). Contrastive learning technology uses data augmentation and optimization using the large amount of unsupervised semantic information in the data itself, which can greatly alleviate the problem of data sparsity. However, existing contrastive learning technologies only use single-level comparisons, such as augmentation based on the user's original behavior sequence and comparison between models. The semantic information is not fully utilized and the perspective is relatively single. Summary of the Invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a product recommendation method and system based on hierarchical contrastive learning, which improves the accuracy of product sequence recommendation by improving the learning of user behavior representation and the recommendation algorithm process.

[0005] The first technical solution adopted by the present invention is: a product recommendation method based on hierarchical contrastive learning, comprising the following steps:

[0006] Based on the Auto-Encoder architecture, it introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence;

[0007] The user's high-level interest representation is fused with the user's low-level interest representation and the fusion process is optimized through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector;

[0008] Perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment;

[0009] The item with the highest probability of user interaction with the candidate item at the next moment is selected as the next recommended item.

[0010] Furthermore, based on the Auto-Encoder architecture, the global comparison and self-comparison learning strategies are introduced to optimize the learning of the user-product interaction sequence, and the output of the user's high-level interest representation and the reconstructed user-product interaction sequence is carried out. This step specifically includes:

[0011] Obtain the interaction sequence between users and products, that is, the user's low-level interest representation;

[0012] Input the interaction sequence between users and products into the Auto-Encoder architecture for learning. The Auto-Encoder architecture includes an item embedding layer, a user encoding layer, and a user decoding layer.

[0013] A global contrastive learning strategy is introduced to optimize the learning process of the Auto-Encoder architecture through reconstruction loss function and global contrastive loss function, and the reconstructed user-item interaction sequence is output;

[0014] A self-contrastive learning strategy is introduced to optimize the learning process of the item embedding layer and user encoding layer in the Auto-Encoder architecture through the self-contrastive loss function, and output the user's high-level interest representation.

[0015] Furthermore, the step of inputting the interaction sequence between the user and the product into the Auto-Encoder architecture for learning specifically includes:

[0016] The project embedding layer based on the Auto-Encoder architecture converts the product ID into a product code vector and uses the common relative position encoding. That is, the absolute position is represented by the sine and cosine functions, and the relative position is obtained by multiplying the two.

[0017] The user encoding layer based on the Auto-Encoder architecture adopts a Transformer-based encoder structure, including a multi-head attention layer, a first residual connection layer, a first normalization layer, a feedforward network layer, a second residual connection layer, and a second normalization layer. The user encoding layer is used to encode the output results of the item embedding layer to obtain the user's high-level interest representation to be optimized;

[0018] The user decoding layer based on the Auto-Encoder architecture adopts the decoder structure in the Transformer architecture, including a self-attention layer, a first residual connection layer, a first normalization layer, a multi-head attention layer, a second residual connection layer, a second normalization layer, a feedforward network layer, a third residual connection layer and a third normalization layer. The user decoding layer is used to decode the user's high-level interest representation to obtain a reconstructed user-product interaction sequence to be optimized.

[0019] Furthermore, the step of introducing a global contrastive learning strategy, optimizing the learning process of the Auto-Encoder architecture by using a reconstruction loss function and a global contrastive loss function, and outputting a reconstructed user-product interaction sequence specifically includes:

[0020] Calculate the similarity of behavior sequences between all users based on Jaccard, and select the most similar user sequence for each user as a positive sample;

[0021] Select the user with Jaccard similarity = 0 as a random negative sample;

[0022] Calculate the user's popularity tendency based on the frequency of product appearance in the user's behavior sequence, and select the user's hard negative sample behavior sequence;

[0023] The positive and negative samples and other data are packaged into DataLoader and fed into the model during the pre-training phase. The user encoding layer generates the user's high-level interest representation to be optimized. The reconstruction loss function and the global contrast loss function are used for optimization, and the product encoding layer is updated in reverse. The reconstructed user-product interaction sequence is output.

[0024] Furthermore, the self-contrastive learning strategy is introduced to optimize the learning process of the item embedding layer and the user encoding layer in the Auto-Encoder architecture through the self-contrastive loss function, and the step of outputting the user's high-level interest representation specifically includes:

[0025] Delete the last 1-2 digits of the interaction sequence between the user and the product, and slice it to obtain the positive sample users;

[0026] Replace the last 1-2 digits of the interaction sequence between the user and the product with negative items to obtain the replacement negative sample user;

[0027] Insert 1-2 negative items at the end of the interaction sequence between the user and the product to obtain the added and modified negative sample users;

[0028] The above positive sample users, replaced negative sample users and added and modified negative sample users are sent to the encoder of the user encoding layer, and the self-contrast loss function is used for optimization to output the user's high-level interest representation.

[0029] Furthermore, the step of fusing the user's high-level interest representation with the user's low-level interest representation and optimizing the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector specifically includes:

[0030] The user's high-level interest representation and the user's low-level interest representation are fused through the attention fusion layer to obtain the user's comprehensive representation vector;

[0031] The structure of the attention fusion layer is the same as the user decoding layer of the Auto-Encoder architecture. The user's high-level interest representation is the user's long-term interest preference, and the user's low-level interest representation is the user's short-term interest preference. The expression of the fusion processing is as follows:

[0032]

[0033] In the above formula, c i represents the comprehensive representation vector of the user, Represents the user's high-level interest representation, Represents the user's low-level interest representation.

[0034] Furthermore, the step of performing a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment specifically includes:

[0035] Based on the user encoding layer of the Auto-Encoder architecture, the user's high-level interest representation is obtained, and the Euclidean distance is calculated within the batch to obtain the calculation result;

[0036] Based on the calculation results, the nearest neighbors and next nearest neighbors of the user's high-level interest representation are selected as positive and negative samples of the user's long-term interest preference representation respectively;

[0037] The interaction sequences between users and items in a batch are converted into item representation vectors through item embedding. The nearest neighbors and second-nearest neighbors of the item representation vectors are calculated using Hausdorff distance. The original behavior sequences corresponding to the nearest neighbors and second-nearest neighbors of the item representation vectors are used as positive and negative samples of the user's short-term interest preference representation.

[0038] Through Attention, the positive and negative samples representing the user's long-term interest preferences are weightedly fused with the positive and negative samples representing the user's short-term interest preferences to obtain the weighted fusion result;

[0039] The weighted fusion result is input into the high-level contrastive learning strategy, and the BPR optimization is performed through the high-level contrastive loss function to obtain the probability of the user interacting with the candidate product at the next moment.

[0040] The expression of BPR optimization is:

[0041]

[0042]

[0043]

[0044] In the above formula, N s represents the negative product candidate set; I represents negative products, i.e. products that the user has not interacted with before (t+1). v represents the embedding of the positive product, σ(·) represents the Sigmoid activation function, which compresses the data into the range of (0, 1). represents the comprehensive representation vector generated by user i at time t, Represents the real interactive product of user i at time (t+1).

[0045] The global contrast loss function, the self contrast loss function and the high-level contrast loss function all use triplet loss, and their expressions are:

[0046]

[0047] In the above formula, α represents the custom boundary value, f(u i ) represents the output of the user encoding layer of the Auto-Encoder architecture, u i Represents the user, Represents user u i The positively correlated user is the output of the historical interaction behavior sequence through the user encoding layer of the Auto-Encoder architecture, that is, ... i There are other user behavior sequences with certain similarities, Represents user u i The negative opposite user history interaction sequence is output by the user encoding layer of the Auto-Encoder architecture, that is, the ... i Other user behavior sequences with significant differences.

[0048] The second technical solution adopted by the present invention is: a product recommendation system based on hierarchical contrastive learning, comprising:

[0049] The user representation learning module, based on the Auto-Encoder architecture, introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence;

[0050] The user representation fusion module is used to fuse the user's high-level interest representation with the user's low-level interest representation and optimize the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector;

[0051] The user probability prediction module is used to perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment;

[0052] The user product recommendation module is used to select the product with the highest probability of user interaction with the candidate product at the next moment as the next recommended product.

[0053] The beneficial effects of the method and system of the present invention are as follows: the present invention first adopts the Auto-Encoder architecture to learn the user's high-level interest representation, and then fuses the user's high-level interest features and low-level interest features through the attention fusion layer to generate the user's comprehensive representation. Finally, based on the user's comprehensive representation and the product information to be evaluated, the probability of interaction between the user and the product is output. In order to optimize the expressive ability of the high-level interest representation, the present invention designs a global comparison and self-comparison strategy. The global comparison strategy can bring together users with similar long-term interests, thereby enhancing the expressive ability of user features. The self-comparison strategy can improve the sensitivity of the user encoding layer to changes in user sequences, and adjust the learning process of user representation by perceiving the degree of change in semantic information. In order to optimize the learning process of the user's comprehensive representation, BPR optimization is adopted and a comprehensive comparison learning strategy is designed, which can assist in the mining of personalized user preferences and attempt to find relatively high-order user behavior patterns to enhance the high-order invariance of user representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flowchart of the steps of a product recommendation method based on hierarchical contrast learning of the present invention;

[0055] Figure 2 This is a structural block diagram of a product recommendation system based on hierarchical contrastive learning in the present invention;

[0056] Figure 3 Schematic diagram of the model optimization process framework of the overall product recommendation method of the present invention;

[0057] Figure 4 This is a schematic diagram of the Encoder structure in the Transformer of the present invention;

[0058] Figure 5 It is a schematic diagram of the user decoder architecture of the present invention;

[0059] Figure 6 It is a schematic diagram of the data transformation process of the user representation learning module of the present invention. DETAILED DESCRIPTION

[0060] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0061] Contrastive learning is a self-supervised learning method used to learn the general characteristics of a dataset without labels by allowing the model to learn which data points are similar or different. Its core idea is to learn the feature space of a dataset by shortening the distance between positive samples (similar sample data) and increasing the distance between dissimilar samples. The goal is to improve the expressiveness of data features. In the context of the problem of this invention, the key point of using contrastive learning technology to learn user representation (user interest preferences) lies in how to define positive and negative samples.

[0062] Existing methods for recommending product sequences include a general product sequence representation learning method in recommendation systems, a self-supervised recommendation method combined with contrastive learning, and a review recommendation method based on meta-learning and heterogeneous graph attention neural networks. However, all of these methods suffer from the following problems: negative sampling often only considers random selection, resulting in poor quality of negative samples and low training efficiency; sampling high-frequency popular products makes some unpopular products even less popular, and a large number of long-tail products cannot participate in training. This problem persists even with methods such as popularity suppression; although there are some improved negative sampling strategies, such as calculating the importance of each sampled negative item, these require the introduction of more auxiliary information or additional network modules, resulting in poor model scalability;

[0063] Therefore, in summary, the main problems of existing sequential recommendation algorithms based on contrastive learning are: the quality of the defined positive and negative samples is poor, there is a certain dependence on auxiliary information (reviews, ratings, product descriptions), and it is not easy to expand to other platforms or application scenarios that lack relevant auxiliary information;

[0064] In this paper, a sequential recommendation method based on hierarchical contrastive learning is proposed. In order to ensure the scalability of the solution, a multi-level, fine-grained contrastive learning strategy is designed by only using the interaction behavior sequence between users and items, which improves the learning of user behavior representation and the recommendation algorithm process, and ultimately improves the accuracy of sequential recommendation.

[0065] Reference Figure 1 and Figure 3 The present invention provides a product recommendation method based on hierarchical contrastive learning, which includes the following steps:

[0066] S1. Based on the Auto-Encoder architecture, it introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence;

[0067] Specifically, the Auto-Encoder architecture is a classic unsupervised learning method that allows the encoder to quickly learn a set of high-dimensional features as user interest preferences. However, to ensure the quality of interest features, more constraints need to be introduced to guide the encoder's learning process. In this solution, we introduced global comparison and self-comparison learning strategies in the user representation learning module, and used BPR optimization and designed a comprehensive comparison learning strategy in the next-item recommendation module.

[0068] S11. Based on the Auto-Encoder architecture, learn the interaction sequence between users and products;

[0069] Specifically, in the user representation learning module, the present invention adopts the Auto-Encoder architecture and designs global comparison and self-comparison optimization strategies to improve the learning process of users' high-level interests. Specifically, the module includes three parts: item embedding layer, user encoding layer and user decoding layer, including item embedding layer, user encoding layer and user decoding layer, wherein;

[0070] (1) The item embedding layer includes the commodity coding layer and the location coding layer;

[0071] Item encoding layer (item_embedding_layer): This layer consists of a (number of items + 1) × (item embedding dimension) trainable embedding layer, which converts item IDs into item encoding vectors to form the item embedding table. The parameters in the table are randomly initialized.

[0072] Position encoding layer (position_embedding_layer): uses the common relative position encoding, Sinusoidal Position Encoding, that is, the absolute position is represented by the sine and cosine functions, and the relative position is obtained by multiplying the two;

[0073] (2) User encoding layer (encoder): This layer adopts the Transformer-based encoder structure, as shown in the following example: Figure 4 As shown in the figure, the multi-head attention layer performs multi-channel calculations on the data and splices them together, the residual connection layer adds the input and output of the multi-head attention layer, and normalizes the data features through layer normalization, and then performs data mapping transformation through the feedforward network layer. Finally, based on the residual connection layer, the untransformed data and the transformed data are added accordingly, and the data features are normalized again through layer normalization.

[0074] User decoding layer (Decoder): This layer uses the decoder layer in the Transformer architecture. The specific structure of the model is as follows Figure 5 As shown in Figure 2, the user decoding layer is introduced to form the Auto-Encoder architecture together with the user encoding layer to achieve the reconstruction task. It is no longer used in the subsequent model fine-tuning and inference stages. The first three layers of the decoder process data in the same way as the first three layers of the encoder above, except that the input of the subsequent multi-head attention layer includes not only the decoder data, but also the encoder (Encoder) output data (such as Figure 5 The two input data are concatenated after multi-way inner product operations in the multi-head attention layer. The data are then directly added to the corresponding input data processed by the first three decoder layers. Layer normalization is then performed to normalize their features. The normalized data is then mapped and transformed by the feedforward network and added to the data. Finally, layer normalization is performed to normalize the data features again.

[0075] like Figure 6 The figure shows the data transformation process of the user representation learning module of the present invention, which provides a symbolic description to facilitate the display of the data transformation and processing process; assuming that the user set is U = {u 1, …,u |U|}, the item set is V = {v 1, …,v |V|}, |U| and |V| represent the total number of users and items respectively. For any user His interaction with items (e.g., which items he purchased) can be expressed as |s i | represents a behavior sequence s i length, Represents user u iThe ID of the purchased item. The data transformation process is as follows:

[0076]

[0077]

[0078]

[0079]

[0080]

[0081] In this solution, the last bit of the user encoding layer Encoder output is selected as the user's high-level interest representation Its expression is:

[0082]

[0083] In the above formula, f(u i ) represents the output of the user encoding layer of the Auto-Encoder architecture, Represents the user's high-level interest representation.

[0084] S12. Introduce global comparison and self-comparison learning strategies to optimize the learning process of S11.

[0085] Specifically, the core of contrastive learning lies in selecting highly discriminative positive and negative sample pairs. During training, this approach narrows the gap between user sequences and positive samples, and widens the gap between user sequences and their negative samples, thereby improving the quality of user embeddings. In this solution, based on user behavior sequences, the contrastive learning strategy is divided into three levels, from coarse to fine, namely, overall, fragment, and element.

[0086] (1) Holistic or global comparison: Treat the user's behavior sequence as a set and use set similarity measurement and related statistical information to find users with similar preferences. This comparative learning strategy is relatively crude, but simple and fast, does not require additional information, and is suitable for the initial stage of user representation learning.

[0087] (2) Segment-to-self comparison: By adding, deleting, and replacing the user's original behavior sequence, positive and negative samples are generated. This allows the user representation to be sensitive to small perturbations in the behavior sequence, perceive the degree of semantic change, and adjust the user representation accordingly. In this solution, self-comparison is used to optimize the representation learning module (Encoder) and improve the expressive power of the user representation.

[0088] (3) Element-level comparison: construct positive and negative samples based on user behavior sequences (low-level semantic space) and user interest preferences (high-level semantic space), respectively, to assist BPR in adjusting the comprehensive representation of users during the fine-tuning stage of the recommendation model.

[0089] S121, global comparison optimization learning process;

[0090] Motivation: Taking video clicks as an example, if two users watch the same videos but in different order, they can be considered to have highly similar interests. To bring these similar users closer together, we consider the user behavior sequence as a whole, treat the sequence as a set (ignoring the order), and query similar users based on the set similarity.

[0091] Method: In the entire dataset, we select positive and negative sample pairs for each user. First, based on the Jaccard similarity, we select the behavior sequence that is most similar to the current user as the positive sample. Negative samples are divided into random negative samples and hard negative samples: random negative samples are random negative samples that select a user sequence with a Jaccard similarity of 0 as a random negative sample, while hard negative samples are selected based on the user's popularity tendency. Specifically, we calculate the popularity tendency of each user, and then compare their difference with the current user in this indicator. We randomly select a user sequence from the 50 user sequences with the largest difference as a hard sample;

[0092] Detailed steps:

[0093] Calculate the similarity of behavior sequences among all users based on Jaccard, and select the most similar user sequence for each user as a positive sample (Jaccard_Sim);

[0094] Select a random negative sample of users (Jaccard similarity = 0);

[0095] Calculate the user's popularity tendency based on the frequency of item appearance in the user's behavior sequence, and select the user's hard negative sample behavior sequence (Pop_Pre);

[0096] The positive and negative samples are packaged into DataLoader and fed into the model during the pre-training phase. User representations are generated by Enocder, optimized using triplet loss, and item embeddings are updated in reverse order.

[0097] The popularity tendency of the user is: Assume that the user set is U = {u 1, …,u |U|}, the item set is V = {v 1, …,v |V|}, |U| and |V| represent the total number of users and items respectively. For any user His interaction with the item can be expressed as |s i | represents a behavior sequence s i Length. Traverse the user behavior sequence set s, count the frequency of all item occurrences, and get item - freq=(f1,…,f |V| ). For any user u i , his popularity tends to Refers to its behavior sequence s i The average value of the corresponding frequency of all items in is calculated as follows:

[0098]

[0099] S122, self-comparison optimization learning process;

[0100] Motivation: This approach constructs stronger positive and negative samples based on the model's own sequence and adjusts the user representation. This allows the model to adjust the output user representation in a timely manner based on small fluctuations in the input data, thereby enhancing the expressive power of the user representation.

[0101] Advantages: After pre-training (reconstruction + global comparison), the representation model has relatively coarse classification and is almost unable to distinguish small input changes (for example, deletion or replacement of the last item). However, after this stage of training, the encoder can amplify small fluctuations and analyze the degree of semantic change.

[0102] Method: Slice the original sequence to form positive samples, and add (for example, add a few items that have not been clicked) or replace (replace the last few bits with items that have not been clicked) the original behavior sequence to form negative samples.

[0103] Detailed steps: (can be performed in the same batch or the entire dataset)

[0104] Delete the last 1 to 2 digits of the original sequence and slice it to get the positive sample user

[0105] Replace the last 1 to 2 digits of the original sequence with negative items to obtain the replaced negative sample user

[0106] Insert 1 to 2 negative items at the end of the original sequence to obtain the added and modified negative sample user

[0107] The above positive sample users, replaced negative sample users and added and modified negative sample users are sent to the encoder of the user encoding layer, and the self-contrast loss function is used for optimization to output the user's high-level interest representation.

[0108] S2. Fusing the user's high-level interest representation with the user's low-level interest representation and optimizing the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector;

[0109] Specifically, the attention fusion layer uses Attention-Transformer, which has the same architecture as the user decoding layer Decoder, and is used to fuse the user's high-level interest representation (output of the user encoding layer). ) and the user's low-level interest representation (the product code of the user-item interaction behavior sequence), thereby obtaining the user's comprehensive representation vector c i , whose expression is:

[0110]

[0111] In the above formula, c i represents the comprehensive representation vector of the user, Represents the user's high-level interest representation, Represents the user's low-level interest representation;

[0112] The next-item recommendation module generates a comprehensive user representation through the attention fusion layer and performs a dot product with the candidate item code to obtain the probability of the user interacting with the candidate item at the next moment. The item with the highest interaction probability in the candidate set is used as the predicted next item.

[0113] S3. Perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment. Select the item with the highest probability of the user interacting with the candidate item at the next moment as the next recommended item.

[0114] Specifically, the high-level comparative learning strategy measures the optimization learning process;

[0115] Motivation: Our recommendation model integrates users' long-term (high-level) and short-term (low-level) interests, and then predicts the next likely item to be clicked based on historical behavior sequences. To improve comprehensive user representation learning and accelerate fine-tuning training of the recommendation model, we construct positive and negative samples based on user behavior sequences (low-level semantic space) and user interest preferences (high-level semantic space) for comparative learning.

[0116] Advantages: Based on users' long-term (high-level semantics) and short-term (low-level semantics) interests, users with similar comprehensive interests are further brought closer together, making it easier to discover higher-level similar behavior patterns.

[0117] Method: Based on the same batch, the encoder finds the nearest neighbors and second-nearest neighbors of each user's representation, and uses them as positive and negative samples of the user's long-term interest. For the original sequence, the nearest neighbors and second-nearest neighbors are calculated based on the item embedding, and used as positive and negative samples of short-term interest.

[0118] Detailed steps: (introduced in the same batch and at the final model fine-tuning stage)

[0119] Based on the encoder, the user representation is obtained, and the Euclidean distance is calculated within the batch to find the nearest neighbor and the next nearest neighbor, which are used as positive and negative samples of the user's long-term (interest) representation respectively;

[0120] The user behavior sequences within the batch are converted into item representation vectors using item embedding, and the nearest neighbor and second nearest neighbor are calculated using Hausdorff distance. The original behavior sequences corresponding to the nearest neighbor and second nearest neighbor are used as positive and negative samples of the user's short-term interest representation, respectively.

[0121] The obtained long-term and short-term representations are fed into the recommendation model, and a comprehensive representation is obtained through attention weighted fusion. The corresponding comprehensive representation is used as the input of triplet loss and optimized together with BPR.

[0122] Reference Figure 2 , a product recommendation system based on hierarchical contrastive learning, including:

[0123] The user representation learning module, based on the Auto-Encoder architecture, introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence;

[0124] The user representation fusion module is used to fuse the user's high-level interest representation with the user's low-level interest representation and optimize the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector;

[0125] The user probability prediction module is used to perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment;

[0126] The user product recommendation module is used to select the product with the highest probability of user interaction with the candidate product at the next moment as the next recommended product.

[0127] In summary, the overall algorithm process steps of the present invention are:

[0128] Input: user set U and product set V, user behavior sequence set

[0129] Randomly initialize the parameters of all trainable modules;

[0130] (1) Pre-training stage;

[0131] The auto-encoder architecture (representation module + user decoding layer) is optimized by reconstructing the loss and using global contrastive loss until the overall loss converges. Global contrastive learning treats user behavior sequences as a set (ignoring the order of user-item interactions) and defines positive and negative samples based on set similarity and user popularity trends. This aims to bring together users with similar long-term interests (stable interest preferences), thereby enhancing the expressiveness of user features.

[0132] (2) User representation correction stage;

[0133] The representation module is optimized through self-contrast loss until the loss converges. Self-contrast learning constructs positive and negative samples by slightly perturbing the original user behavior sequence, improving the encoder's sensitivity to changes in user sequences and adjusting the user representation learning process by sensing the degree of change in semantic information.

[0134] (3) Model fine-tuning stage;

[0135] The representation module and interactive recommendation module are jointly optimized using BPR loss and high-level contrast loss until the overall loss converges. Comprehensive contrastive learning constructs positive and negative samples based on users' high-level and low-level interest features, assists in mining personalized user preferences, and attempts to identify relatively high-level user behavior patterns to enhance the high-order invariance of user representations. BPR optimization is a classic optimization method in personalized recommendation. It provides item recommendations based on user interactions, aiming to maximize the difference between the positive and negative sample scores.

[0136] The three losses involved in the model are as follows:

[0137] Reconstruct the loss formula:

[0138]

[0139] In the above formula, for u i The interaction sequence reconstructed by the user at time t and before;

[0140] Triplet loss formula;

[0141] In this scheme, all comparison losses, namely global comparison, self-comparison, and high-level comparison losses, use triplet loss, which is expressed as:

[0142]

[0143] In the above formula, α represents the custom boundary value, f(u i ) represents the output of the user encoding layer of the Auto-Encoder architecture, u i Represents the user, Represents user u i The positively correlated user is the output of the historical interaction behavior sequence through the user encoding layer of the Auto-Encoder architecture, that is, ... i There are other user behavior sequences with certain similarities, Represents user u i The negative opposite user history interaction sequence is output by the user encoding layer of the Auto-Encoder architecture, that is, the ... i Other user behavior sequences with significant differences;

[0144] The BPR loss formula is:

[0145]

[0146]

[0147]

[0148] In the above formula, N s represents the negative product candidate set; I represents negative products, i.e. products that the user has not interacted with before (t+1). v represents the embedding of the positive product, σ(·) represents the Sigmoid activation function, which compresses the data into the range of (0, 1). represents the comprehensive representation vector generated by user i at time t, Represents the real interactive product of user i at time (t+1).

[0149] Based on the above method, the simulation experimental data of the present invention are as follows:

[0150] In order to evaluate the technical effect of this solution, the present invention uses Hit-rate and NDCG to evaluate the final sequence recommendation effect for some Amazon open source datasets;

[0151] Dataset Introduction

[0152] We used Amazon's open-source dataset to extract user interaction data from four business areas: sport, cloth, movie, and book. Specifically, we retained interactions with a user rating of 4 as positive feedback, and removed users with fewer than 5 interactions and projects with an initial interaction frequency of less than 5. We only retained interaction records after October 1, 2017, and sorted them by time. The specific statistical information is shown in Table 1.

[0153] Table 1 Statistics of the dataset

[0154]

[0155] Experimental setup

[0156] The experimental environment is python 3.9, pyotrch 1.7.1, and GPU Quadro RTX 8000;

[0157] Evaluation metrics: Hit rate@10, NDCG@10;

[0158] Other settings: The dimensions of both user and item representations are 64-dimensional, the batch size is 1024, the maximum sequence length is 50, the number of heads is uniformly set to 2, and the optimizer uses Amsgrad with a learning rate of 0.01. All baseline methods use the same embedding dimensions as our method, and all other hyperparameters are set according to the optimal parameters reported in their respective papers. The dataset is partitioned using the leave-one-out method, where the last digit of each interaction sequence is used as the test set, the second-to-last digit as the validation set, and the remaining digits as the training set for each user.

[0159] Comparison algorithms;

[0160] SAS: The first to use transformer for single-domain sequence recommendation and one of the most advanced baseline methods.

[0161] Bert4Seq: Drawing on the idea of BERT, it replaces the next item prediction with a cloze method, allowing the product information in the user sequence to be better integrated with the contextual information.

[0162] CL4SRec: First, contrastive learning is introduced into sequence recommendation, and the original user interaction sequence is used for data augmentation to obtain multi-angle self-supervision signals and maximize the consistent expression of the same sequence from different perspectives.

[0163] Recommended performance evaluation;

[0164] As shown in Table 2, the algorithm proposed in this paper significantly outperforms existing cutting-edge solutions in recommendation performance across various datasets. This demonstrates that single-level comparison often fails to fully utilize unsupervised semantic information, while the introduction of hierarchical contrastive learning can improve the learning process of user interest representations, thereby enhancing recommendation performance.

[0165] Table 2 Comprehensive performance evaluation

[0166]

[0167]

[0168] Ablation experiments;

[0169] The ablation experiment mainly explores the impact of contrastive learning at each level in this scheme on the final recommendation performance. The experimental results are shown in Table 3.

[0170] Table 3 Ablation experiment results

[0171]

[0172] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0173] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A product recommendation method based on hierarchical contrastive learning, characterized in that: The following steps are involved: Based on the Auto-Encoder architecture, it introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence; The user's high-level interest representation is fused with the user's low-level interest representation and the fusion process is optimized through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector; Perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment; The item with the highest probability of user interaction with the candidate item at the next moment is selected as the next recommended item; Based on the user's behavior sequence, the contrastive learning strategy is divided into three levels from coarse to fine: whole-fragment-element; The step of introducing global comparison and self-comparison learning strategies based on the Auto-Encoder architecture to optimize and learn the interaction sequence between users and products, and outputting the user's high-level interest representation and the reconstructed user-product interaction sequence, specifically includes: Obtain the interaction sequence between users and products, that is, the user's low-level interest representation; Input the interaction sequence between users and products into the Auto-Encoder architecture for learning. The Auto-Encoder architecture includes an item embedding layer, a user encoding layer, and a user decoding layer. A global contrastive learning strategy is introduced to optimize the learning process of the Auto-Encoder architecture through reconstruction loss function and global contrastive loss function, and the reconstructed user-item interaction sequence is output; A self-contrastive learning strategy is introduced to optimize the learning process of the item embedding layer and user encoding layer in the Auto-Encoder architecture through a self-contrastive loss function, outputting a high-level interest representation of the user; The step of introducing a global contrastive learning strategy, optimizing the learning process of the Auto-Encoder architecture through a reconstruction loss function and a global contrastive loss function, and outputting a reconstructed user-product interaction sequence specifically includes: Calculate the similarity of behavior sequences between all users based on Jaccard, and select the most similar user sequence for each user as a positive sample; Select the user with Jaccard similarity = 0 as a random negative sample; Calculate the user's popularity tendency based on the frequency of product appearances in the user's behavior sequence, and select the user's hard negative sample behavior sequence; The data including positive and negative samples are packaged into DataLoader and fed into the model during the pre-training phase. The user encoding layer generates the user's high-level interest representation to be optimized. This is optimized using the reconstruction loss function and the global contrast loss function. The product encoding layer is then updated in reverse order, and the reconstructed user-product interaction sequence is output. The self-contrastive learning strategy is introduced to optimize the learning process of the item embedding layer and the user encoding layer in the Auto-Encoder architecture through the self-contrastive loss function, and output the user's high-level interest representation. This step specifically includes: Delete the last 1-2 digits of the interaction sequence between the user and the product, and slice it to obtain the positive sample users; Replace the last 1-2 bits of the interaction sequence between the user and the product with negative items to obtain the replacement negative sample user; Insert 1-2 negative items at the end of the interaction sequence between the user and the product to obtain the added and modified negative sample users; The above positive sample users, replaced negative sample users and added and modified negative sample users are sent to the encoder of the user encoding layer, and the self-contrast loss function is used for optimization to output the user's high-level interest representation.

2. The product recommendation method based on hierarchical contrastive learning according to claim 1, characterized in that: The step of inputting the user-product interaction sequence into the Auto-Encoder architecture for learning specifically includes: The project embedding layer based on the Auto-Encoder architecture converts the product ID into a product code vector and uses the common relative position encoding. That is, the absolute position is represented by the sine and cosine functions, and the relative position is obtained by multiplying the two. The user encoding layer based on the Auto-Encoder architecture adopts a Transformer-based encoder structure, including a multi-head attention layer, a first residual connection layer, a first normalization layer, a feedforward network layer, a second residual connection layer, and a second normalization layer. The user encoding layer is used to encode the output results of the item embedding layer to obtain the user's high-level interest representation to be optimized; The user decoding layer based on the Auto-Encoder architecture adopts the decoder structure in the Transformer architecture, including a self-attention layer, a first residual connection layer, a first normalization layer, a multi-head attention layer, a second residual connection layer, a second normalization layer, a feedforward network layer, a third residual connection layer and a third normalization layer. The user decoding layer is used to decode the user's high-level interest representation to obtain a reconstructed user-product interaction sequence to be optimized.

3. The product recommendation method based on hierarchical contrastive learning according to claim 2, characterized in that: The step of fusing the user's high-level interest representation with the user's low-level interest representation and optimizing the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector specifically includes: The user's high-level interest representation and the user's low-level interest representation are fused through the attention fusion layer to obtain the user's comprehensive representation vector; The structure of the attention fusion layer is the same as the user decoding layer of the Auto-Encoder architecture. The user's high-level interest representation is the user's long-term interest preference, and the user's low-level interest representation is the user's short-term interest preference. The expression of the fusion processing is as follows: In the above formula, c i represents the comprehensive representation vector of the user, Represents the user's high-level interest representation, Represents the user's low-level interest representation.

4. The product recommendation method based on hierarchical contrastive learning according to claim 3, characterized in that: The step of performing a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment specifically includes: Based on the user encoding layer of the Auto-Encoder architecture, the user's high-level interest representation is obtained, and the Euclidean distance is calculated within the batch to obtain the calculation result; Based on the calculation results, the nearest neighbors and next nearest neighbors of the user's high-level interest representation are selected as positive and negative samples of the user's long-term interest preference representation respectively; The interaction sequences between users and items in a batch are converted into item representation vectors through item embedding. The nearest neighbors and second-nearest neighbors of the item representation vectors are calculated using Hausdorff distance. The original behavior sequences corresponding to the nearest neighbors and second-nearest neighbors of the item representation vectors are used as positive and negative samples of the user's short-term interest preference representation. Through Attention, the positive and negative samples representing the user's long-term interest preferences are weightedly fused with the positive and negative samples representing the user's short-term interest preferences to obtain the weighted fusion result; The weighted fusion result is input into the high-level contrastive learning strategy, and the BPR optimization is performed through the high-level contrastive loss function to obtain the probability of the user interacting with the candidate product at the next moment.

5. The product recommendation method based on hierarchical contrastive learning according to claim 4, characterized in that: The expression of BPR optimization is: In the above formula, N s represents the negative product candidate set; I represents negative products, i.e. products that the user has not interacted with before (t+1). v represents the embedding of the positive product, σ(·) represents the Sigmoid activation function, which compresses the data into the range of (0, 1). represents the comprehensive representation vector generated by user i at time t, Represents the real interactive product of user i at time (t+1).

6. The product recommendation method based on hierarchical contrastive learning according to claim 5, characterized in that: The global contrast loss function, the self contrast loss function, and the high-level contrast loss function all use triplet loss, and their expressions are: In the above formula, α represents the custom boundary value, f(u i ) represents the output of the user encoding layer of the Auto-Encoder architecture, u i Represents the user, Represents user u i The positively correlated user is the output of the historical interaction behavior sequence through the user encoding layer of the Auto-Encoder architecture, that is, ... i There are other user behavior sequences with certain similarities, Represents user u i The negative opposite user history interaction sequence is output by the user encoding layer of the Auto-Encoder architecture, that is, the ... i Other user behavior sequences with significant differences.

7. A product recommendation system based on hierarchical contrastive learning, characterized in that: The method for recommending products based on hierarchical contrastive learning according to claim 1 comprises the following modules: The user representation learning module, based on the Auto-Encoder architecture, introduces global comparison and self-comparison learning strategies to optimize the interaction sequence between users and products, outputting the user's high-level interest representation and the reconstructed user-product interaction sequence; The user representation fusion module is used to fuse the user's high-level interest representation with the user's low-level interest representation and optimize the fusion process through a high-level contrastive learning strategy to obtain the user's comprehensive representation vector; The user probability prediction module is used to perform a dot product calculation on the reconstructed user-item interaction sequence and the user's comprehensive representation vector to obtain the probability of the user interacting with the candidate item at the next moment; The user product recommendation module is used to select the product with the highest probability of user interaction with the candidate product at the next moment as the next recommended product.

Images