A contrastive learning sequence recommendation method based on dimensional relationship

Abstract

The present invention relates to a contrastive learning sequence recommendation method based on dimensional relationship, and belongs to the field of recommendation system. In order to cope with the problem of data sparsity, the present invention improves the problem that the sparsity of data sets in the sequence recommendation system leads to poor model training, and proposes a contrastive learning sequence recommendation method based on dimensional relationship. The present invention adopts a contrastive learning method to generate self-supervision signals by constructing dimensional relationships, and maps high-dimensional raw data into dense and expressive feature representations. By optimizing the item representation, high-quality user interest representations can be learned, which can effectively improve the performance of the recommendation system and provide a feasible solution for the development of sequence recommendation.

Description

Technical Field

[0001] The invention relates to a contrastive learning sequence recommendation method based on dimensional relationship, belonging to the technical field of recommendation systems. Background Art

[0002] In the era of information explosion, recommendation systems play an important role in satisfying user interests. In order to model users' dynamic interests, sequential recommendation predicts the next item that users may be interested in through the user's historical behavior sequence to provide personalized recommendations.

[0003] Although sequential recommendation has proven successful in a range of tasks, it suffers from the problem of data sparsity, which makes it difficult to learn high-quality representations of user interests. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a contrastive learning sequence recommendation method based on dimensional relationship, so as to solve the existing problem of data sparsity, which makes it difficult to learn high-quality user interest representation.

[0005] The technical solution of the present invention is: a contrastive learning sequence recommendation method based on dimensional relationship, the specific steps are:

[0006] Step 1: Query the user's historical browsing records, and clean and organize the data;

[0007] Step 1.1: Data reading, traversing the user's historical browsing records. If the individual user's browsing records are less than N, the user record will be deleted and will not participate in the subsequent model training;

[0008] Step 1.2: Data sorting: sort the adjusted user data in the order of the browsing time of each user. k ={(name k , item1), (name k , item2), ..., (name k , item m )}; where name represents the nickname k of users browse product items on the product shopping platform m , where m represents the number of items in the user's browsing data.

[0009] Step 2: Use the tour information to encode it to obtain sequence features;

[0010] Step 2.1: Select product items from the preprocessed user browsing data n As the predicted product, item n forward Encode the item sequence, item n ∈(item 1， item2, ..., item m ); the sub-elements in the sequence will be encoded into dimensions The overall dimension is , users will also be encoded as Vector ;

[0011] Step 2.2: By combining the sub-elements in the sequence in pairs, the elements at sequence position i and sequence position j are concatenated front and back to form a dimension of By traversing the entire sub-element, using the above method, a vector with dimension The three-dimensional vector .

[0012] Step 3: The three-dimensional vector generated in Step 2.2 , input into the contrastive learning framework for feature encoding to obtain the user's interest features. The specific steps are:

[0013] Step3.1: Convert the three-dimensional vector in Step2.2 Input to the singular value decomposition agent task to obtain the enhanced vector in dimension . Remove by singular value decomposition Redundant information within the dimension. Here the data enhancement method can be formulated as:

[0014]

[0015]

[0016] in, is a three-dimensional vector A vector on dimension t has a dimension size of . Is the size of The diagonal matrix of retains the first k elements of the diagonal as 1 and the rest of the elements as 0. By performing data augmentation in the dimension direction, a new three-dimensional vector can be obtained , the overall dimension is 。 represents an orthogonal left singular matrix, represents a diagonal matrix, It is an orthogonal right singular matrix, with the lower right corner t Indicates dimension, upper right corner T represents transpose, represents the enhanced diagonal matrix, Represents the new three-dimensional vector after data enhancement.

[0017] Step3.2: Convert the three-dimensional vector in Step2.2 Input to the SE-Block proxy task to obtain channel-related image feature maps by learning the importance between dimensions It goes through three stages: compression stage, excitation stage, and rescaling stage. Its formula is as follows:

[0018]

[0019] in yes Descriptors in the dimension direction, is the global average pooling operation. , The weight matrix of represents the ReLU activation function, represents the Sigmoid activation function, r is a hyperparameter used to control the compression ratio, and s is the attention weight obtained in the excitation phase. By performing data augmentation in the dimensional direction, a new three-dimensional vector can be obtained. , the overall dimension is .

[0020] Step3.3: Combine the results from Step2.2, Step3.1 and Step3.2 above Input to the 2D convolutional code to obtain its feature code, which are , the size dimension is The main structure of the encoder is shown in the following table:

[0021]

[0022] Step 3.4: Encode the features obtained in Step 3.3 above Input to the nonlinear projection head function Get the projection head code , the dimension size is , the calculation process is as follows:

[0023]

[0024] in, is the linear layer weight of the projection head, Dropout Indicates random inactivation, which is used to randomly set the output of some neuron parameters to 0. h represents the hidden layer embedding vector.

[0025] Step 3.5: Encode the projection head obtained in Step 3.4 above Input into the contrast loss function InfoNCE Loss to obtain the contrast loss The contrastive loss function is learned to minimize the difference between different enhanced views of the same image feature map and maximize the difference between different image feature maps derived from the same batch. , the above data enhancement method generates as positive sample pairs, and the other 2(N-1) enhanced samples in the same batch are considered Negative samples. The similarity between sample pairs is calculated by cosine similarity. Contrastive loss function The formula is as follows:

[0026]

[0027] in, is the temperature coefficient. The main function of the temperature coefficient is to adjust the emphasis on the distinction between positive and negative samples: the smaller the temperature coefficient, the more inclined to distinguish the current sample from the most similar negative samples.

[0028] Step 4: Transform the user vector User interest characteristics Concatenate and learn the user's interest representation through the fully connected layer. The specific steps are:

[0029]

[0030] in is the weight of the fully connected layer, where n is the number of products in the data set. This method can reflect the user's interest characteristics for all products. is the bias term in the fully connected layer.

[0031] Step 5: Characterize based on user interests Recommend user products, calculate the error between the prediction and the true value and optimize the network to obtain a sequence recommendation model for recommending user products. The specific steps are:

[0032] Step 5.1: The predicted value and the true value are calculated by using the binary cross entropy loss function as the main objective function to calculate the main loss . Its expression is:

[0033]

[0034] The above formula represents an extended form of the intersection-entropy loss function, which is mainly used for binary classification problems. is the loss value, which is calculated by summing over all samples u and time steps t. The first part of the formula is the positive sample S u The loss calculation performed is the logarithmic loss of the model's predicted probability for the positive class, part 2 The loss for negative samples is the logarithmic loss of the model's prediction error probability for negative samples. The purpose of this formula is to maximize the prediction probability of positive samples while minimizing the prediction error rate of negative samples, thereby optimizing the classification effect of the model by minimizing the overall loss.

[0035] Step 5.3: The contrast loss obtained in Step 3.5 , as an auxiliary task with the same main loss as Step 5.2 above Perform joint training. Its expression is:

[0036]

[0037] in To design the weight of the auxiliary task, it can control the proportion of the contrast loss participating in the loss function.

[0038] Step 5.3: According to the above loss value , optimize the network using the optimizer;

[0039] Step 5.4: Define the size of Epoch and the number of optimizations as needed, and repeat the above steps several times;

[0040] Step 5.5: Get the trained feature-fused multi-task recommendation model.

[0041] The beneficial effects of the present invention are:

[0042] This paper proposes a dimensional relationship-based contrastive learning framework to achieve data enhancement by contrastive learning of dimensional relationships between items. Two dimensional relationship-based data enhancement methods are also proposed to achieve data enhancement to solve the data noise problem, including Se-block and singular value decomposition.

[0043] Specifically, the items are encoded into three-dimensional vectors in pairs and input into the contrastive learning framework. The proxy tasks in the framework are used to obtain information between dimensions and within dimensions for data enhancement, and the enhanced sequence features are encoded using the CNN encoder. The nonlinear feature representation is then obtained through the projection head function. In the framework, the data representation enhanced by the proxy task is used to calculate the difference between different enhanced views through InfoNEC to calculate the contrastive learning loss. After the contrast framework, the user interest features are concatenated with the user features and the user vector to obtain the user recommendation encoding, and the fully connected layer is used to learn the user interest representation, through which the main target loss is calculated through the binary cross entropy loss function. Finally, the contrastive learning loss is used as an auxiliary task for joint training with the main task. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the steps of the present invention;

[0045] Figure 2 It is a schematic diagram of a model of the present invention;

[0046] Figure 3 Schematic diagram of the contrastive learning framework of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods.

[0048] Example 1: Figure 2 As shown in the figure, a comparative learning framework based on dimensional relationship is used. This example takes the public dataset Gowalla dataset as an example. The items clicked by each user are processed and then put into the built model to train the model. The specific process includes: processing the data into a data form that can be input into the model, batch reading the training data from the processed data, and then obtaining the sequence features through pairwise encoding, and then obtaining the dimension-enhanced interest features through the comparative learning framework, and obtaining the user interest representation by splicing the user vector and the user interest feature and passing it through the fully connected layer. Recommend products based on the user interest representation, calculate the error between the prediction and the true value, and optimize the above network to obtain the recommendation sequence model of the user product.

[0049] like Figure 1 As shown in FIG. 1 , a contrastive learning sequence recommendation method based on dimensional relationship is shown in FIG. 1 . The specific steps are as follows:

[0050] Step 1: Query the user's historical browsing records, clean and organize the data. The specific steps are as follows:

[0051] Step 1.1: Traverse the user's historical browsing records. If the individual user's browsing records are less than 5, the user record will be deleted and will not participate in the subsequent model training;

[0052] Step 1.2: Data sorting: sort the adjusted user data in the order of the browsing time of each user. k ={(name k , item1), (name k , item2), ..., (name k , item m )}; where name represents the nickname k of users browse product items on the product shopping platform m , where m represents the number of items in the user's browsing data.

[0053] Step 2: Use the tour information to encode it to obtain sequence features

[0054] Step 2.1: Encode the first five items of the user's predicted products. The sub-elements in the sequence will be encoded into a vector of dimension 128. The overall dimension is 5*128. The user will also be encoded as a vector of dimension 128. ;

[0055] Step 2.2: By combining the sub-elements in the sequence in pairs, the elements at sequence position i and sequence position j are concatenated to form a vector with a dimension of 256. By traversing the entire sub-element, using the above method, a three-dimensional vector with a dimension of 5*5*256 will be formed. .

[0056] Step 3: The three-dimensional vector generated in Step 2.2 , input into the contrastive learning framework for feature encoding to obtain the user's interest features, such as Figure 3 shown.

[0057] Step3.1: Convert the three-dimensional vector in Step2.2 Input to the singular value decomposition agent task to obtain the enhanced vector in dimension . Remove by singular value decomposition Redundant information within the dimension. Here the data enhancement method can be formulated as:

[0058]

[0059]

[0060] in, is a three-dimensional vector The vector on dimension t has a dimension size of 5*5. It is a 5*5 diagonal matrix with the first k elements of the diagonal being 1 and the rest being 0. By performing data augmentation in the dimension direction, a new three-dimensional vector can be obtained. , the overall dimension is 5*5*256.

[0061] Step3.2: Convert the three-dimensional vector in Step2.2 Input to the SE-Block proxy task to obtain channel-related image feature maps by learning the importance between dimensions It goes through three stages: compression stage, excitation stage, and rescaling stage. Its formula is as follows:

[0062]

[0063] in yes Descriptors in the dimension direction, is the global average pooling operation. , The weight matrix of represents the ReLU activation function, represents the Sigmoid activation function, r is a hyperparameter used to control the compression ratio, set to 8, and s is the attention weight obtained in the excitation phase. By performing data augmentation in the dimensional direction, a new three-dimensional vector can be obtained. , the overall dimension is 5*5*256.

[0064] Step3.3: Combine the results from Step2.2, Step3.1 and Step3.2 above Input to the 2D convolutional code to obtain its feature code, which are , the size dimension is 256. The main structure of the encoder is shown in Table 1 below:

[0065] Table 1 Encoder structure

[0066]

[0067] Step 3.4: Encode the features obtained in Step 3.3 above Input to the nonlinear projection head function Get the projection head code , the dimension size is 128, and the calculation process is as follows:

[0068]

[0069] in is the weight of the linear layer of the projection head.

[0070] Step 3.5: Encode the projection head obtained in Step 3.4 above Input into the contrast loss function InfoNCE Loss to obtain the contrast loss The contrastive loss function is learned to minimize the difference between different enhanced views of the same image feature map and maximize the difference between different image feature maps derived from the same batch. , the above data enhancement method generates are considered as positive sample pairs, and other enhanced samples in the same batch are considered Negative samples. The similarity between sample pairs is calculated by cosine similarity. Contrastive loss function The formula is as follows:

[0071]

[0072] in, is the temperature coefficient, which is set to 0.4. The main function of the temperature coefficient is to adjust the emphasis on the distinction between positive and negative samples: the smaller the temperature coefficient, the more likely it is to distinguish the current sample from the most similar negative samples.

[0073] Step 4: Transform the user vector User interest characteristics Concatenate and learn the user's interest representation through the fully connected layer. The specific steps are:

[0074]

[0075] in is the weight of the fully connected layer, where is the number of products in the data set. This method can reflect the user's interest characteristics for all products. is the bias term in the fully connected layer.

[0076] Step 5: Characterize based on user interests Recommend user products, calculate the error between the prediction and the true value and optimize the network to obtain a sequence recommendation model for recommending user products. The specific steps are:

[0077] Step 5.1: The predicted value and the true value are calculated by using the binary cross entropy loss function as the main objective function to calculate the main loss . Its expression is:

[0078]

[0079] Step 5.3: The contrast loss obtained in Step 3.5 , as an auxiliary task with the same main loss as Step 5.2 above Perform joint training. Its expression is:

[0080]

[0081] in To design the weight of the auxiliary task, it can control the proportion of the contrast loss participating in the loss function.

[0082] Step 5.3: According to the above loss value , optimize the network using the optimizer;

[0083] Step 5.4: Define the size of Epoch and the number of optimizations as needed, and repeat the above steps several times;

[0084] Step 5.5: Get the trained feature fusion multi-task recommendation model. The results are shown in Table 2, which shows the superiority of the model proposed in this paper. The abbreviations in the first row of the table are the names of the models. MovieLen and Gowalla are used as the verification data sets of this paper. For other model methods in the data set, 1. The sequence model (eg Caser) is better than the non-sequential model (eg BPR), proving that the sequence pattern has a significant improvement in model prediction. 2. Contrasting the sequence modeling form, FPMC is modeled by a first-order Markov chain, while CosRec achieves better results by capturing high-order sequence relationships. For the convolutional neural method in the data set, the three convolutional neural methods in the comparative experiment, Caser captures interest through a one-dimensional convolution kernel, and CosRec is modeled by pairwise encoding and then using two-dimensional convolution. Different from the above two, the model of the present invention introduces a contrastive learning framework to utilize the dimensional relationship of the image feature map to complete the learning of the contrastive framework, which generally exceeds the former in terms of model effect, and has a 2%-7% performance improvement on both the ML-1M and Gowalla data sets.

[0085] Table 2 Comparison of model performance of this method and other methods on ML-1M and Gowalla datasets

[0086]

[0087] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A contrastive learning sequence recommendation method based on dimensional relationship, characterized in that: The steps include: Step 1: Query the user's historical browsing records and pre-process the data, including cleaning and sorting; Step 2: Encode the tour information to obtain sequence features; Step 3: Use contrastive learning framework to encode sequence features to obtain user interest features; Step 4: Concatenate the user vector and user interest features and learn the user interest representation through the fully connected layer; Step 5: Recommend user products based on user interest representation, calculate the error between the prediction and the true value and optimize the network to obtain a sequence recommendation model for recommending user products; The specific steps of Step 1 are: Step 1.1: Read data and traverse the user's historical browsing records. If the number of browsing records of an individual user is less than 5, delete the browsing record of the user and do not participate in the subsequent model training; Step 1.2: Data sorting: sort the user data adjusted in Step 1.1 into the order of the visit time of each user. k ={(name k , item1), (name k , item2), ..., (name k , item m )}, indicating nickname name k of users browse product items on the product shopping platform m , m represents the number of items in the user's browsing data; The specific steps of Step 2 are: Step 2.1: Select product items from the preprocessed user browsing data n As the predicted product, item n forward Encode the item sequence, item n ∈(item 1， item2, ..., item m The sub-elements in the sequence are encoded into a matrix with dimensions The overall dimension is , users are encoded into dimensions Vector ; Step 2.2: Combine the sub-elements in the sequence in pairs and encode the sequence into a three-dimensional vector ,in, u Represents the user, t represents the time step, and the overall dimension is ; The specific steps of Step 3 are: Step3.1: Convert the three-dimensional vector in Step2.2 Input to the singular value decomposition agent task to obtain the enhanced vector in dimension , the overall dimension is ; Step3.2: Convert the three-dimensional vector in Step2.2 Input into the SE-Block proxy task to obtain the enhancement vector between dimensions , the overall dimension is ; Step 3.3: Input into the 2D convolutional network to obtain feature encoding, which are , the dimension size is ; Step 3.4: Encode the features obtained in Step 3.3 above Input to the nonlinear projection head function Get the projection head code , the dimension size is ; Step 3.5: Encode the projection head obtained in Step 3.4 above Input into the contrast loss function InfoNCE Loss to get the contrast loss ; The specific steps of Step 4 are: The feature vector With user vector Combined and fully connected layer to get user interest representation , whose expression is: ； in is the weight of the fully connected layer, is the bias term in the fully connected layer; The specific steps of Step 5 are: Step 5.1: The predicted value and the true value are calculated by using the binary cross entropy loss function as the main objective function to calculate the main loss ; Step 5.2: The contrast loss obtained in Step 3.5 , as an auxiliary task with the same main loss as Step 5.1 above For joint training, the expression is: ； in To design the weights of auxiliary tasks; Step 5.3: According to the above loss value , use Adam optimizer to optimize the network; Step 5.4: Get the trained feature-fused multi-task recommendation model to recommend products to users.

Images