Sequence recommendation method and system and medium

Abstract

The invention discloses a sequence recommendation method and system and a medium, and belongs to the technical field of information processing, and the method comprises the following steps: obtaining a historical behavior sequence corresponding to a user, and embedding two different levels of sequence data Su and # imgabs0 #; modeling user item-level and feature-level preferences using an attention mechanism encoder; the learnable group clustering module learns potential interest groups among the users and distributes each user to different interest groups; source embedding fusion module carefully aggregates information of different sources to obtain a final user interest representation Hfinal, and finally calculates a recommendation result # imgabs1 # by using the obtained final user interest representation

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to a sequence recommendation method, system and medium. Background Art

[0002] In the era of information explosion, individuals are faced with a large amount of data every day. Identifying meaningful and relevant information from this rich information has become a key challenge, and the development of recommendation systems has emerged.

[0003] Recommendation systems use user historical behavior records and item attributes to help users efficiently discover content that matches their interests. Among them, sequential recommendation has received significant attention, focusing on modeling user preferences based on their historical interaction sequences to more accurately predict future behaviors. Sequential recommendation aims to solve the information overload problem in many online scenarios by modeling the order of users' historical interaction behaviors and recommending the next potential item of interest to users. Traditional sequential recommendation methods usually regard each user's behavior as independent and capture user preferences by modeling individual historical interaction sequences. Such methods ignore the potential complex interactions between users, resulting in poor recommendation performance. Some state-of-the-art methods can effectively capture their common preferences by grouping users with similar behavior patterns together, thereby improving the accuracy of the recommendation system. The intuition behind this is that similar users have highly consistent preferences and decision-making patterns.

[0004] Existing studies usually only capture user group interests at the item level, relying on a single user behavior sequence, and do not consider the impact of other side information on recommendation performance. However, user behavior data usually contains multiple types of side information, such as product brands and categories. Effective use of this information can capture user group interests more carefully at the feature level. For example Figure 3 As shown in Figure 2, although the user's behavior sequence can classify them as electronic product enthusiasts at the item level, it reveals their preference for specific brands (such as Brand A) at the feature level. This neglect makes it impossible for existing methods to fully capture feature-level details such as user brand preferences, limiting the comprehensive understanding of their true interests.

[0005] It remains challenging to effectively utilize information from different sources in recommendation systems. Existing methods usually use simple methods such as pooling or concatenation to process information from different sources. However, in practice, these methods cannot effectively utilize multi-source information due to the heterogeneity of information from different sources and the semantic gap caused by generation by different encoders. Without careful fusion and complementarity, multi-source information may not improve recommendation performance, on the contrary, it may reduce the overall recommendation effect.

[0006] Based on this, the present invention designs a sequence recommendation method, system and medium to solve the above problems. Summary of the invention

[0007] In view of the above shortcomings of the prior art, the present invention provides a method, system and medium for sequential recommendation. By encoding the user historical interaction information at the project level and feature level, the project level and feature level user embeddings are obtained, and then the users are grouped according to their potential interests at the project level and feature level to obtain the user potential group interest embeddings at the project level and feature level. Finally, the embeddings from four different sources are carefully processed by the multi-source information embedding module to obtain the final accurate interest representation of the user, so as to calculate the recommendation score, thereby improving the recommendation accuracy.

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] A sequence recommendation method comprises the following steps:

[0010] S1: Get the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence

[0011] S2: Sequence data S of two different levels u and Embed and get the first user embedding E at the project level i and the second user embedding E at the feature level f ;

[0012] S3: Use two independent parameter attention mechanism encoders to model user item-level and feature-level preferences, and obtain the item-level user first preference mode representation H i and the user's second preference mode at the feature level is represented by H f ;

[0013] S4: Represent the user's first preference mode at the item level H i and the user's second preference mode at the feature level is represented by H f They are input into the learnable group clustering module to learn the potential interest groups among users and assign each user to a different interest group to obtain the first group interest embedding at the user-item level. and the second group interest embedding at the user feature level

[0014] S5: The obtained user's first preference mode is represented as H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Input into the multi-source embedding fusion module, the information from these four different sources is carefully aggregated to obtain the final user interest representation H final Finally, the final user interest representation is used to calculate the recommendation result

[0015] Furthermore, in step S1, the item-level historical behavior sequence S u Refers to a sequence consisting of project numbers, a feature-level historical behavior sequence It refers to the characteristics corresponding to the project, which include brand, type and price.

[0016] Furthermore, the specific steps of the attention mechanism encoder described in step S3 are as follows:

[0017] S31: Perform a linear transformation on the user embedding using the projection matrix and then input it into the scaled dot product attention as follows:

[0018] H=SDPA(E i W Q ,E i W K ,E i W V )

[0019] Where W Q , W K , W V is the projection matrix, E i is the user embedding, SDPA represents the scaled dot product attention function, which is described as follows:

[0020]

[0021] Among them, Q represents query, K represents key, V represents Value, and d represents embedding dimension;

[0022] S32: Use the multi-head attention mechanism in the self-attention mechanism. The formula is as follows:

[0023] M i =MH(E i )=Concat(h1,h2,...,hl f )W o

[0024] h i =SDPA(E i W i Q ,E i W iK ,E i W i V )

[0025] Where W o ,W i Q ,W i K ,W i V are all learnable parameters, lf represents the number of attention heads in the item encoder;

[0026] S33: The final output of the item-level self-attention block is represented as follows:

[0027] M i =LayerNorm(M i +E i ),

[0028] O i =ReLU((M i W1+b1)W2+b2),

[0029] O i =LayerNorm( i +M i ),

[0030] Among them, the weight W * and bias b * are learnable model parameters, O i is the output obtained after a single self-attention block;

[0031] The entire self-attention block is defined as follows:

[0032] O i =SAB(E i )

[0033] S34: In order to capture more complex feature conversion patterns, multiple self-attention blocks are stacked in the self-attention mechanism encoder. The qth self-attention block is defined as follows:

[0034]

[0035] in,

[0036] S35: Then The final output is obtained through a fully connected layer:

[0037]

[0038] Among them, W iis the weight of the fully connected layer, b i is the bias of the fully connected layer, H i This is the final user preference embedding.

[0039] Furthermore, the learnable group clustering module described in step S4 includes the following steps:

[0040] S41: Calculation of group correlation;

[0041] S42: Construction of group embeddedness;

[0042] S43: Calculate the user group embedding representation.

[0043] Furthermore, the specific steps of the learnable group clustering module described in step S4 are as follows:

[0044] S41: Calculation of group correlation;

[0045] First, initialize an interest pooling matrix W P ; Then, use the following formula to calculate the user's interest in various groups:

[0046] C P =MLP(W P H i )

[0047] Among them, C P represents the user's interest score for each group, W P Represents the initial interest pooling matrix;

[0048] S42: Construction of group embeddedness;

[0049] First, initialize a group embedding G o To represent the initial interest group; then, obtain the final group embedding representation:

[0050] G = MLP (GA (G o W Q ,H i W K ,H i ,W V ))

[0051] Where W Q , W K , W V is the parameter to be learned, G is the final group embedding representation, and GA(·) represents the group attention function, which is defined as follows:

[0052]

[0053] Among them, Q represents query, K represents key, V represents Value, and D represents embedding dimension;

[0054] S43: Calculate the user group embedding representation;

[0055] After obtaining the user interest group score C P After the group embedding representation G is obtained, the user group embedding representation is calculated to represent the user's group interests. The calculation formula is as follows:

[0056]

[0057] in, is the user's interest group representation, and softmax represents the soft maximum function.

[0058] Furthermore, the multi-source embedding fusion module described in step S5 includes the following steps:

[0059] S51: The multi-source embedding fusion module consists of a multi-layer perceptron and multi-source factor bilinear pooling. The process of fusing multiple sources using multi-source factor bilinear pooling is as follows: First, the matrices are processed and activated through a fully connected layer, and then they are element-wise multiplied; next, this result passes through a Dropout layer to produce the final output H to , the specific formula is as follows:

[0060]

[0061] Among them, σ(·) represents the Sigmod activation function, FC(·) represents the fully connected network, and H to Indicates the output obtained after the operation;

[0062] S52: Next, to Ht o Apply sum pooling to reshape the dimension and obtain the final output H MFB ;

[0063] S53: In order to capture the nonlinear relationship between multi-source embeddings, a multi-layer feedforward neural network enhanced by a residual module based on a multi-layer perceptron is added. The formula is as follows:

[0064]

[0065] Among them, H MLP is the final output, FNN represents the feed-forward neural network, l represents the number of layers in the fully connected network, and the L2 normalization function is applied to ensure a comparable representation with the output from the MFB module;

[0066] S54: Finally, H MLP and H MFBConcatenate them and project them into a fully connected layer with the following formula:

[0067] H final =[H MLP |H BFB ]W f +b f

[0068] Among them, W f is the weight of the fully connected layer, b f is the bias, H final It is the final output, used to represent the user's precise interests.

[0069] Furthermore, the final output is H MFB The formula is as follows:

[0070] H MFB =Norm(SumPooling(H to ,k))

[0071] Among them, k is a parameter used to control the size of non-overlapping windows, and Norm represents the power normalization and L2 normalization functions.

[0072] Furthermore, in step S5 The calculation method is as follows:

[0073]

[0074] in, Represents the transpose of the embedding of the target item; the model is optimized using the cross entropy loss function:

[0075]

[0076] Among them, y i represents the true score of target item i, represents the prediction score of target item i, and M represents the number of items in the user interaction sequence.

[0077] In order to better achieve the purpose of the present invention, the present invention also provides a sequence recommendation system, comprising:

[0078] The acquisition unit is used to obtain the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence

[0079] Embedding unit, used to embed two different levels of sequence data S u and Embed and get the first user embedding E at the project level iand the second user embedding E at the feature level f ;

[0080] The attention mechanism encoder is used to model the user's item-level and feature-level preferences, and obtain the item-level user's first preference mode representation H i and the user's second preference mode at the feature level is represented by H f ;

[0081] Learnable group clustering module, used to learn potential interest groups among users, assign each user to different interest groups to obtain the first group interest embedding at the user-item level and the second group interest embedding at the user feature level

[0082] Multi-source embedding fusion module is used to represent the user's first preference mode H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Aggregate and get the final user interest representation H final ;

[0083] The computing unit is used to utilize the final user interest representation H final To calculate the recommendation results And output the result.

[0084] In order to better achieve the purpose of the present invention, the present invention also provides a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the sequence recommendation method.

[0085] Compared with the prior art, the present invention has the following beneficial effects: the method of the present invention captures user preferences by combining sequence information at the item level and feature level. In order to have a deeper understanding of user behavior, the present invention designs a learnable group clustering module, which can group users at the item level and feature level, thereby capturing user interests more comprehensively. In addition, the present invention also introduces a multi-source embedding fusion module, which can effectively integrate information from different sources, thereby improving the robustness and accuracy of the model, thereby improving the accuracy of the model and the generalization ability of the network. It can be better applied to various shopping apps, giving users a better shopping experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0087] Figure 1 A process for the method recommended for a sequence of the present invention Figure 1 ;

[0088] Figure 2 A process for the method recommended for a sequence of the present invention Figure 2 ;

[0089] Figure 3 This is an example diagram of group clustering obtained from different perspectives in the present invention;

[0090] Figure 4 This is an example diagram of the attention mechanism encoder in the present invention. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0092] Embodiment 1: In some embodiments, please refer to the drawings of the specification Figure 1 , Figure 2 and Figure 4 ,A sequence recommendation method ,named FAGIMSR, includes the following steps:

[0093] S1: Get the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence

[0094] Item-level historical behavior sequence S u Refers to a sequence consisting of project numbers, a feature-level historical behavior sequence It refers to a sequence composed of the characteristics corresponding to the project, such as project brand, project type, project price, etc.

[0095] Specifically, a historical behavior sequence refers to a record of a series of purchase behaviors that a user has performed continuously over a period of time. These behaviors are arranged in chronological order to form an interaction sequence. For example, a user's historical behavior sequence may be [clothes, pants, computer], which means that the user has purchased these three items in sequence. Each item represents a historical interaction behavior of the user, and the entire sequence is a collection of all items that the user has interacted with.

[0096] At the item level, the historical behavior sequence S u In the present invention, the focus is on the specific commodities that the user has interacted with. For example, if a user has purchased a computer, a keyboard, and a mouse in sequence, then these three commodities constitute the item-level historical behavior sequence of the user, namely {computer, keyboard, mouse}.

[0097] At the feature level, historical behavior sequences In this paper, we focus on the user's behavioral characteristics at a finer granularity, such as the specific attributes of goods or brand preferences. This sequence can reveal the user's interests and preferences in more depth. For example, a user may always prefer to buy a certain brand of electronic products, and his feature-level historical behavior sequence may include {brand A computer, brand A mobile phone, brand A tablet}. In this paper, we use U = {u1,u2,...,u N} represents all users, I={i1,i2,...,i M} represents all items, S = {s1,s2,...,s |S|} represents the sequence of all items that user u has interacted with, Represents the corresponding item feature sequence.

[0098] The present invention uses data sets disclosed by Amazon and Movielens in the field of recommendation systems. These data sets cover user information, historical interaction items of users (i.e., historical behavior sequences of users), and specific times of interactions. In addition, the data set also contains a complete project set, which includes all items, and the items that the user has interacted with are a subset of this set. Different data sets contain different information in the project set. Taking the "Amazon-sports" data set provided by Amazon as an example, its user interaction history data set includes the following information: user ID, interaction item ID, user rating of the item, and timestamp (i.e., the time when the interaction occurred); and the project data set includes: project ID, project name, sales type, category, price, brand and other detailed information.

[0099] In order to ensure the fairness and effectiveness of the modeling model, the present invention removes all items and users with less than 5 interactions from the data set, thereby obtaining the final user historical behavior sequence. The "interaction" here refers to the interaction between users and items. For example, on Taobao, when a user adds a piece of clothing to the shopping cart while browsing the goods, this constitutes an interaction between the user and the clothing. "Interacting once" for an item means that the item has been clicked, purchased, etc. by a user, while "less than 5 times" means that the total number of times the item has been clicked or purchased is less than 5 times (this refers to the total number of interactions of all users, not just one user). Similarly, "interacting once" for a user means that the user has clicked, purchased, or added to the shopping cart for a certain item. Those users with less than 5 interactions mean that the total number of clicks or purchases made by these users on all items is less than 5 times.

[0100] S2: Sequence data S of two different levels u and Embed and get the first user embedding E at the project level i and the second user embedding E at the feature level f .

[0101] E i The embedding representation of the user's item-level historical behavior sequence, E f An embedding representation that represents a user’s feature-level historical behavior sequence.

[0102] Specifically, the present invention uses a library function to convert sequence data into an embedding vector by mapping, thereby obtaining the initialized project-level project-level first user embedding E i and the second user embedding E at the feature level f , so that the model can capture the user's interests and preferences more carefully. In this way, the present invention can more comprehensively understand the user's behavior pattern and provide richer user feature information for the recommendation system.

[0103] In order to further capture time series information, the present invention adds position coding. Specifically, the present invention adds a position coding vector to each embedding vector, and the coding vector can represent the position information of each element in the sequence data. In this way, the model can not only capture the interests and preferences of users, but also understand the changing trends of these interests and preferences over time. By combining position coding, the model of the present invention can better handle the temporal characteristics of sequence data, thereby more accurately predicting the future behavior of users in the recommendation system. In addition, the introduction of position coding also enables the model to assign different weights to user behaviors at different time points, thereby enhancing the model's sensitivity to dynamic changes in user interests. In this way, the present invention can not only fully understand the user's behavior patterns, but also dynamically capture the evolution of user interests, providing more accurate and personalized recommendation results for the recommendation system.

[0104] S3: Use two independent parameter attention mechanism encoders to model user item-level and feature-level preferences, and obtain the item-level user first preference mode representation H i and the user's second preference mode at the feature level is represented by H f .

[0105] Specifically, a preference pattern refers to the sequential habits that users exhibit when purchasing goods. For example, from the item level, if a user's historical purchase sequence is {mobile phone, mobile phone case, screen protector}, then it can be understood that after purchasing a mobile phone, the user is likely to purchase a mobile phone case, and after purchasing the mobile phone case, the next purchase behavior may be a screen protector. Correspondingly, from the feature level, if users always tend to buy a certain brand of electronic products, then their feature-level preference pattern may be expressed as {Brand A mobile phone, Brand A tablet, Brand A headphones}, which indicates that after users choose Brand A products, subsequent purchases may continue to prefer other products of the brand. Each purchase behavior reflects the regular changes in user preferences, and this regular change is what the present invention refers to as a preference pattern. Through these two levels of preference pattern representation, the present invention can more comprehensively capture users' shopping habits and tendencies.

[0106] The present invention learns the user's preference pattern by using an attention mechanism encoder. The reason for using an attention mechanism encoder to learn user preferences is that the attention mechanism can automatically identify and emphasize the most important part of the sequence data for the prediction task, thereby improving the model's ability to capture key features in user behavior. This mechanism allows the model to assign different attention weights in different contexts, so that the model can learn the user's diverse interests and preferences more flexibly and effectively, rather than simply averaging all information. In addition, the attention mechanism can also handle long-distance dependency problems, which is crucial for understanding long-term behavior patterns in time series data. Therefore, through the attention mechanism encoder, the model of the present invention can more accurately model the user's dynamic preferences and provide more accurate personalized recommendations for the recommendation system.

[0107] The specific steps of the attention mechanism encoder described in step S3 are as follows:

[0108] S31: Perform a linear transformation on the user embedding using the projection matrix and then input it into the scaled dot product attention as follows:

[0109] H=SDPA(E i W Q ,E i W K ,E i W V )

[0110] Where W Q , W K , W V is the projection matrix, E i is the user embedding, SDPA represents the scaled dot product attention function, which is described as follows:

[0111]

[0112] Among them, Q represents query, K represents key, V represents Value, and d represents embedding dimension;

[0113] S32: In order to allow the model to simultaneously focus on various representation subspaces at different positions, a multi-head attention mechanism (MH) is used in the self-attention mechanism. The formula is as follows:

[0114] M i =MH(E i )=Concat(h1,h2,...,hl f )W o

[0115] h i =SDPA(E i W iQ ,E i W i K ,E i W i V )

[0116] Where W o ,W i Q ,W i K ,W i V are all learnable parameters, lf represents the number of attention heads in the item encoder;

[0117] S33: The self-attention block also contains residual connections, layer normalization, and a two-layer fully connected network with ReLU activation function, which improves the overall performance of the self-attention block. The final output of the project-level self-attention block can be expressed as follows:

[0118] M i =LayerNorm(M i +E i ),

[0119] O i =ReLU((M i W1+b1)W2+b2),

[0120] O i =LoyerNorm(O i +M i ),

[0121] Among them, the weight W * and bias b * are learnable model parameters, O i is the output of a single self-attention block. For simplicity, the entire self-attention block (SAB) is defined as follows:

[0122] O i =SAB(E i )

[0123] S34: In order to capture more complex feature conversion patterns, multiple self-attention blocks are stacked in the self-attention mechanism encoder. The qth (q>1) self-attention block is defined as follows:

[0124]

[0125] in,

[0126] S35: Then The final output is obtained through a fully connected layer:

[0127]

[0128] Among them, W i is the weight of the fully connected layer, b i is the bias of the fully connected layer, H i This is the final user preference embedding.

[0129] The method proposed in the present invention adopts a decoupling strategy and uses a multi-head self-attention network to process the user's item-level information and the item's feature-level information separately to capture the user's preference transfer pattern. Specifically, the multi-head self-attention mechanism involves splitting the input sequence into multiple heads, calculating the attention weight for each head separately, and merging the results of these heads. This method enables the model to focus more on different parts of the sequence and allows different heads to learn different feature representations. For example, suppose a user used to buy science fiction novels but recently became interested in history books. The multi-head self-attention mechanism can identify this shift in interest by learning different feature representations. The model calculates the user's feature representations at different time points through the multi-head self-attention mechanism and compares the similarity of these representations to identify the user's preference transfer.

[0130] S4: The item-level user first preference pattern obtained from S3 is represented as H i and the user's second preference mode at the feature level is represented by H f , respectively input into the learnable group clustering module to learn the potential interest groups among users and assign each user to a different interest group to obtain the first group interest embedding at the user-item level and the second group interest embedding at the user feature level

[0131] Specifically, by grouping users with similar behavior patterns together, their common preferences can be effectively captured, thereby improving the accuracy of the recommendation system. The intuition behind this is that similar users have highly consistent preferences and decision-making patterns. Therefore, if users are divided into different groups and more reliable item dependencies within each group are used, users' preferences and interests can be more deeply understood from a group perspective. For example, on a music streaming platform, by analyzing users' listening habits, such as the time, frequency, and song types of listening, the platform can identify different user groups. For example, some users like to listen to relaxing jazz and blues after 10 pm, perhaps because they need to relax after work; other users listen to fast-paced pop music and electronic dance music when exercising in the morning or afternoon to stay energetic; and some users listen to classical music or ambient music in a quiet environment, such as a library or a personal room, to help concentrate. By classifying these users with similar listening habits into different groups, the platform is able to provide more personalized music recommendations based on the specific preferences of each group, thereby improving user satisfaction and the attractiveness of the platform.

[0132] The specific steps of the learnable group clustering module described in step S4 are as follows:

[0133] S41: Calculation of group relevance. The present invention uses a learnable soft clustering assignment matrix to calculate the relevance between users and different groups. First, initialize an interest pooling matrix W P . The following formula can then be used to calculate the user's interest in various groups:

[0134] C P =MLP(W P H i )

[0135] Among them, C P represents the user's interest score for each group, W P Represents the initial interest pooling matrix.

[0136] S42: Construction of group embedding. First, initialize a group embedding G o To represent the initial interest group. Then, obtain the final group embedding representation:

[0137] G = MLP (GA (G o W Q ,H i W K ,H i ,W V ))

[0138] Where W Q , W K , WV is the parameter to be learned, G is the final group embedding representation, and GA(·) represents the group attention function, which is defined as follows:

[0139]

[0140] Among them, Q represents query, K represents key, V represents Value, and D represents embedding dimension;

[0141] S43: Calculate the user group embedding representation. After obtaining the user interest group score C P After the group embedding representation G is obtained, the user group embedding representation is calculated to represent the user's group interests. The calculation formula is as follows:

[0142]

[0143] in, is the user's interest group representation, and softmax represents the soft maximum function.

[0144] S5: The obtained user first preference mode is represented as H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Input into the multi-source embedding fusion module, the information from these four different sources is carefully aggregated to obtain the final user interest representation H final Finally, the final user interest representation is used to calculate the recommendation result

[0145] The multi-source embedding fusion module described in step S5 includes the following steps:

[0146] S51: The multi-source embedding fusion module consists of a multi-layer perceptron (MLP) and a multi-source factor bilinear pooling (MFB).

[0147] The process of fusing multiple sources using multi-source factorial bilinear pooling (MFB) is as follows: First, the matrices are processed and activated through a fully connected layer, and then they are element-wise multiplied. Next, this result passes through a Dropout layer to produce the final output H to , the specific formula is as follows:

[0148]

[0149] Among them, σ(·) represents the Sigmod activation function, FC(·) represents the fully connected network, and H to Indicates the output obtained after the operation;

[0150] S52: Next, to Ht o Apply sum pooling to resize and get the final output H MFB The specific steps and formulas are as follows:

[0151] H MFB =Norm(SumPooling(H to ,k))

[0152] Among them, k is a parameter used to control the size of non-overlapping windows, and Norm represents the power normalization and L2 normalization functions, which are used to avoid potential local optimal risks that may occur during the operation.

[0153] S53: In order to capture the nonlinear relationship between multi-source embeddings, the present invention designs a multi-layer feedforward neural network enhanced by residual modules based on multi-layer perceptrons (MLPs), and the formula is as follows:

[0154]

[0155] Among them, H MLP is the final output, FNN represents the feed-forward neural network, l represents the number of layers in the fully connected network, and an L2 normalization function is applied to ensure a comparable representation to the output from the MFB module.

[0156] S54: Finally, H MLP and H MFB Concatenate them and project them into a fully connected layer with the following formula:

[0157] H final =[[ HMLP |H BFB] W f +b f

[0158] Among them, W f is the weight of the fully connected layer, b f is the bias, H final It is the final output, used to represent the user's precise interests.

[0159] Said The calculation method is as follows:

[0160]

[0161] in, Represents the transpose of the embedding of the target item. In order to enable the model to better learn the optimization parameters, the present invention uses a cross entropy loss function to optimize the model of the present invention:

[0162]

[0163] Among them, y i represents the true score of target item i, represents the prediction score of target item i, and M represents the number of items in the user interaction sequence (sequence length).

[0164] The present invention proposes a sequence recommendation method named FAGIMSR, which captures user preferences by combining sequence information at the item level and feature level. In order to have a deeper understanding of user behavior, the present invention designs a learnable group clustering module, which can group users at the item level and feature level, thereby capturing user interests more comprehensively. In addition, the present invention also introduces a multi-source embedding fusion module, which can effectively integrate information from different sources, thereby improving the robustness and accuracy of the model, thereby improving the accuracy of the model and the generalization ability of the network. It can be better applied to various shopping apps and give users a better shopping experience.

[0165] Embodiment 2: In some embodiments, a sequence recommendation system includes:

[0166] The acquisition unit is used to obtain the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence

[0167] Embedding unit, used to embed two different levels of sequence data S u and Embed and get the first user embedding E at the project level i and the second user embedding E at the feature level f .

[0168] The attention mechanism encoder is used to model the user's item-level and feature-level preferences, and obtain the item-level user's first preference mode representation H i and the user's second preference mode at the feature level is represented by H f .

[0169] Learnable group clustering module, used to learn potential interest groups among users, assign each user to different interest groups to obtain the first group interest embedding at the user-item level and the second group interest embedding at the user feature level

[0170] Multi-source embedding fusion module is used to represent the user's first preference mode H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Aggregate and get the final user interest representation H final ;

[0171] The computing unit is used to utilize the final user interest representation H final To calculate the recommendation results And output the result.

[0172] Embodiment 3: In some embodiments, a computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the sequence recommendation method of embodiment 1.

[0173] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for sequence recommendation, characterized in that: The following steps are involved: S1: Get the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence S2: Sequence data S of two different levels u and Embed and get the first user embedding E at the project level i and the second user embedding E at the feature level f ; S3: Use two independent parameter attention mechanism encoders to model user item-level and feature-level preferences, and obtain the item-level user first preference mode representation H i and the user's second preference mode at the feature level is represented by H f ; S4: Represent the user's first preference mode at the item level as H i and the user's second preference mode at the feature level is represented by H f They are input into the learnable group clustering module to learn the potential interest groups among users and assign each user to a different interest group to obtain the first group interest embedding at the user-item level. and the second group interest embedding at the user feature level S5: The obtained user first preference mode is represented as H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Input into the multi-source embedding fusion module, the information from these four different sources is carefully aggregated to obtain the final user interest representation H final Finally, the final user interest representation is used to calculate the recommendation result 2. The method for sequence recommendation according to claim 1, characterized in that: In step S1, the item-level historical behavior sequence S u Refers to a sequence consisting of project numbers, a feature-level historical behavior sequence It refers to the characteristics corresponding to the project, which include brand, type and price.

3. The method for sequence recommendation according to claim 2, characterized in that: The specific steps of the attention mechanism encoder described in step S3 are as follows: S31: Perform a linear transformation on the user embedding using the projection matrix and then input it into the scaled dot product attention as follows: H=SDPA(E i IN Q ,E i IN K ,E i IN V ) Where W Q , W K , W V is the projection matrix, E i is the user embedding, SDPA represents the scaled dot product attention function, which is described as follows: Among them, Q represents query, K represents key, V represents Value, and d represents embedding dimension; S32: Use the multi-head attention mechanism in the self-attention mechanism. The formula is as follows: M i =MH(E i )=Concat(h1,h2,...,h lf )W o h i =SDPA(E i IN i Q ,E i IN i K ,E i IN i V ) Where W o ,W i Q ,W i K ,W i V are all learnable parameters, lf represents the number of attention heads in the item encoder; S33: The final output of the item-level self-attention block is represented as follows: M i =LayerNorm(M i +E i ), <h2 style=";text-align:left;direction:ltr">O<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> =ReLU((M<h2 style=";text-align:left;direction:ltr"> i <h2 style=";text-align:left;direction:ltr"> W1+b1)W2+b2), O i =LayerNorm(O i +M i ), Among them, weight W* and bias b* are learnable model parameters, O i is the output obtained after a single self-attention block; The entire self-attention block is defined as follows: THE i =SAB(E i ) S34: In order to capture more complex feature conversion patterns, multiple self-attention blocks are stacked in the self-attention mechanism encoder. The qth self-attention block is defined as follows: in, S35: Then The final output is obtained through a fully connected layer: Among them, W i is the weight of the fully connected layer, b i is the bias of the fully connected layer, H i This is the final user preference embedding.

4. The method for sequence recommendation according to claim 3, characterized in that: The learnable group clustering module described in step S4 includes the following steps: S41: Calculation of group correlation; S42: Construction of group embeddedness; S43: Calculate the user group embedding representation.

5. The method for sequence recommendation according to claim 4, characterized in that: The specific steps of the learnable group clustering module described in step S4 are as follows: S41: Calculation of group correlation; First, initialize an interest pooling matrix W P ; Then, use the following formula to calculate the user's interest in various groups: C P =MLP(W P H i ) Among them, C P represents the user's interest score for each group, W P Represents the initial interest pooling matrix; S42: Construction of group embeddedness; First, initialize a group embedding G o To represent the initial interest group; then, obtain the final group embedding representation: G=MLP(GA(G o W Q ,H i W K ,H i ,W V )) Where W Q , W K , W V is the parameter to be learned, G is the final group embedding representation, and GA(·) represents the group attention function, which is defined as follows: Among them, Q represents query, K represents key, V represents Value, and D represents embedding dimension; S43: Calculate the user group embedding representation; After obtaining the user interest group score C P After the group embedding representation G is obtained, the user group embedding representation is calculated to represent the user's group interests. The calculation formula is as follows: in, is the user's interest group representation, and softmax represents the soft maximum function.

6. The method for sequence recommendation according to claim 5, characterized in that: The multi-source embedding fusion module described in step S5 includes the following steps: S51: The multi-source embedding fusion module consists of a multi-layer perceptron and multi-source factor bilinear pooling. The process of fusing multiple sources using multi-source factor bilinear pooling is as follows: First, the matrices are processed and activated through a fully connected layer, and then they are element-wise multiplied; next, this result passes through a Dropout layer to produce the final output H to , the specific formula is as follows: Among them, σ(·) represents the Sigmod activation function, FC(·) represents the fully connected network, and H to Indicates the output obtained after the operation; S52: Next, to H to Apply sum pooling to reshape the dimension and obtain the final output H MFB ; S53: In order to capture the nonlinear relationship between multi-source embeddings, a multi-layer feedforward neural network enhanced by a residual module based on a multi-layer perceptron is added. The formula is as follows: Among them, H MLP is the final output, FNN represents the feed-forward neural network, l represents the number of layers in the fully connected network, and the L2 normalization function is applied to ensure a comparable representation with the output from the MFB module; S54: Finally, H MLP and H MFB Concatenate them and project them into a fully connected layer with the following formula: H final =[H MLP |H BFB ]W f +b f Among them, W f is the weight of the fully connected layer, b f is the bias, H final It is the final output, used to represent the user's precise interests.

7. The method for sequence recommendation according to claim 6, characterized in that: Final output H MFB The formula is as follows: H MFB =Norm(SumPooling(H to ,k)) Among them, k is a parameter used to control the size of non-overlapping windows, and Norm represents the power normalization and L2 normalization functions.

8. The method for sequence recommendation according to claim 7, characterized in that: In step S5 The calculation method is as follows: in, Represents the transpose of the embedding of the target item; the cross entropy loss function is used to optimize the model: Among them, y i represents the true score of target item i, represents the prediction score of target item i, and M represents the number of items in the user interaction sequence.

9. A sequence recommendation system according to claim 1, characterized in that: include: The acquisition unit is used to obtain the historical behavior sequence corresponding to the user, which is the item-level historical behavior sequence S u and feature-level historical behavior sequence Embedding unit, used to embed two different levels of sequence data S u and Embed and get the first user embedding E at the project level i and the second user embedding E at the feature level f ; The attention mechanism encoder is used to model the user's item-level and feature-level preferences, and obtain the item-level user's first preference mode representation H i and the user's second preference mode at the feature level is represented by H f ; Learnable group clustering module, used to learn potential interest groups among users, assign each user to different interest groups to obtain the first group interest embedding at the user-item level and the second group interest embedding at the user feature level Multi-source embedding fusion module is used to represent the user's first preference mode H i , user's second preference mode indicates H f , and the corresponding user's first group interest embedding Second group interest embedding Aggregate and get the final user interest representation H final ; The computing unit is used to utilize the final user interest representation H final To calculate the recommendation results And output the result.

10. A computer storage medium, characterized in that: The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the sequence recommendation method according to any one of claims 1 to 8.

Images