An interest-guided cross-domain knowledge transfer sequence recommendation method

Abstract

The application discloses a cross-domain knowledge transfer sequence recommendation method based on interest guidance, which comprises the following steps: 1) shielding and filling the user interaction sequence to form two types of sequences, i.e., specific domain sequences and cross-domain shared sequences, and embedding and initializing the sequence items; 2) designing a mechanism of layer-by-layer dynamic integration based on a large language model, layer by layer, to strengthen the preference features of the user in the specific domain and effectively suppress interference information; 3) constructing a cross-domain shared interest extraction module to extract fine-grained user interest representation from the cross-domain shared sequences; 4) using a heterogeneous attention mechanism to fuse the cross-domain shared user interest and the behavior information of the specific domain; and 6) performing end-to-end training optimization to reduce the calculation burden in the model training stage. The method effectively solves the problems in cross-domain recommendation, such as not explicitly distinguishing the cross-domain shared and specific domain user preferences and insufficient user interest mining, and significantly improves the recommendation accuracy and personalization effect.

Description

Technical Field

[0001] This invention belongs to the field of cross-domain recommendation technology, specifically relating to an interest-guided cross-domain knowledge transfer sequence recommendation method, and particularly to modeling and learning user preferences of specific domains and user interests of shared domains in cross-domain scenarios. Background Technology

[0002] In today's information age, with the rapid development of the internet, massive amounts of data are constantly emerging, while users actually only encounter and interact with a tiny fraction of this total. Therefore, data sparsity remains a major challenge for recommender systems. Cross-domain sequence recommendation, as an effective solution, gradually alleviates this problem by transferring knowledge between different domains. However, a common learning strategy in this process is to integrate user sequences from various specific domains into cross-domain shared sequences, using these sequences as bridges to facilitate knowledge transfer between different domains. But this method indiscriminately introduces cross-domain information into the modeling of specific domains during training, inevitably including irrelevant noise features, thus interfering with the accurate modeling of target user preferences. Furthermore, traditional methods typically model user interests as a single global representation, ignoring the diversity of user interests in cross-domain scenarios. Therefore, there is an urgent need for a new cross-domain recommendation method designed to overcome these limitations and address rapidly changing user needs and complex cross-domain behavioral patterns. Therefore, this invention presents an interest-guided cross-domain knowledge transfer sequence recommendation method that can effectively identify and handle differences in user interests across different domains, reduce noise interference, and thus achieve more accurate and personalized recommendations. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and provide an interest-guided cross-domain knowledge transfer sequence recommendation method. This method breaks through the traditional cross-domain sequence recommendation representation learning method and can effectively handle the problem of inaccurate recommendations caused by the failure to clearly distinguish between cross-domain shared and specific domain user preferences and the modeling of user interests in a single way. By integrating specific domain user preferences layer by layer, extracting cross-domain shared interests, and performing cross-domain knowledge transfer fusion, the method can improve recommendation accuracy while maintaining a lightweight design.

[0004] To achieve the above objectives, the present invention provides a method for recommending cross-domain knowledge transfer sequences based on interest guidance, comprising the following steps:

[0005] 1. An interest-guided cross-domain knowledge transfer sequence recommendation method, which includes the following modules and functional steps:

[0006] 1) The input user interaction sequences are masked and padded to construct user interaction sequences shared across specific and cross-domain domains. Then, the items in these interaction sequences are initialized and embedded, and location information is fused to generate the model's initial embedding sequence.

[0007] 2) Based on the user's initial embedding sequence in a specific domain obtained in step 1), design a layered hybrid expert structure for that domain: First, use a multi-head attention mechanism to capture the global dependencies between items. Then, effectively combine low-rank adaptation with hybrid experts to mine user preference information. Automatically select the most relevant experts through a gating network and enhance user preference information layer by layer to continuously refine the user behavior representation in the specific domain.

[0008] 3) Based on the user's initial embedding sequence in the shared domain obtained in step 1), design a shared domain multi-interest extraction module: adopt a soft clustering mechanism based on dynamic routing to learn the user's multiple, fine-grained interest representations at the cross-domain sharing level, and use them as interest guidance signals to feed back to the recommendation process, so as to help the model mine interest subspaces with transfer potential and realize targeted knowledge transfer.

[0009] 4) Based on the user's specific domain preference representation and cross-domain shared interest representation obtained in steps 2) and 3), design a cross-domain bridging migration module: by using interest information as a bridge, selectively fuse the two using a heterogeneous attention mechanism.

[0010] 5) The user preference representation obtained in step 4) is used as the final embedding representation, and the proposed model is optimized by backpropagation using the designed loss function.

[0011] 2. In step 1), the input data is first preprocessed by disabling interactive items that do not belong to the domain, and then using padding to complete the sequence, thereby obtaining the interactive sequences of the X and Y domains. and Subsequently, an embedding operation is performed on each sequence, calculated using the following formula:

[0012] ,

[0013] in, and These represent the item embedding matrices for the shared domain, X domain, and Y domain, respectively. and These represent the position embedding matrices of the shared domain, X domain, and Y domain, respectively.

[0014] 3. The specific implementation process in step 2) is as follows:

[0015] a. First, a multi-head attention mechanism is used to learn the global dependencies between items. The calculation formula is as follows:

[0016] ,

[0017] in, This indicates a dropout operation. This indicates a bullish self-attention strategy. Representation layer normalization operation. Similarly, embedding representation of domain Y and shared domain. and Encode the sequences to obtain their respective representations. and .

[0018] b. Secondly, hybrid expert features are incorporated into the low-rank adaptation. In each layer, a gating network is used to calculate the attention weight coefficients of each expert on the layer's input, using the following formula:

[0019] ,

[0020] in, A function representing a probability distribution. Indicates the first Attention weights of layers.

[0021] c. Subsequently, dynamic expert selection is achieved by weighting the obtained weight coefficients into a weighted combination. The calculation formula is as follows:

[0022] ,

[0023] in, Let l be the freeze transformation matrix of the l-th layer. Indicates the l-th layer The weighting coefficient of each expert Indicates the l-th layer Each projection matrix Indicates the l-th layer Each projection matrix.

[0024] 4. In step 3), the user's cross-domain behavior sequence is... By mapping the behavior to the candidate interest space through a linear transformation, we obtain the behavioral representation, namely: ,in It is a transformation matrix. Indicates the number of interest capsules.

[0025] a. Introduce routing weights to calculate the coupling coefficient, as shown below:

[0026] ,

[0027] in, This represents the coupling coefficient.

[0028] b. Calculate the aggregated representation of each interest capsule based on the coupling coefficient, as shown below:

[0029] .

[0030] c. Update the route weights, which are calculated as follows:

[0031] ,

[0032] .

[0033] 5. In step 4), to achieve personalized knowledge transfer, the model utilizes heterogeneous attention to combine user domain preferences with cross-domain interest representations. The calculation method is as follows:

[0034] a. Calculate the query vector, key vector, and value vector of the heterogeneous attention mechanism as follows:

[0035] ,

[0036] in, , and These represent the projection matrices of the query vector, key vector, and value vector, respectively.

[0037] b. The correlation between different positions in the input sequence is calculated using an attention mechanism, as follows:

[0038] ,

[0039] in, It is an attention function. Indicates the first One point of attention.

[0040] c. Aggregate all attention heads to obtain the final multi-head sub-attention representation, calculated as follows:

[0041] ,

[0042] in, This indicates a splicing operation. Indicates the first One point of attention, This represents the projection matrix.

[0043] 6. In step 5), the model is optimized using the InfoNCE loss function.

[0044] a. The similarity between positive and negative samples is calculated as follows:

[0045]

[0046] in, This indicates the final user embedding. Indicates positive sample embedding. This represents the temperature coefficient.

[0047] b. Combining the loss functions of the X and Y domains, the parameters are fitted based on the training data, calculated as follows:

[0048] .

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. This invention realizes personalized knowledge transfer in cross-domain sequence recommendation systems. While enhancing the model's ability to mine user preferences in complex domains, it introduces user interests as a guiding condition, realizing feature selection and personalized transfer of user preferences, and improving the accuracy of recommendation results.

[0051] 2. The method of this invention has a wide range of applications in mining fine-grained user representations and is highly practical, especially in the fields of personalized recommendation and cross-domain recommendation, where it has broad application prospects. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the logic flow of the present invention.

[0053] Figure 2 This is a framework diagram of the method of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0055] This implementation discloses an interest-guided cross-domain knowledge transfer sequence recommendation method, such as... Figure 1 As shown. The method is logically implemented as follows: Figure 2 As shown, this method first inputs user interaction data into an interest-guided cross-domain knowledge transfer sequence recommendation framework, obtaining an initial user embedding sequence through a shared item and location embedding layer. Then, a domain-specific layer-by-layer hybrid expert module is used to mine fine-grained user preferences within that domain. Considering the limitations of single-user interest modeling, a shared-domain multi-interest extraction module is introduced to capture users' multiple interest tendencies. Finally, through a cross-domain bridging transfer module, leveraging a heterogeneous attention mechanism, the fine-grained user preferences obtained from the domain-specific layer-by-layer hybrid expert module are fused with the multiple user interests obtained from the shared-domain multi-interest extraction module, thereby predicting the next potential recommendation item. Specifically, the method includes the following steps:

[0056] 1) In Figure 2 Given the original user interaction sequence, it is fed into the shared item and location embedding layer;

[0057] a. First, preprocess the input data by disabling interactive items that do not belong to the domain, and then use padding to complete the sequence, thus obtaining the interactive sequences for domains X and Y. and ;

[0058] b. Subsequently, an embedding operation is performed on each sequence, and the calculation formula is as follows:

[0059] ,

[0060] in, and These represent the item embedding matrices for the shared domain, X domain, and Y domain, respectively. and These represent the position embedding matrices of the shared domain, X domain, and Y domain, respectively.

[0061] 2) The specific implementation process for extracting user preferences from a specific domain is as follows:

[0062] a. First, a multi-head attention mechanism is used to learn the global dependencies between items. The calculation formula is as follows:

[0063] ,

[0064] in, This indicates a dropout operation. This indicates a bullish self-attention strategy. Representation layer normalization operation. Similarly, embedding representation of domain Y and shared domain. and Encode the sequences to obtain their respective representations. and .

[0065] b. Secondly, hybrid expert features are incorporated into the low-rank adaptation. In each layer, a gating network is used to calculate the attention weight coefficients of each expert on the layer's input, using the following formula:

[0066] ,

[0067] in, A function representing a probability distribution. Indicates the first Attention weights of layers.

[0068] c. Subsequently, dynamic expert selection is achieved by weighting the obtained weight coefficients into a weighted combination. The calculation formula is as follows:

[0069] ,

[0070] in, Let l be the freeze transformation matrix of the l-th layer. Indicates the l-th layer The weighting coefficient of each expert Indicates the l-th layer Each projection matrix Indicates the l-th layer Each projection matrix.

[0071] 3) The specific implementation process for extracting multiple user interests from a shared domain is as follows:

[0072] a. Sequence of user cross-domain behaviors By mapping the behavior to the candidate interest space through a linear transformation, we obtain the behavioral representation, namely: ,in , This represents the number of interest capsules. Routing weights are introduced to calculate the coupling coefficient, as shown below: ,

[0073] in, Represents the coupling coefficient. Indicates the route weight;

[0074] b. Calculate the aggregated representation of each interest capsule based on the coupling coefficient, as shown below:

[0075] ,

[0076] in, Aggregation represents the result. Capsules indicating interest;

[0077] c. Update the route weights, which are calculated as follows:

[0078] ,

[0079] .

[0080] 4) The specific implementation process of integrating user preferences from a specific domain with user interests from a shared domain is as follows:

[0081] a. Calculate the query vector, key vector, and value vector of the heterogeneous attention mechanism as follows:

[0082] ,

[0083] in, , and These represent the projection matrices of the query vector, key vector, and value vector, respectively.

[0084] b. The correlation between different positions in the input sequence is calculated using an attention mechanism, as follows:

[0085] ,

[0086] in, It is an attention function. Indicates the first One point of attention.

[0087] c. Aggregate all attention heads to obtain the final multi-head sub-attention representation, calculated as follows:

[0088] ,

[0089] in, This indicates a splicing operation. This represents the i-th attention head. This represents the projection matrix.

[0090] 5) The specific implementation process of the training phase is as follows:

[0091] a. The similarity between positive and negative samples is calculated as follows:

[0092] ,

[0093] in, Represents the similarity function. This indicates the final user embedding. Indicates positive sample embedding. This represents the temperature coefficient.

[0094] b. Combining the loss functions of the X and Y domains, the parameters are fitted based on the training data, calculated as follows:

[0095] .

[0096] Experiments were conducted on three subsets of the Amazon dataset, encompassing six different domains: Food-Kitchen (FK), Beauty-Electronics (BE), and Movie-Book (MB). The technical results are shown in Tables 1, 2, and 3.

[0097] Table 1 shows the experimental results of the proposed IGKM method and existing methods on the Food-Kitchen (FK) dataset subset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0098]

[0099] Table 2 shows the experimental results of the proposed IGKM method and existing methods on a subset of the Beauty-Electronics (BE) dataset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0100]

[0101] Table 3 shows the experimental results of the proposed IGKM method and existing methods on a subset of the Movie-Book (MB) dataset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0102]

[0103] The proposed IGKM method, through a layer-by-layer dynamic ensemble mechanism, can extract users' stable and specific invariant interests across various domains more deeply, thus alleviating the problem of insufficient user interest modeling in traditional models. It significantly outperforms existing models on multiple metrics. Furthermore, cross-domain shared interests are introduced as a guiding signal for recommendations, acting as a noise filter to some extent and effectively improving the accuracy of the recommendation results.

[0104] In summary, by adopting the above scheme, this invention provides a new solution to the problem of low recommendation accuracy in existing cross-domain recommendation systems due to users' singular interests and difficulty in effectively distinguishing user preferences between specific and shared domains. Through the collaborative processing of a specific domain-level hybrid expert module, a shared domain multi-interest extraction module, and a cross-domain bridging and transfer module, it can not only fully explore users' fine-grained preferences in specific domains but also capture users' multiple interests in cross-domain environments. Furthermore, it achieves effective fusion of these two aspects through a heterogeneous attention mechanism, thereby significantly improving the accuracy of recommendation results. This invention has good scalability and application prospects, possesses practical promotional value, and is worthy of widespread adoption.

Claims

1.A method for recommending a sequence of interest-driven cross-domain knowledge transfer, characterized in that, The method comprises the following modules and functional steps: 1) shielding and filling operations are performed on the input user interaction sequence to construct the interaction sequence shared by the user in a specific domain and across domains, and then the items in the interaction sequence are initialized and embedded to generate an initialized embedding sequence of the model by fusing position information; 2) according to the initialized embedding sequence of the user in the specific domain obtained in step 1), a specific domain layer-by-layer mixed expert structure is designed: first, the global dependency between items is captured by using a multi-head attention mechanism, then the user preference information is mined by effectively combining low-rank adaptation and mixed experts, the most relevant experts are automatically selected by a gating network, and the user preference information is enhanced layer by layer to continuously refine the user behavior representation in the specific domain; 3) according to the initialized embedding sequence of the user in the shared domain obtained in step 1), a shared domain multi-interest extraction module is designed: a dynamic routing based soft clustering mechanism is used to learn the multi, fine-grained interest representation of the user at the cross-domain sharing level, and the interest representation is fed back to the recommendation process as an interest guide signal to help the model mine interest subspaces with migration potential and realize targeted knowledge migration; 4) according to the preference representation of the user in the specific domain and the interest representation shared across domains obtained in steps 2) and 3), a cross-domain bridging migration module is designed: the heterogeneous attention mechanism is used to selectively fuse the two by using the interest information as a bridge; 5) the user preference representation obtained in step 4) is used as the final embedding representation, and the designed loss function is used for back propagation optimization of the proposed model. 2.The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 1), the input data is preprocessed first, and the interaction items not belonging to the domain are shielded respectively, and padding is used for sequence padding, so as to obtain the interaction sequences of the X domain and the Y domain and ; then, embedding operation is performed on each sequence, and the calculation formula is as follows: ； wherein, and denote the item embedding matrices for the shared domain, X domain and Y domain, respectively, and denote the location embedding matrices for the shared domain, X domain and Y domain, respectively. 3.The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 2), a, first, the multi-head attention mechanism is used to learn the global dependency between items, and the calculation formula is as follows: ； wherein, denotes a dropout operation, denotes a multi-head self-attention operation, denotes a layer normalization operation, and similarly for the embeddings of the domains Y and the shared domain and are encoded, resulting in sequence representations and ; b, second, the characteristics of mixed experts are integrated on the basis of low-rank adaptation, and in each layer, the gating network is used to calculate the attention weight coefficient of each expert for the input of the layer, and the calculation formula is as follows: ； wherein, a function representing a probability distribution, represents the attention weight of the layer; c, then, the dynamic selection of experts is realized by weighting combination according to the obtained weight coefficient, and the calculation formula is as follows: ； wherein, is a frozen transformation matrix of the lth layer, denotes a weight coefficient of the lth layer and the th expert, denotes an upper projection matrix of the lth layer and the th expert, denotes a lower projection matrix of the lth layer and the th expert. 4.The interest-guided cross-domain knowledge transfer sequence recommendation method of claim 1, wherein: In step 3), the user cross-domain behavior sequence is mapped to the candidate interest space by a linear transformation, obtaining a behavior representation, i.e.: wherein is a transformation matrix, represents the number of interest capsules; a, the routing weight is introduced to calculate the coupling coefficient, and the calculation method is as follows: ； wherein, represents a coupling coefficient, represents a routing weight; b, the aggregation representation of each interest capsule is calculated according to the coupling coefficient, and the calculation method is as follows: ； wherein, represents the result of the polymerization, represents an interesting capsule; c, update the routing weight, and the calculation method is as follows: ； ； wherein, represents the probability that an entity exists. 5.The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 4), to realize personalized knowledge migration, the model combines the user domain preference and cross-domain interest representation by using heterogeneous attention, and the calculation method is as follows: a, calculate the query vector, key vector and value vector of the heterogeneous attention mechanism, as follows: ； wherein, , and represent projection matrices of the query vector, the key vector and the value vector, respectively, and represent the enhanced user preference and the condensed multi-interest, respectively. b, the relevance of different positions in the input sequence is calculated by the attention mechanism, as follows: ； wherein, is an attention function, denotes the i-th attention head; c, all attention heads are aggregated to obtain the final multi-head sub-attention representation, as follows: ； wherein, denotes a concatenation operation, denotes the i-th attention head, denotes a projection matrix; similarly, can be computed . 6.The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 5), the model is optimized using the InfoNCE loss function; a, the similarity of positive and negative samples is calculated, and the calculation method is as follows: ； wherein, represents a similarity function, represents a final user embedding, represents a positive sample embedding, represents a temperature coefficient; b, the loss functions of X domain and Y domain are combined, and the parameters are fitted according to the training data, and the calculation method is as follows: 。

Images