Cross-domain knowledge migration sequence recommendation method based on interest guidance

Abstract

The invention discloses a cross-domain knowledge migration sequence recommendation method based on interest guidance, and the method comprises the steps: 1) shielding and supplementing a user interaction sequence, forming two types of sequences of a specific domain and cross-domain sharing, and carrying out the embedding initialization of sequence items; 2) designing a layer-by-layer dynamic integration mechanism based on a large language model, strengthening preference characteristics of a user in a specific domain layer by layer, and effectively suppressing interference information; 3) constructing a cross-domain shared interest extraction module, and extracting fine-grained user interest representation from the cross-domain shared sequence; 4) fusing cross-domain shared user interests with behavior information of a specific domain by using a heterogeneous attention mechanism; 6, end-to-end training optimization is carried out, and the calculation burden of the model training stage is reduced. According to the method, the problems that in cross-domain recommendation, user preferences of cross-domain sharing and specific domains are not clearly distinguished, and user interest mining is insufficient are effectively solved, and the recommendation precision and the personalized effect are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of cross-domain recommendation technology, specifically relating to an interest-guided cross-domain knowledge transfer sequence recommendation method, and particularly to modeling and learning user preferences of specific domains and user interests of shared domains in cross-domain scenarios. Background Technology

[0002] In today's information age, with the rapid development of the internet, massive amounts of data are constantly emerging, while users actually only encounter and interact with a tiny fraction of this total. Therefore, data sparsity remains a major challenge for recommender systems. Cross-domain sequence recommendation, as an effective solution, gradually alleviates this problem by transferring knowledge between different domains. However, a common learning strategy in this process is to integrate user sequences from various specific domains into cross-domain shared sequences, using these sequences as bridges to facilitate knowledge transfer between different domains. But this method indiscriminately introduces cross-domain information into the modeling of specific domains during training, inevitably including irrelevant noise features, thus interfering with the accurate modeling of target user preferences. Furthermore, traditional methods typically model user interests as a single global representation, ignoring the diversity of user interests in cross-domain scenarios. Therefore, there is an urgent need for a new cross-domain recommendation method designed to overcome these limitations and address rapidly changing user needs and complex cross-domain behavioral patterns. Therefore, this invention presents an interest-guided cross-domain knowledge transfer sequence recommendation method that can effectively identify and handle differences in user interests across different domains, reduce noise interference, and thus achieve more accurate and personalized recommendations. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and provide an interest-guided cross-domain knowledge transfer sequence recommendation method. This method breaks through the traditional cross-domain sequence recommendation representation learning method and can effectively handle the problem of inaccurate recommendations caused by the failure to clearly distinguish between cross-domain shared and specific domain user preferences and the modeling of user interests in a single way. By integrating specific domain user preferences layer by layer, extracting cross-domain shared interests, and performing cross-domain knowledge transfer fusion, the method can improve recommendation accuracy while maintaining a lightweight design.

[0004] To achieve the above objectives, the present invention provides a method for recommending cross-domain knowledge transfer sequences based on interest guidance, comprising the following steps:

[0005] 1. An interest-guided cross-domain knowledge transfer sequence recommendation method, which includes the following modules and functional steps:

[0006] 1) The input user interaction sequences are masked and padded to construct user interaction sequences shared across specific and cross-domain domains. Then, the items in these interaction sequences are initialized and embedded, and location information is fused to generate the model's initial embedding sequence.

[0007] 2) Based on the user's initial embedding sequence in a specific domain obtained in step 1), design a layered hybrid expert structure for that domain: First, use a multi-head attention mechanism to capture the global dependencies between items. Then, effectively combine low-rank adaptation with hybrid experts to mine user preference information. Automatically select the most relevant experts through a gating network and enhance user preference information layer by layer to continuously refine the user behavior representation in the specific domain.

[0008] 3) Based on the user's initial embedding sequence in the shared domain obtained in step 1), design a shared domain multi-interest extraction module: adopt a soft clustering mechanism based on dynamic routing to learn the user's multiple, fine-grained interest representations at the cross-domain sharing level, and use them as interest guidance signals to feed back to the recommendation process, so as to help the model mine interest subspaces with transfer potential and realize targeted knowledge transfer.

[0009] 4) Based on the user's specific domain preference representation and cross-domain shared interest representation obtained in steps 2) and 3), design a cross-domain bridging migration module: by using interest information as a bridge, selectively fuse the two using a heterogeneous attention mechanism.

[0010] 5) The user preference representation obtained in step 4) is used as the final embedding representation, and the proposed model is optimized by backpropagation using the designed loss function.

[0011] 2. In step 1), the input data is first preprocessed by disabling interactive items that do not belong to the domain, and then using padding to complete the sequence, thereby obtaining the interactive sequences of the X and Y domains. and Subsequently, an embedding operation is performed on each sequence, calculated using the following formula:

[0012] ,

[0013] in, and These represent the item embedding matrices for the shared domain, X domain, and Y domain, respectively. and These represent the position embedding matrices of the shared domain, X domain, and Y domain, respectively.

[0014] 3. The specific implementation process in step 2) is as follows:

[0015] a. First, a multi-head attention mechanism is used to learn the global dependencies between items. The calculation formula is as follows:

[0016] ,

[0017] in, This indicates a dropout operation. This indicates a bullish self-attention strategy. Representation layer normalization operation. Similarly, embedding representation of domain Y and shared domain. and Encode the sequences to obtain their respective representations. and .

[0018] b. Secondly, hybrid expert features are incorporated into the low-rank adaptation. In each layer, a gating network is used to calculate the attention weight coefficients of each expert on the layer's input, using the following formula:

[0019] ,

[0020] in, A function representing a probability distribution. Indicates the first Attention weights of layers.

[0021] c. Subsequently, dynamic expert selection is achieved by weighting the obtained weight coefficients into a weighted combination. The calculation formula is as follows:

[0022] ,

[0023] in, Let l be the freeze transformation matrix of the l-th layer. Indicates the l-th layer The weighting coefficient of each expert Indicates the l-th layer Each projection matrix Indicates the l-th layer Each projection matrix.

[0024] 4. In step 3), the user's cross-domain behavior sequence is... By mapping the behavior to the candidate interest space through a linear transformation, we obtain the behavioral representation, namely: ,in It is a transformation matrix. Indicates the number of interest capsules.

[0025] a. Introduce routing weights to calculate the coupling coefficient, as shown below:

[0026] ,

[0027] in, This represents the coupling coefficient.

[0028] b. Calculate the aggregated representation of each interest capsule based on the coupling coefficient, as shown below:

[0029] .

[0030] c. Update the route weights, which are calculated as follows:

[0031] ,

[0032] .

[0033] 5. In step 4), to achieve personalized knowledge transfer, the model utilizes heterogeneous attention to combine user domain preferences with cross-domain interest representations. The calculation method is as follows:

[0034] a. Calculate the query vector, key vector, and value vector of the heterogeneous attention mechanism as follows:

[0035] ,

[0036] in, , and These represent the projection matrices of the query vector, key vector, and value vector, respectively.

[0037] b. The correlation between different positions in the input sequence is calculated using an attention mechanism, as follows:

[0038] ,

[0039] in, It is an attention function. Indicates the first One point of attention.

[0040] c. Aggregate all attention heads to obtain the final multi-head sub-attention representation, calculated as follows:

[0041] ,

[0042] in, This indicates a splicing operation. Indicates the first One's attention, This represents the projection matrix.

[0043] 6. In step 5), the model is optimized using the InfoNCE loss function.

[0044] a. The similarity between positive and negative samples is calculated as follows:

[0045]

[0046] in, Indicates the final user embedding, Indicates positive sample embedding. This represents the temperature coefficient.

[0047] b. Combining the loss functions of the X and Y domains, the parameters are fitted based on the training data, calculated as follows:

[0048] .

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] 1. This invention realizes personalized knowledge transfer in cross-domain sequence recommendation systems. While enhancing the model's ability to mine user preferences in complex domains, it introduces user interests as a guiding condition, realizing feature selection and personalized transfer of user preferences, and improving the accuracy of recommendation results.

[0051] 2. The method of this invention has a wide range of applications in mining fine-grained user representations and is highly practical, especially in the fields of personalized recommendation and cross-domain recommendation, where it has broad application prospects. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the logic flow of the present invention.

[0053] Figure 2 This is a framework diagram of the method of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0055] This implementation discloses an interest-guided cross-domain knowledge transfer sequence recommendation method, such as... Figure 1 As shown. The method is logically implemented as follows: Figure 2 As shown, this method first inputs user interaction data into an interest-guided cross-domain knowledge transfer sequence recommendation framework, obtaining an initial user embedding sequence through a shared item and location embedding layer. Then, a domain-specific layer-by-layer hybrid expert module is used to mine fine-grained user preferences within that domain. Considering the limitations of single-user interest modeling, a shared-domain multi-interest extraction module is introduced to capture users' multiple interest tendencies. Finally, through a cross-domain bridging transfer module, leveraging a heterogeneous attention mechanism, the fine-grained user preferences obtained from the domain-specific layer-by-layer hybrid expert module are fused with the multiple user interests obtained from the shared-domain multi-interest extraction module, thereby predicting the next potential recommendation item. Specifically, the method includes the following steps:

[0056] 1) In Figure 2 Given the original user interaction sequence, it is fed into the shared item and location embedding layer;

[0057] a. First, preprocess the input data by disabling interactive items that do not belong to the domain, and then use padding to complete the sequence, thus obtaining the interactive sequences for domains X and Y. and ;

[0058] b. Subsequently, an embedding operation is performed on each sequence, and the calculation formula is as follows:

[0059] ,

[0060] in, and These represent the item embedding matrices for the shared domain, X domain, and Y domain, respectively. and These represent the position embedding matrices of the shared domain, X domain, and Y domain, respectively.

[0061] 2) The specific implementation process for extracting user preferences from a specific domain is as follows:

[0062] a. First, a multi-head attention mechanism is used to learn the global dependencies between items. The calculation formula is as follows:

[0063] ,

[0064] in, This indicates a dropout operation. This indicates a bullish self-attention strategy. Representation layer normalization operation. Similarly, embedding representation of domain Y and shared domain. and Encode the sequences to obtain their respective representations. and .

[0065] b. Secondly, hybrid expert features are incorporated into the low-rank adaptation. In each layer, a gating network is used to calculate the attention weight coefficients of each expert on the layer's input, using the following formula:

[0066] ,

[0067] in, A function representing a probability distribution. Indicates the first Attention weights of layers.

[0068] c. Subsequently, dynamic expert selection is achieved by weighting the obtained weight coefficients into a weighted combination. The calculation formula is as follows:

[0069] ,

[0070] in, Let l be the freeze transformation matrix of the l-th layer. Indicates the l-th layer The weighting coefficient of each expert Indicates the l-th layer Each projection matrix Indicates the l-th layer Each projection matrix.

[0071] 3) The specific implementation process for extracting multiple user interests from a shared domain is as follows:

[0072] a. Sequence of user cross-domain behaviors By mapping the behavior to the candidate interest space through a linear transformation, we obtain the behavioral representation, namely: ,in , This represents the number of interest capsules. Routing weights are introduced to calculate the coupling coefficient, as shown below: ,

[0073] in, Represents the coupling coefficient. Indicates the route weight;

[0074] b. Calculate the aggregated representation of each interest capsule based on the coupling coefficient, as shown below:

[0075] ,

[0076] in, Aggregation represents the result. Capsules indicating interest;

[0077] c. Update the route weights, which are calculated as follows:

[0078] ,

[0079] .

[0080] 4) The specific implementation process of integrating user preferences from a specific domain with user interests from a shared domain is as follows:

[0081] a. Calculate the query vector, key vector, and value vector of the heterogeneous attention mechanism as follows:

[0082] ,

[0083] in, , and These represent the projection matrices of the query vector, key vector, and value vector, respectively.

[0084] b. The correlation between different positions in the input sequence is calculated using an attention mechanism, as follows:

[0085] ,

[0086] in, It is an attention function. Indicates the first One point of attention.

[0087] c. Aggregate all attention heads to obtain the final multi-head sub-attention representation, calculated as follows:

[0088] ,

[0089] in, This indicates a splicing operation. This represents the i-th attention head. This represents the projection matrix.

[0090] 5) The specific implementation process of the training phase is as follows:

[0091] a. The similarity between positive and negative samples is calculated as follows:

[0092] ,

[0093] in, Represents the similarity function. Indicates the final user embedding, Indicates positive sample embedding. This represents the temperature coefficient.

[0094] b. Combining the loss functions of the X and Y domains, the parameters are fitted based on the training data, calculated as follows:

[0095] .

[0096] Experiments were conducted on three subsets of the Amazon dataset, encompassing six different domains: Food-Kitchen (FK), Beauty-Electronics (BE), and Movie-Book (MB). The technical results are shown in Tables 1, 2, and 3.

[0097] Table 1 shows the experimental results of the proposed IGKM method and existing methods on the Food-Kitchen (FK) dataset subset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0098]

[0099] Table 2 shows the experimental results of the proposed IGKM method and existing methods on a subset of the Beauty-Electronics (BE) dataset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0100]

[0101] Table 3 shows the experimental results of the proposed IGKM method and existing methods on a subset of the Movie-Book (MB) dataset, where the best and second-best experimental results are indicated by underline and bold, respectively.

[0102]

[0103] The proposed IGKM method, through a layer-by-layer dynamic ensemble mechanism, can extract users' stable and specific invariant interests across various domains more deeply, thus alleviating the problem of insufficient user interest modeling in traditional models. It significantly outperforms existing models on multiple metrics. Furthermore, cross-domain shared interests are introduced as a guiding signal for recommendations, acting as a noise filter to some extent and effectively improving the accuracy of the recommendation results.

[0104] In summary, by adopting the above scheme, this invention provides a new solution to the problem of low recommendation accuracy in existing cross-domain recommendation systems due to users' singular interests and difficulty in effectively distinguishing user preferences between specific and shared domains. Through the collaborative processing of a specific domain-level hybrid expert module, a shared domain multi-interest extraction module, and a cross-domain bridging and transfer module, it can not only fully explore users' fine-grained preferences in specific domains but also capture users' multiple interests in cross-domain environments. Furthermore, it achieves effective fusion of these two aspects through a heterogeneous attention mechanism, thereby significantly improving the accuracy of recommendation results. This invention has good scalability and application prospects, possesses practical promotional value, and is worthy of widespread adoption.

Claims

1. A method for recommending cross-domain knowledge transfer sequences based on interest guidance, characterized in that, This method includes the following modules and functional steps: 1) The input user interaction sequence is masked and completed to construct the user's interaction sequence shared in a specific domain and across domains. Then, the items in these interaction sequences are initialized and embedded, and the location information is fused to generate the model's initial embedding sequence. 2) Based on the user's initial embedding sequence in a specific domain obtained in step 1), design a layered hybrid expert structure for the specific domain: First, use a multi-head attention mechanism to capture the global dependencies between items, then effectively combine low-rank adaptation with hybrid experts to mine user preference information, automatically select the most relevant experts through a gating network, and enhance user preference information layer by layer to continuously refine the user behavior representation in the specific domain. 3) Based on the user's initial embedding sequence in the shared domain obtained in step 1), design a shared domain multi-interest extraction module: adopt a soft clustering mechanism based on dynamic routing to learn the user's multiple, fine-grained interest representations at the cross-domain sharing level, and use them as interest guidance signals to feed back to the recommendation process, so as to help the model mine interest subspaces with transfer potential and realize targeted knowledge transfer. 4) Based on the user's specific domain preference representation and cross-domain shared interest representation obtained in steps 2) and 3), design a cross-domain bridging migration module: by using interest information as a bridge, a heterogeneous attention mechanism is used to selectively fuse the two; 5) The user preference representation obtained in step 4) is used as the final embedding representation, and the proposed model is optimized by backpropagation using the designed loss function.

2. The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 1), the input data is first preprocessed by masking interactive items that do not belong to the domain and using padding to complete the sequence, thereby obtaining the interactive sequences of the X and Y domains. and Subsequently, an embedding operation is performed on each sequence, calculated using the following formula: ； in, and These represent the item embedding matrices for the shared domain, the X domain, and the Y domain, respectively. and These represent the position embedding matrices of the shared domain, X domain, and Y domain, respectively.

3. The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 2), a. First, a multi-head attention mechanism is used to learn the global dependencies between items, and the calculation formula is as follows: ； in, This indicates a dropout operation. This indicates a bullish self-attention strategy. The representation layer normalization operation, similarly, applies to the embedding representation of the domain Y and the shared domain. and Encode the sequences to obtain their respective representations. and ; b. Secondly, based on low-rank adaptation, the characteristics of hybrid experts are incorporated, and in each layer, a gating network is used to calculate the attention weight coefficient of each expert for the input of that layer. The calculation formula is as follows: ； in, A function representing a probability distribution. Indicates the first Attention weights of layers; c. Subsequently, dynamic expert selection is achieved by weighting the obtained weight coefficients into a weighted combination. The calculation formula is as follows: ； in, Let l be the freeze transformation matrix of the l-th layer. Indicates the l-th layer The weighting coefficient of each expert Indicates the l-th layer Each projection matrix Indicates the l-th layer Each projection matrix.

4. The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 3), the user's cross-domain behavior sequence is... By mapping the behavior to the candidate interest space through a linear transformation, we obtain the behavioral representation, namely: ,in It is a transformation matrix. Indicates the number of interest capsules; a. Introduce routing weights to calculate the coupling coefficient, as shown below: ； in, Represents the coupling coefficient. Indicates the route weight; b. Calculate the aggregated representation of each interest capsule based on the coupling coefficient, as shown below: ； in, Aggregation represents the result. Capsules indicating interest; c. Update the route weights, which are calculated as follows: ； ； in, It represents the probability of an entity existing.

5. The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 4), to achieve personalized knowledge transfer, the model utilizes heterogeneous attention to combine user domain preferences with cross-domain interest representations, and its calculation method is shown below: a. Calculate the query vector, key vector, and value vector of the heterogeneous attention mechanism as follows: ； in, , and These represent the projection matrices of the query vector, key vector, and value vector, respectively. and These represent enhanced user preferences and refined multi-interests, respectively. b. The correlation between different positions in the input sequence is calculated using an attention mechanism, as follows: ； in, It is an attention function. Indicates the first One point of attention; c. Aggregate all attention heads to obtain the final multi-head sub-attention representation, calculated as follows: ； in, This indicates a splicing operation. This represents the i-th attention head. This represents the projection matrix; similarly, it can be calculated. .

6. The interest-guided cross-domain knowledge transfer sequence recommendation method according to claim 1, characterized in that: In step 5), the model is optimized using the InfoNCE loss function; a. The similarity between positive and negative samples is calculated as follows: ； in, Represents the similarity function. This indicates the final user embedding. Indicates positive sample embedding. Indicates the temperature coefficient; b. Combining the loss functions of the X and Y domains, the parameters are fitted based on the training data, calculated as follows: 。

Images