Multi-domain recommendation method and system based on inter-domain knowledge transfer and cross-attention

Abstract

The present invention proposes a multi-domain recommendation method and system based on inter-domain knowledge transfer and cross-attention. First, feature learning is performed on the user-item embeddings of each domain. By mining high-order similarity relationships, users with similar behaviors in multiple domains are distinguished to overcome the strong dependence on overlapping users. At the same time, a contrastive learning mechanism is introduced to align cross-domain behavioral features and effectively realize information transfer. Secondly, a cross-attention mechanism is formulated to enable the target domain to dynamically aggregate key information from other domains, generate domain-specific representations, adapt to complex data distribution and balance feature differences. Finally, the embeddings of each domain after feature learning are fused to obtain a global general domain embedding, and adaptive mapping is achieved through a multi-layer perceptron. The final prediction representation with both global commonality and domain characteristics is fused with the domain-specific representation to adapt to scenarios with dynamic changes in user preferences and improve the accuracy of recommendations.

Description

Technical Field

[0001] The present invention relates to the field of recommendation technology, and in particular to a multi-domain recommendation method and system based on inter-domain knowledge transfer and cross-attention. Background Art

[0002] With the rapid development of digital platforms and the increasing diversification of user needs, modern recommendation systems face unprecedented challenges. Users exhibit both interconnected and distinct behavioral patterns across different domains. Traditional single-domain recommendation (SDR) can no longer meet the personalized needs of users across multiple domains. Multi-domain recommendation (MDR), a key technology to alleviate data sparsity, improves recommendation effectiveness by fully exploring and leveraging cross-domain user behavior correlations. This technology holds significant theoretical and practical significance.

[0003] Multi-Domain Recommendation (MDR) aims to alleviate the data sparsity problem in single-Domain Recommendation (SDR) by leveraging rich user-item interaction data across multiple domains. Existing MDR models struggle to effectively transfer information when overlapping users are scarce. Most ignore the complex data distribution and feature differences between domains. While some can exploit node structure information, they fail to address the dynamic changes in user preferences across scenarios.

[0004] Recent research has applied techniques such as disentangled representations, attention mechanisms, and global feature fusion to MDR. Existing MDR models rely on explicit connections, failing to adequately mine implicit behavioral patterns in scenarios with sparse overlapping users. Others fail to effectively model asymmetric dependencies between domains, making it difficult to accurately capture the dynamic relationships between users in different domains. Still others lack the ability to adaptively adjust to domain characteristics, making it difficult to balance domain commonality with specificity. Optimizing these three areas can improve recommendation accuracy. Summary of the Invention

[0005] In view of the above situation, the main purpose of the present invention is to propose a multi-domain recommendation method and system based on inter-domain knowledge transfer and cross-attention to solve the above technical problems.

[0006] The present invention proposes a multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention, which includes the following steps:

[0007] Step 1: Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field;

[0008] Step 2: Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain.

[0009] Step 3: Design a cross-attention mechanism for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain.

[0010] Step 4: Fusion the overall embeddings of each domain into a global general domain embedding, then adjust the global general domain embedding into an adaptive representation for a specific domain, concatenate the adaptive representation for the specific domain with the specific representation of each domain, and make predictions for each domain to obtain the user-item prediction interaction score for each domain.

[0011] The present invention also proposes a multi-domain recommendation system based on inter-domain knowledge transfer and cross-attention, wherein the system applies the multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention as described above, and the system includes:

[0012] Inter-domain knowledge transfer module, used to:

[0013] Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field;

[0014] Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain.

[0015] Cross-Attention Module, used to:

[0016] A cross-attention mechanism is designed for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain.

[0017] Global feature fusion module, used for:

[0018] The overall embeddings of each domain are fused into a global general domain embedding, which is then adjusted into an adaptive representation for a specific domain. The adaptive representation for a specific domain is then concatenated with the specific representation of each domain to make predictions for each domain, thereby obtaining the user-item prediction interaction scores for each domain.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. Utilize a graph convolutional network (GCN) to learn features from user-item embeddings in various domains, obtaining structured representations within the domain. Furthermore, a high-order similarity matrix is ​​constructed using a random walk strategy to identify users with similar behaviors across multiple domains. A contrastive learning mechanism is then employed to align cross-domain behavioral features, effectively enabling information transfer when overlapping users are scarce.

[0021] 2. Design a cross-attention mechanism for each domain, enabling the target domain to dynamically aggregate key information from other domains. By adaptively adjusting the weight of each domain's influence on the target domain, the model can capture the complex relationships between domains, effectively extract cross-domain complementary information, and generate more expressive domain-specific representations that adapt to complex data distributions and balance feature differences.

[0022] 3. After integrating feature learning, the embeddings of various fields are used to obtain a global general field embedding. Through MLP, the global general field embedding is adjusted to an adaptive representation for a specific field, which is then spliced ​​with the specific representation of each field to make predictions for each field, realizing the fusion of global features and field-specific features to adapt to scenarios where user preferences change dynamically.

[0023] 4. The present invention first performs feature learning on user-item embeddings in various fields, and distinguishes users with similar behaviors in multiple fields by mining high-order similarity relationships to overcome the strong dependence on overlapping users. At the same time, a contrastive learning mechanism is introduced to align cross-domain behavioral features and effectively realize information transfer; secondly, a cross-attention mechanism is formulated to enable the target field to dynamically aggregate key information from other fields, generate domain-specific representations, adapt to complex data distribution and balance feature differences; finally, the embeddings of various fields after feature learning are fused to obtain a global general domain embedding, and adaptive mapping is achieved through a multi-layer perceptron. The final prediction representation with both global commonality and domain characteristics is fused with the domain-specific representation to adapt to scenarios where user preferences change dynamically and improve the accuracy of recommendations.

[0024] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Flowchart of the multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention proposed in this invention;

[0026] Figure 2 This is a schematic diagram of the structure of the multi-domain recommendation system based on inter-domain knowledge transfer and cross-attention proposed in this invention. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0028] These and other aspects of the embodiments of the present invention will become clear with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to illustrate some ways of implementing the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0029] See also Figure 1 This embodiment provides a multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention, the method comprising the following steps:

[0030] Step 1: Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field;

[0031] Traditional independent modeling methods struggle to capture the underlying connections between domains, while directly sharing features can lead to information confusion, hindering personalized recommendation effectiveness. To address this issue, this paper employs GCN to construct an independent user / item interaction graph for each domain and learn high-order features through neighborhood aggregation. Compared to traditional shallow embedding methods, GCN can leverage high-order connections between users and items to effectively extract structured information within the domain, providing a more robust embedding for subsequent predictions.

[0032] In step 1, during the propagation process, learning domain features through neighborhood aggregation to obtain the overall embedding of each domain specifically includes the following steps:

[0033] In this step, due to the inherent differences in interactions between different domains, the Xavier method is used to independently randomly initialize the user-item interaction graph in each domain to obtain user embedding. The corresponding process has the following relationship:

[0034] ;

[0035] in, represents uniform distribution, and Respectively represent the number of nodes corresponding to input and output, Represents a user Initial embedding of

[0036] The Xavier method is used to independently randomly initialize the user-item interaction graph in each domain to obtain item embeddings;

[0037] User embeddings and item embeddings are mapped to a low-dimensional vector space. In the low-dimensional vector space, the GCN-based neighborhood aggregation strategy is used to perform multi-layer iterative updates on the user and item embeddings of each field to obtain the user and item embeddings after feature learning in each field. In the multi-layer iterative update process of the user and item embeddings of each field, the following relationship exists in each iterative update process:

[0038] ;

[0039] in, represents the hyperparameter that balances own information and neighbor information, ; and Represents users and projects The set of neighbor nodes of represents the normalization term used to resize the embeddings during the neighbor aggregation process, Respectively represent the current field Layer embedding of users and items;

[0040] Integrate the user embedding after feature learning and the item embedding after feature learning to obtain the overall embedding of the field. The corresponding process has the following relationship:

[0041] ;

[0042] in, Representation field The overall embedding of represents the concatenation of vectors, Respectively represent fields After feature learning, users and projects Embedded, Respectively represent fields User set after feature learning and project sets Embedded.

[0043] Step 2: Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain.

[0044] To achieve effective cross-domain knowledge transfer, the key is to identify related users between different fields. Traditional methods rely on overlapping users as a bridge for inter-domain knowledge transfer, which requires users to interact in multiple fields. However, as the number of fields increases, there are fewer and fewer users who have explicit interactions in all fields. Relying solely on these overlapping users cannot achieve effective cross-domain knowledge transfer. This invention is based on the embedding of domain features learned , we construct a user similarity matrix for each field and use a random walk strategy to mine high-order similarity relationships to capture implicit connections between users in the field. By comparing the high-order similarity matrices of each field, we can identify similar users with consistent behavior patterns, even if they lack direct interaction.

[0045] For each domain, since user behavior has domain characteristics, it is necessary to first reflect the direct similarity between users in a single domain. In this step, domain For example, users in this field With users The embedding vectors are First, we use the Euclidean distance to measure the difference between any two user embedding vectors, and use the negative exponential function to convert the distance into a similarity score. We then arrange the similarity scores between all two users into a matrix according to the user index, thus obtaining the user similarity matrix. The corresponding process has the following relationship:

[0046] ;

[0047] in, represents the user similarity matrix, Represents a user With users The similarity score of express The general elements of Represents users With users The embedding vector of , represents the temperature parameter used to adjust the smoothness of the user similarity distribution within the domain, ; represents the L2 norm;

[0048] However, there may be implicit indirect connections between users, and this high-order similarity is only known through the similarity matrix To extend local similarity to the global level, a random walk strategy is used to obtain a high-order similarity matrix, ultimately distinguishing users with similar behaviors across all domains. Compared to other high-order relationship modeling methods, random walks have the advantages of high computational efficiency and strong interpretability, making them particularly suitable for processing multi-domain sparse data. The details are as follows:

[0049] Similarity Matrix Perform row normalization to obtain the transition probability, and construct the transition probability matrix according to the user index arrangement. The corresponding process has the following relationship:

[0050] ;

[0051] in, represents the transition probability matrix, express No. Rank Column elements, represents the summation indicator variable, express general elements of;

[0052] Set the number of walking steps to , calculated by continuous multiplication The transfer probability matrix after the step; if the number of steps is too small, it cannot capture enough indirect connections, while if the number of steps is too large, it may introduce noise. Based on the experimental performance of real data sets, the appropriate number of steps can better balance local features and global associations. In this embodiment, the number of steps is set to If it is set to 3, the corresponding process has the following relationship:

[0053] ;

[0054] in, express The transfer probability matrix after steps;

[0055] In order to ensure that the domain's own characteristics will not be diluted during the random walk process, the calculated The transfer probability matrix after the step is constructed using a self-loop mechanism to construct a high-order similarity matrix. The corresponding process has the following relationship:

[0056] ;

[0057] in, represents the high-order similarity matrix, Represents the weight attenuation factor that regulates the fusion ratio of self-loop information and high-order neighbor information, , represents the identity matrix;

[0058] According to the data distribution characteristics of different fields, the high-order similarity threshold is dynamically set, and any two households are With users The high-order similarity of is compared with the high-order similarity threshold, and the corresponding process has the following relationship:

[0059] ;

[0060] in, represents the high-order similarity threshold, Indicates household With users Higher-order similarity of

[0061] According to the comparison results, users who meet the similarity judgment in all fields are divided into similar behavior users, and users who do not meet the conditions are classified as dissimilar behavior users. The similar behavior user embedding and dissimilar behavior user embedding are obtained. The corresponding process has the following relationship:

[0062] ;

[0063] in, Represents a user set, Represents a set of users with similar behaviors, represents a set of users with non-similar behaviors, Indicates similar behavior user embedding, Represents user embeddings of non-similar behaviors.

[0064] The above steps have identified user embeddings with similar behaviors across multiple domains. However, data distribution and user behavior patterns vary across domains, causing highly interactive domains to cause noise interference in less interactive domains. Therefore, we use the embeddings of users with similar behaviors across multiple domains as positive samples and the embeddings of users with dissimilar behaviors as negative samples for comparative learning.

[0065] To promote the effective alignment of cross-domain behavioral features while maintaining the uniqueness of each domain, we define the positive sample set. The union of all similar behavior user embeddings in various fields, negative sample set is the union of all non-similar behavior user embeddings in each field, and the corresponding process has the following relationship:

[0066] ;

[0067] in, represents the positive sample set, represents the negative sample set, represents the domain set;

[0068] Based on the positive and negative sample sets, a contrastive learning loss is constructed. This is used to narrow the distance between the embeddings of users who exhibit similar behavior patterns through embedding contrast learning, while simultaneously moving the embeddings of users who do not exhibit similar behavior patterns further apart in space, resulting in an overall embedding aligned with their respective domains. The contrastive learning loss is expressed as follows:

[0069] ;

[0070] in, Represents the positive sample set For any two different embedding vectors in ; Represents the negative sample set Any embedding vector in represents the dot product operation, Represents the temperature parameter that controls the degree of differentiation between positive and negative samples in contrastive learning.

[0071] Through a comparative learning mechanism, features from various domains are effectively aligned, forming a consistent representation space across domains. Based on this, the present invention integrates the embeddings of users with similar and dissimilar behaviors in each domain with the corresponding item embeddings. This integrated embedding optimizes the relationship between users and items while preserving the original dimensions, more accurately reflecting potential matches between users and items.

[0072] Step 3: Design a cross-attention mechanism for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain.

[0073] Unlike traditional attention mechanisms, this paper designs a cross-domain interactive attention computation method, enabling each domain to obtain complementary information from other domains. This paper uses linear transformations to directly map the embeddings of each domain into the space required for attention computation, eliminating the need for feature splitting.

[0074] In this step, the aligned overall embedding of the target domain is first transformed linearly to generate a value vector, and the non-target domain is transformed linearly to generate its corresponding query vector and key vector. The corresponding process has the following relationship:

[0075] ;

[0076] in, and Represent the target domain and non-target domains The aligned overall embedding of 、 and Respectively represent Different trainable parameter matrices of the same dimension, Indicates non-target domain The query vector, Indicates non-target domain The key vector of Indicates the target domain A vector of values ​​for ;

[0077] According to the query vector and key vector, the attention score of each non-target domain is calculated. The corresponding process has the following relationship:

[0078] ;

[0079] in, represents the attention score of each non-target domain, Represents a specific bias term that adjusts the weight of each field’s influence on the target domain, The scaling factor representing the equilibrium dot product value;

[0080] Accumulate the scores of all non-target domains to obtain the multi-domain attention score ,use Softmax The function normalizes it and obtains the attention weight. The corresponding process has the following relationship:

[0081] ;

[0082] in, represents the attention weight, Softmax represents the normalized exponential function;

[0083] The attention weight is used to weight the value vector of the target domain, thereby fusing cross-domain information from various fields and generating an updated specific representation. The corresponding process has the following relationship:

[0084] ;

[0085] in, represents the updated specific representation. Thus, The target domain is retained The core features of the model are integrated with the supplementary information of other domains. Similarly, specific representations of other domains can be generated.

[0086] Based on this, the present invention achieves efficient cross-domain information fusion by improving the cross-attention mechanism. This design not only simplifies the complex feature splitting process in traditional methods, but also enables flexible cross-domain information regulation through domain-specific bias terms, providing more accurate input for subsequent global feature fusion.

[0087] Step 4: Fusion the overall embeddings of each domain into a global general domain embedding, then adjust the global general domain embedding into an adaptive representation for a specific domain, concatenate the adaptive representation for the specific domain with the specific representation of each domain, and make predictions for each domain to obtain the user-item prediction interaction score for each domain.

[0088] While the cross-attention module enables dynamic information interaction between domains, it still lacks a global modeling of common knowledge across all domains. To address this issue, the present invention designs a global feature fusion module to construct a unified representation that reflects users' common preferences across domains.

[0089] In this step, to effectively capture the common behavioral characteristics and preference patterns exhibited by users in multi-domain scenarios, the overall embeddings of each domain are organically integrated to form a unified global universal domain embedding. Therefore, for each domain, its overall embedding is accumulated and fused into a global universal domain embedding. The corresponding process is expressed as follows:

[0090] ;

[0091] in, represents the global general domain embedding.

[0092] This fusion strategy not only preserves the core features of each domain but also strengthens common cross-domain patterns. Compared to average or max pooling operations, the accumulation approach better preserves the unique contributions of each domain and avoids information loss. Furthermore, this global, universal domain embedding provides a rich foundation for subsequent adaptive representation generation, enabling the model to extract and transform the most relevant information based on the needs of specific domains, effectively integrating global knowledge with local features.

[0093] While global, general-purpose domain embeddings can capture users' common preferences, they struggle to accurately match the specific needs of users in each domain. To address this, we designed an MLP-based adaptation layer that dynamically adapts the global, general-purpose domain embeddings to domain-specific adaptive representations through nonlinear transformations. This adaptive representation reflects global commonalities while also adapting to the specific characteristics of each domain.

[0094] The global general domain is embedded in the MLP adaptation layer to learn domain-related parameters, mapped to a space that is more consistent with the current domain distribution, and generate domain-specific Adaptive representation of , the corresponding process has the following relationship:

[0095] ;

[0096] in, represents domain-specific adaptive representations, represents the adaptation layer consisting of a multi-layer fully connected network and a nonlinear activation function, For each field A separately learned set of parameters, consisting of multiple layers of weight matrices and bias vectors, enables the model to adaptively adjust how it applies global knowledge.

[0097] The adaptive representation of a specific field is then combined with the specific representation of each field to form the final representation of each field. The corresponding process has the following relationship:

[0098] ;

[0099] in, represents the final representation of each field;

[0100] This splicing strategy achieves the fusion of global general knowledge and domain-specific information. It is suitable for complex scenarios where user preferences are both related and different in different domains, and can provide richer and more accurate interaction feature representation for subsequent processing.

[0101] It is characterized in that the above steps 1 to 4 are implemented based on a multi-field recommendation model, and the training method of the multi-field recommendation model includes the following training steps:

[0102] In the MDR task, in order to effectively distinguish samples from different domains and improve the generalization ability of the model, a fully connected layer (FCL) is used to process the final representation of each domain. The domain discrimination loss is constructed in the form of multi-classification cross entropy, so that samples from different domains can be effectively distinguished in the feature space.

[0103] By giving the real domain labels, the domain discrimination loss is constructed according to the final representation of each domain and the real domain labels. The corresponding process has the following relationship:

[0104] ;

[0105] in, represents the indicator function, represents the real domain label, represents the domain discrimination loss, when Equal to the true domain label When , the value of the indicator function is 1, otherwise it is 0; Indicates that The output after mapping by the fully connected layer, Indicates a specific area, represents the final representation of a specific domain;

[0106] In order to enable the model to accurately distinguish the unique patterns in each field and effectively align similar behavioral features across domains, the contrastive learning loss and the discriminant loss are weighted together to construct the total loss function. The corresponding process has the following relationship:

[0107] ;

[0108] in, represents the total loss function, , denote the hyperparameters controlling the weights of embedding contrastive learning and domain discrimination loss, respectively. Represents the regularization coefficient used to prevent model overfitting when the domain data scale is unbalanced, represents the set of trainable parameters of the model, express Regularization;

[0109] By minimizing the total loss function to optimize the performance of the domain recommendation model, the final prediction score probability is obtained. The corresponding process has the following relationship:

[0110] ;

[0111] ;

[0112] in, represents the optimal parameter set obtained by minimizing the total loss function, represents the final predicted score probability, Represents the predicted score probability function.

[0113] See also Figure 2 This embodiment further provides a multi-domain recommendation system based on inter-domain knowledge transfer and cross-attention, wherein the system applies the multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention as described above, and the system includes:

[0114] Inter-domain knowledge transfer module, used to:

[0115] Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field;

[0116] Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain.

[0117] Cross-Attention Module, used to:

[0118] A cross-attention mechanism is designed for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain.

[0119] Global feature fusion module, used for:

[0120] The overall embeddings of each domain are fused into a global general domain embedding, which is then adjusted into an adaptive representation for a specific domain. The adaptive representation for a specific domain is then concatenated with the specific representation of each domain to make predictions for each domain, thereby obtaining the user-item prediction interaction scores for each domain.

[0121] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0122] It should be understood that various components of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0123] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0124] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention, characterized by: The method comprises the following steps: Step 1: Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field; Step 2: Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain. Step 3: Design a cross-attention mechanism for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain. Step 4: The overall embeddings of each domain are fused into a global general domain embedding, which is then adjusted into an adaptive representation for a specific domain. The adaptive representation for the specific domain is then concatenated with the specific representation for each domain to make predictions for each domain, thereby obtaining the user-item predicted interaction score for each domain. In step 1, during the propagation process, learning domain features through neighborhood aggregation to obtain the overall embedding of each domain specifically includes the following steps: The Xavier method is used to independently randomly initialize the user-item interaction graph in each domain to obtain user embedding. The corresponding process has the following relationship: ； in, represents uniform distribution, and Respectively represent the number of nodes corresponding to input and output, Represents a user Initial embedding of The Xavier method is used to independently randomly initialize the user-item interaction graph in each domain to obtain item embeddings; User embeddings and item embeddings are mapped to a low-dimensional vector space. In the low-dimensional vector space, the GCN-based neighborhood aggregation strategy is used to perform multi-layer iterative updates on the user and item embeddings of each field to obtain the user and item embeddings after feature learning in each field. In the multi-layer iterative update process of the user and item embeddings of each field, the following relationship exists in each iterative update process: ； in, represents the hyperparameter that balances own information and neighbor information, ; and Represents users and projects The set of neighbor nodes of represents the normalization term used to resize the embeddings during the neighbor aggregation process, Respectively represent the current field Layer embedding of users and items; Integrate the user embedding after feature learning and the item embedding after feature learning to obtain the overall embedding of the field. The corresponding process has the following relationship: ； in, Representation field The overall embedding of represents the concatenation of vectors, Respectively represent fields After feature learning, users and projects Embedded, Respectively represent fields User set after feature learning and project sets Embedding In step 2, based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. Specifically, the following steps are included: The Euclidean distance is used to measure the difference between any two user embedding vectors, and the negative exponential function is used to convert the distance into a similarity score. The similarity scores between all two users are arranged into a matrix according to the user index, and the user similarity matrix is ​​obtained. The corresponding process has the following relationship: ； in, represents the user similarity matrix, Represents a user With users The similarity score of express The general elements of Represents users With users The embedding vector of , represents the temperature parameter used to adjust the smoothness of the user similarity distribution within the domain, ; represents the L2 norm; Similarity Matrix Perform row normalization to obtain the transition probability, and construct the transition probability matrix according to the user index arrangement. The corresponding process has the following relationship: ； in, represents the transition probability matrix, express No. Rank Column elements, represents the summation indicator variable, express general elements of; Set the number of walking steps to , calculated by continuous multiplication The transmission probability matrix after the step, the corresponding process has the following relationship: ； in, express The transfer probability matrix after steps; According to the calculated The transfer probability matrix after the step is constructed using a self-loop mechanism to construct a high-order similarity matrix. The corresponding process has the following relationship: ； in, represents the high-order similarity matrix, Represents the weight attenuation factor that regulates the fusion ratio of self-loop information and high-order neighbor information, , represents the identity matrix; According to the data distribution characteristics of different fields, the high-order similarity threshold is dynamically set, and any two users With users The high-order similarity of is compared with the high-order similarity threshold, and the corresponding process has the following relationship: ； in, represents the high-order similarity threshold, Indicates household With users Higher-order similarity of According to the comparison results, users who meet the similarity judgment in all fields are divided into similar behavior users, and users who do not meet the conditions are classified as dissimilar behavior users. The similar behavior user embedding and dissimilar behavior user embedding are obtained. The corresponding process has the following relationship: ； in, Represents a user set, Represents a set of users with similar behaviors, represents a set of users with non-similar behaviors, Indicates similar behavior user embedding, Represents user embeddings of non-similar behaviors; In step 4, the overall embeddings of each domain are fused into a global universal domain embedding. The corresponding process has the following relationship: ； in, represents a global general domain embedding; In step 4, the global general domain embedding is adjusted to an adaptive representation for a specific domain. The corresponding process has the following relationship: ； in, represents domain-specific adaptive representations, represents the adaptation layer consisting of a multi-layer fully connected network and a nonlinear activation function, For each field Separately learned sets of parameters.

2. The multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention according to claim 1 is characterized in that: In step 2, a contrastive learning mechanism is used to align the cross-domain behavioral features of users with similar behaviors in multiple domains. The overall embedding obtained after alignment in each domain specifically includes the following steps: The positive sample set is defined as the union of all embeddings of users with similar behaviors in each field, and the negative sample set is defined as the union of all embeddings of users with non-similar behaviors in each field. The corresponding process has the following relationship: ； in, represents the positive sample set, represents the negative sample set, represents the domain set; Based on the positive and negative sample sets, a contrastive learning loss is constructed. This is used to narrow the distance between the embeddings of users who exhibit similar behavior patterns through embedding contrast learning, while simultaneously moving the embeddings of users who do not exhibit similar behavior patterns further apart in space, resulting in an overall embedding aligned with their respective domains. The contrastive learning loss is expressed as follows: ； in, Represents the positive sample set For any two different embedding vectors in ; Represents the negative sample set Any embedding vector in represents the dot product operation, Represents the temperature parameter that controls the degree of differentiation between positive and negative samples in contrastive learning.

3. The multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention according to claim 2 is characterized in that: In step 3, a cross-attention mechanism is designed for each domain, with the target domain as the main focus and the non-target domain as the auxiliary focus. The overall embedding after alignment of the respective domains is subjected to attention calculation using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain. Specifically, the following steps are included: The aligned overall embedding of the target domain is linearly transformed to generate a value vector, and the non-target domain is linearly transformed to generate its corresponding query vector and key vector. The corresponding process has the following relationship: ； in, and Represent the target domain and non-target domains The aligned overall embedding of 、 and Respectively represent Different trainable parameter matrices of the same dimension, Indicates non-target domain The query vector, Indicates non-target domain The key vector of Indicates the target domain A vector of values ​​of ; According to the query vector and key vector, the attention score of each non-target domain is calculated. The corresponding process has the following relationship: ； in, represents the attention score of each non-target domain, Represents a specific bias term that adjusts the weight of each field’s influence on the target domain, The scaling factor representing the equilibrium dot product value; The scores of all non-target domains are accumulated to obtain the multi-domain attention score, using Softmax The function normalizes it and obtains the attention weight. The corresponding process has the following relationship: ； in, represents the attention weight, Softmax represents the normalized exponential function, represents the multi-domain attention score; The attention weight is used to weight the value vector of the target domain, thereby fusing cross-domain information from various fields and generating an updated specific representation. The corresponding process has the following relationship: ； in, Indicates a specific representation after an update.

4. The multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention according to claim 3 is characterized in that In step 3, the adaptive representation of a specific domain is spliced ​​with the specific representation of each domain. The corresponding process has the following relationship: ； in, represents the final representation of each field.

5. The multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention according to claim 4 is characterized in that: The above steps 1 to 4 are implemented based on the multi-domain recommendation model. The training method of the multi-domain recommendation model includes the following training steps: Given the true domain label, the domain discrimination loss is constructed based on the final representation of each domain and the true domain label. The corresponding process has the following relationship: ； in, represents the indicator function, represents the real domain label, represents the domain discrimination loss, when Equal to the true domain label When , the value of the indicator function is 1, otherwise it is 0; Indicates that The output after mapping by the fully connected layer, Indicates a specific area, represents the final representation of a specific domain; The total loss function is constructed by weighting the contrastive learning loss and the discriminative loss. The corresponding process has the following relationship: ； in, represents the total loss function, , denote the hyperparameters controlling the weights of embedding contrastive learning and domain discrimination loss, respectively. Represents the regularization coefficient used to prevent model overfitting when the domain data scale is unbalanced, represents the set of trainable parameters of the model, express Regularization; By minimizing the total loss function to optimize the performance of the domain recommendation model, the final prediction score probability is obtained. The corresponding process has the following relationship: ； ； in, represents the optimal parameter set obtained by minimizing the total loss function, represents the final predicted score probability, Represents the predicted score probability function.

6. A multi-domain recommendation system based on inter-domain knowledge transfer and cross-attention, characterized by: The system applies the multi-domain recommendation method based on inter-domain knowledge transfer and cross-attention according to any one of claims 1 to 5, and the system includes: Inter-domain knowledge transfer module, used to: Obtain user-item interaction data from different fields, build an independent user-item interaction graph for each field, use GCN to propagate the user-item interaction graph, and learn domain features through neighborhood aggregation during the propagation process to obtain the overall embedding of each field; Based on the overall embedding of each domain, a high-order similarity matrix is ​​constructed through a random walk strategy to identify users with similar behaviors in multiple domains. A contrastive learning mechanism is then used to align the cross-domain behavioral features of similar users in multiple domains to obtain the overall embedding after alignment in each domain. Cross-Attention Module, used to: A cross-attention mechanism is designed for each domain, focusing on the target domain and supplemented by the non-target domain. The overall embedding after alignment of each domain is calculated using a cross-domain interactive approach, so that each domain can obtain complementary information from the non-target domain to obtain a specific representation of each domain. Global feature fusion module, used for: The overall embeddings of each domain are fused into a global general domain embedding, which is then adjusted into an adaptive representation for a specific domain. The adaptive representation for a specific domain is then concatenated with the specific representation of each domain to make predictions for each domain, thereby obtaining the user-item prediction interaction scores for each domain.

Images