Group purchase recommendation system based on multi-head attention mechanism and multi-task learning

Abstract

The invention relates to the technical field of recommendation systems and deep learning, in particular to a multi-head attention mechanism and multi-task learning-based group purchase recommendation system, which comprises an input learning module, a multi-head attention mechanism and multi-task learning module and a prediction module. A complex relationship among an initiator, a commodity and a participant is modeled through a plurality of graph convolutional networks, and embedded representation of each object in different views is learned. The multi-head attention mechanism and the multi-task learning module can enhance the feature representation capability, and promote the information interaction between two sub-tasks (initiating group purchase commodity recommendation and group purchase participant recommendation) through the expert network and the gating unit, so that the mutual promotion between the sub-tasks enhances the recommendation effect. The system designs two auxiliary loss functions for optimizing embedded representation and enhancing generalization and recommendation accuracy. The method can be widely applied to an e-commerce platform, solves the recommendation problem in a group buying scene, and achieves efficient and accurate group buying recommendation.

Description

Technical Field

[0001] The present invention relates to the field of recommendation systems, and in particular to a group purchase recommendation system based on a multi-head attention mechanism and multi-task learning, which aims to solve recommendation tasks in e-commerce group purchase scenarios. Background Art

[0002] In recent years, with the rapid development of e-commerce platforms, group buying has gradually become the main shopping method for users due to its characteristics of bulk discounts. However, the existing recommendation systems mainly focus on individual purchase scenarios and have obvious deficiencies in dealing with group buying scenarios. Although group recommendation methods can learn the common preferences of a group of users, they cannot effectively distinguish between the two roles of initiators and participants in group buying, and cannot handle the information differences in the initiation and participation processes. Therefore, the existing group recommendation models cannot be directly applied to complex group buying scenarios. Summary of the invention

[0003] To solve the above problems, the present invention proposes a group purchase recommendation system based on multi-head attention mechanism and multi-task learning, and integrates a variety of GCN variants (LightGCN, layergcn, UltraGCN, IPMGCN) in the system for comparative experiments to compare GCN variants that are more suitable for group purchases. It is mainly constructed by a multi-head attention mechanism and a multi-task learning module. In order to further improve the recommendation performance, some expert networks and gating units are designed to promote information interaction between the two subtasks. A multi-head attention mechanism is embedded in each expert network to achieve deep extraction of input features and capture the global dependencies between input data, significantly improving the accuracy and success rate of group purchase recommendations. The present invention can be widely used in e-commerce platforms to solve the recommendation problem in group purchase scenarios and achieve efficient and accurate group purchase recommendations.

[0004] The specific technical solutions are:

[0005] A group purchase recommendation system based on multi-head attention mechanism and multi-task learning, including an embedding learning module, a multi-head attention mechanism and multi-task learning module, and a prediction module;

[0006] Through the embedding learning module, the relationship between users (i.e., initiators and participants) and products is learned from the perspective of the initiator, the perspective of the participant, and the social perspective using the graph convolutional network (GCN). The embedded information of the learning object is obtained and then input into the multi-head attention mechanism and multi-task learning module.

[0007] The multi-head attention mechanism and multi-task learning module promote the information interaction between the two tasks of initiating and completing group purchases through expert networks and gating units. At the same time, a multi-head attention mechanism is embedded in each expert network to achieve deep extraction of input features and capture the global dependencies between input data.

[0008] Finally, the group purchase recommendation results are output through the prediction module.

[0009] Furthermore, the embedding learning module uses multiple GCNs to learn node embedding from the initiator perspective, participant perspective and social perspective: GI (initiator), GP (participan) and GIP (social), and then obtains the embedding information of the initiator, product and participant, and generates the corresponding embedding matrix, which can effectively extract the preference information of the initiator, product and participant.

[0010] The multi-head attention mechanism and multi-task learning module separate the features of learning tasks I and II through multiple expert networks, and dynamically adjust the flow of shared information through the gate control unit to ensure that the two subtasks promote each other in the learning process; in task I, the user has to select one or several products to initiate a group purchase, and the system recommends products that are not only liked by the initiator, but also liked by the participants; in task II, the participants want to join the group purchase, and the system will recommend a group purchase group consisting of products that both the participants and the initiator like, so that the participants can join;

[0011] Specifically, in each expert network, a multi-head attention mechanism is integrated, which is combined with a feedforward neural network to process input data and generate feature representations as a feature extractor for specific tasks or shared tasks; after the data is processed by the multi-head attention mechanism, the features are passed to the feedforward neural network for further processing; the feedforward neural network maps the input features to the hidden layer through two layers of linear transformation:

[0012] x hidden =ReLU(W 1 x+b 1 ) (1)

[0013] Output=W 2 x hidden +b 2 (2)

[0014] in is the weight matrix of the linear transformation, is the corresponding bias vector, x is the output of the multi-head attention mechanism, ReLU is the activation function, defined as ReLU(x)=max(0,x), and Output is the result of the data passing through two linear layers in the expert network;

[0015] To prevent overfitting and enhance the stability of the training process, batch normalization and Dropout are introduced in the feedforward neural network;

[0016] The role of batch normalization is to standardize the output of the intermediate layer to accelerate training and improve the stability of the system. Its formula is:

[0017]

[0018] Among them: μ and σ 2 Especially in the current small batch data A 1 The mean and variance of ; ∈ is a small positive number to prevent the denominator from being zero; in addition, the Dropout regularization method is used in the training process to randomly set the output of some neurons to zero to reduce the risk of overfitting;

[0019] Finally, after the feature transformation of the multi-head attention mechanism and the feedforward neural network, the output E(x) of the expert network is expressed as:

[0020] E(x) = Dropout(Output);

[0021] Next, the gating unit, as a key component of the system, is responsible for selecting the outputs of different expert networks. Each task corresponds to a dedicated gating network, which generates weights based on the input and dynamically selects the expert output that best suits the task. By assigning weights to different experts, the system can control the degree of attention paid to different expert networks.

[0022] The gating unit generates gating weights through two fully connected layers and ReLU activation function, and generates the selection weights of the expert network through the softmax function. The formula is:

[0023]

[0024] where α i represents the i-th output weight of the gating network, h 2j is the output value of the i-th neuron in the second layer.

[0025] It also includes capturing different features in Task I and Task II through multiple attention heads;

[0026] A multi-head attention mechanism is embedded in each expert network to achieve deep extraction of input features and capture the global dependencies between input data. The multi-head attention mechanism calculates the similarity weights between the query Q and the key K and uses these weighted values ​​V to generate a weighted feature representation. Specifically, the calculation formulas for Q, K, and V are:

[0027] Q=XW Q , K = XW K ,V=XW V (5)

[0028] in, That is, the input of multiple "heads".

[0029] The multi-head attention mechanism divides the input of the expert network into multiple "heads" and calculates the attention score on each head; the attention score is calculated by the dot product of the query and key vectors, and then scaled by division, the formula is:

[0030]

[0031] in are the query vector and the key vector, respectively, d head is the dimension of the head, Scaled; then, the attention score is converted into attention weight through the softmax function:

[0032]

[0033] In the formula, α i，j Represents the query Q i and K j relevance;

[0034] Finally, the weighted value vector of each head is calculated by the following formula, and the outputs of multiple heads are combined, and then the input dimension is restored through linear transformation:

[0035] O i =∑ j α i，j V j , (8)

[0036] MultiHeadOutput=Concat(head 1 , head 2 , ..., head h )W o , (9)

[0037] in is the weight matrix of the linear transformation.

[0038] This mechanism enables the system to capture shared features between different tasks through multi-head attention in a shared expert network. In a task-specific expert network, the multi-head attention mechanism can focus on specific features related to the task, thereby enhancing task specificity. By introducing the multi-head attention mechanism, not only is the feature representation capability improved, but long-distance dependencies are also effectively captured, further enhancing the system's multi-task learning capabilities.

[0039] The prediction module calculates the recommendation scores for Task I and Task II respectively through two multi-layer perceptrons, combines the feature representation of the multi-head attention mechanism, and finally outputs the recommendation results of group purchase products and participants.

[0040] The prediction module calculates the recommendation scores for Task I and Task II respectively through two multi-layer perceptrons, combines the feature representation of the multi-head attention mechanism, and finally outputs the recommendation results of group purchase products and participants.

[0041] Features and advantages of the present invention:

[0042] 1. By combining the multi-head attention mechanism, the system can effectively capture the global dependencies between initiators, products, and participants, making the embedding representation more accurate and significantly improving the recommendation effect.

[0043] 2. The system has designed a special embedding learning module, which is learned through multiple graph convolutional networks (LightGCN, layergcn, UltraGCN, IPMGCN), making full use of the multi-dimensional information between initiators, participants and products. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a framework diagram of the group purchase recommendation system MAMGBR of the present invention.

[0045] Figure 2 This is a diagram of the multi-head attention mechanism of the present invention. DETAILED DESCRIPTION

[0046] The system of the present invention is as follows Figure 1 The framework of the group purchase recommendation system MAMGBR is shown in Figure 1. The multi-head attention mechanism used is as follows Figure 2 shown.

[0047] 1. Evaluation indicators.

[0048] The present invention uses MRR@N (mean reciprocal rank) and NDCG@N (normalized discounted cumulative gain) as indicators to evaluate the recommendation performance of all baseline systems. These indicators are widely used in the evaluation of ranking-based recommendation systems, especially NDCG pays more attention to the specific ranking position of positive samples / participants in the recommendation list compared to MRR. For the ratio of positive and negative samples in the test set, MRR / NDCG@10 is first calculated using a positive-to-negative sample ratio of 1:9, and MRR / NDCG@100 is further calculated under the condition that the positive-to-negative sample ratio is 1:99, so as to comprehensively evaluate the recommendation performance of all compared systems.

[0049] All experiments were conducted on a server equipped with a GeForce RTX 4090 graphics processor (24GB memory), running in the Ubuntu 22.04 operating system, and using torch 2.3.0 and cuda 12.1 for system training and inference.

[0050] 2. Comparative test results and analysis

[0051] Table 1 Comparative test evaluation results of MGBR in the present invention

[0052]

[0053] The evaluation results of all baselines in this invention are shown in Table 1. The results are shown in the two tasks at the ratio of 1:9 and 1:99 positive and negative samples, MRR and NDCG. From Table 1, we can see that IPMGCN has the best performance in both subtasks, especially in Task I. This shows the advantages of introducing multi-view structure, multi-head attention mechanism and GCN.

Claims

1. A group purchase recommendation system based on multi-head attention mechanism and multi-task learning, characterized in that: Includes embedding learning module, multi-head attention mechanism and multi-task learning module, prediction module; Through the embedding learning module, the relationship between users and products is learned from the perspective of the initiator, the perspective of the participant, and the social perspective, and the graph convolutional network GCN is used to obtain the embedding information of the learning object, which is then input into the multi-head attention mechanism and the multi-task learning module; the users are the initiators and participants; The multi-head attention mechanism and multi-task learning module promote the information interaction between the two tasks of initiating and completing group purchases through expert networks and gating units. At the same time, a multi-head attention mechanism is embedded in each expert network to achieve deep extraction of input features and capture the global dependencies between input data. Finally, the group purchase recommendation results are output through the prediction module.

2. According to claim 1, a group purchase recommendation system based on multi-head attention mechanism and multi-task learning is characterized in that: The embedding learning module uses multiple GCNs to learn node embedding from the initiator's perspective, the participant's perspective and the social perspective: GI, GP and GIP, and then obtains the embedding information of the initiator, goods and participants, and generates the corresponding embedding matrix, which can effectively extract the preference information of the initiator, goods and participants.

3. According to claim 1, a group purchase recommendation system based on multi-head attention mechanism and multi-task learning is characterized in that: The multi-head attention mechanism and multi-task learning module separate the features of learning tasks I and II through multiple expert networks, and dynamically adjust the flow of shared information through the gate control unit to ensure that the two subtasks promote each other in the learning process; in task I, the user has to select one or several products to initiate a group purchase, and the system recommends products that are not only liked by the initiator, but also liked by the participants; in task II, the participants want to join the group purchase, and the system will recommend a group purchase group consisting of products that both the participants and the initiator like, so that the participants can join; Specifically, in each expert network, a multi-head attention mechanism is integrated, which is combined with a feedforward neural network to process input data and generate feature representations as a feature extractor for specific tasks or shared tasks; after the data is processed by the multi-head attention mechanism, the features are passed to the feedforward neural network for further processing; the feedforward neural network maps the input features to the hidden layer through two layers of linear transformation: x hidden =ReLU(W1x+b1) (1) Output=W2x hidden +b2, (2) in is the weight matrix of the linear transformation, is the corresponding bias vector, x is the output of the multi-head attention mechanism, ReLU is the activation function, defined as ReLU(x)=max(0,x), and Output is the result of the data passing through two linear layers in the expert network; To prevent overfitting and enhance the stability of the training process, batch normalization and Dropout are introduced in the feedforward neural network; The role of batch normalization is to standardize the output of the intermediate layer to accelerate training and improve the stability of the system. Its formula is: Among them: μ and σ 2 In particular, they are the mean and variance of A1 in the current mini-batch data; ∈ is a very small positive number to prevent the denominator from being zero; in addition, the Dropout regularization method is used during training to randomly set the output of some neurons to zero to reduce the risk of overfitting; Finally, after the feature transformation of the multi-head attention mechanism and the feedforward neural network, the output E(x) of the expert network is expressed as: E(x) = Dropout(Output); Next, the gating unit, as a key component of the system, is responsible for selecting the outputs of different expert networks. Each task corresponds to a dedicated gating network, which generates weights based on the input and dynamically selects the expert output that best suits the task. By assigning weights to different experts, the system can control the degree of attention paid to different expert networks. The gating unit generates gating weights through two fully connected layers and ReLU activation function, and generates the selection weights of the expert network through the softmax function. The formula is: where α i represents the i-th output weight of the gating network, h 2j is the output value of the i-th neuron in the second layer.

4. According to claim 3, a group purchase recommendation system based on multi-head attention mechanism and multi-task learning is characterized in that: It also includes capturing different features in Task I and Task II through multiple attention heads; A multi-head attention mechanism is embedded in each expert network to achieve deep extraction of input features and capture the global dependencies between input data. The multi-head attention mechanism calculates the similarity weights between the query Q and the key K and uses these weighted values ​​V to generate a weighted feature representation. Specifically, the calculation formulas for Q, K, and V are: Q=XW Q ,N=XW K ,V=XW V (5) in, That is, the input of multiple "heads"; The multi-head attention mechanism divides the input of the expert network into multiple "heads" and calculates the attention score on each head; the attention score is calculated by the dot product of the query and key vectors, and then scaled by division, the formula is: in are the query vector and the key vector, respectively, and d head is the dimension of the head, using Scaled; then, the attention score is converted into attention weight through the softmax function: In the formula, α i，j Represents the query Q i and K j relevance; Finally, the weighted value vector of each head is calculated by the following formula, and the outputs of multiple heads are combined, and then the input dimension is restored through linear transformation: The i =∑ j a i，j V j , (8) MultiHeadOutput=Concat(head1,head2,...,head h )W O , (9) in is the weight matrix of the linear transformation.

5. A group purchase recommendation system based on multi-head attention mechanism and multi-task learning according to claim 3 or 4, characterized in that: The prediction module calculates the recommendation scores for Task I and Task II respectively through two multi-layer perceptrons, combines the feature representation of the multi-head attention mechanism, and finally outputs the recommendation results of group purchase products and participants.

Images