Hybrid popular recommendation method based on multiple tasks

Abstract

The invention provides a mixed popular recommendation method based on multiple tasks. The method comprises the following steps: performing multi-task-based popularity-deviation-removing causal recommendation, performing causal recommendation analysis, modeling popularity deviation influence under different causal paths by utilizing multiple tasks, and combining a deviation amplification score and other popularity deviation factors to realize popularity-deviation-removing recommendation; based on popularity utilization of project popularity propagation modeling, a dynamic span is proposed to model popularity short-term fluctuation, and future popularity is predicted in combination with an adaptive attention mechanism; and modeling propagation momentum based on future popularity, and finally fusing the debiased recommendation score and the propagation momentum to realize hybrid popularity recommendation. According to the method, the recommendation precision and accuracy are effectively improved, the popularity deviation can be accurately eliminated, the project popularity is utilized to spread the information, and particularly, the effect on a long-tail data set is more remarkable.

Description

Technical Field

[0001] The present invention relates to the technical field of recommendation systems, and specifically to a hybrid popular recommendation method based on multi-tasks, which eliminates popular deviations in the recommendation system and combines popular propagation information of items for accurate recommendation, so as to improve the accuracy of the recommendation system and user satisfaction. Background Art

[0002] In recent years, personalized recommendation has become an effective way to alleviate information overload and plays an important role in online applications such as e-commerce, video websites and social media. Existing mainstream recommendation methods usually make recommendations based on historical interaction data. However, the distribution of items in historical interactions is usually non-uniform (mainly long-tail), with a few items occupying most interactions, leading to the problem of popularity bias. The recommendation model trained based on this long-tail distribution data further amplifies the bias, making the recommendation system tend to recommend items with higher popularity, causing problems such as echo chambers, filter bubbles, and the Matthew effect, which hinders the recommendation method from accurately obtaining user interests and reduces the accuracy and fairness of recommendations. However, blindly pursuing the elimination of popularity bias may lead to unfair suppression of items that deserve to be more popular. Therefore, from a compromise perspective, hybrid popularity recommendation is studied to eliminate the negative effects of popularity bias while giving play to the positive role of popularity, which helps to improve recommendation accuracy and user satisfaction.

[0003] Current hybrid popular recommendation research either adopts a two-stage strategy to utilize popularity while eliminating bias, or adopts a fusion strategy to jointly train biased and unbiased models, and achieves the final recommendation based on the fusion of the two models. However, most of these hybrid popular recommendation methods ignore the amplification effect of the recommendation algorithm on the popular bias when eliminating the popular bias, making it difficult to accurately remove the bias. At the same time, when predicting popularity, they ignore the impact of short-term subtle fluctuations in popularity and the impact of the popular propagation trend of the project in future popularity prediction, which affects the final recommendation performance.

[0004] In order to address the limitations of removing popularity bias in recommendation research, which is difficult to accurately remove bias due to ignoring the amplifying effect of recommendation methods on popularity bias, and the fact that most existing popularity utilization recommendation methods ignore the short-term subtle fluctuations in popularity in future popularity predictions and are difficult to learn project propagation trends and influence, the present invention designs and implements a multi-task based hybrid popularity recommendation method, which eliminates popularity bias through causal recommendation analysis and multi-task modeling, and realizes popularity utilization in combination with project popularity propagation modeling, thereby improving recommendation performance. Summary of the invention

[0005] This paper proposes a hybrid popular recommendation method based on multi-tasks, which aims to make accurate recommendations by eliminating the popular bias in the recommendation system and combining the popular propagation information of items.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme.

[0007] A hybrid popular recommendation method based on multi-tasks includes the following steps:

[0008] Step 1: Perform causal recommendation analysis on the popularity bias in the recommendation system, identify and model different causal paths of popularity bias;

[0009] Step 2: Use a multi-task learning model to model the conformity score, exposure score, and bias amplification score respectively, and combine them with the popularity bias in the approximate recommendation results to achieve de-popularity biased recommendations and obtain de-biased recommendation scores generated by user interests;

[0010] Step 3: By combining dynamic span and adaptive attention mechanism with popularity position embedding, a popularity feature extraction model based on dynamic span is established to capture both the long-term dependency and short-term fluctuation of item popularity, thus improving popularity prediction and recommendation performance.

[0011] Step 4: Model the propagation momentum based on the future popularity prediction value as a representation of the project's popularity and propagation ability;

[0012] Step 5: Combine the recommendation score without popularity bias and the propagation momentum to obtain the hybrid popularity recommendation score result. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the framework of the multi-task based hybrid popular recommendation MPHR method of the present invention;

[0014] Figure 2 It is a comparison chart of the test results of the present invention and other methods on the test data set. DETAILED DESCRIPTION

[0015] The method of the present invention is further described in detail below based on the implementation principle and specific data processing flow.

[0016] 1. Data preprocessing

[0017] First, we collect data from user-item interaction logs, including user ID, item ID, interaction timestamp, etc. We define the item set and user set as I = {i 1 ,i 2 ,...i m} and U={u 1 ,u 2 ,...u n}, m and n represent the number of items and users respectively, and the actual user-item interaction y u,i :

[0018]

[0019] The popularity of each item is calculated based on the interaction timestamp. The present invention uses the number of interactions between the user and the item within a certain period of time as a measurement indicator of popularity.

[0020] Define the popularity of an item over a period of time The calculation method is as follows:

[0021]

[0022] Next, the interaction log is divided into s time periods according to time, and the total time is T = {t 1 ,t 2 ,...t s},based on Construct a popularity set. It can be constructed as a popularity time series matrix Z for subsequent analysis and modeling. Its definition is as follows:

[0023]

[0024] 2. Causal recommendation analysis

[0025] The causal graph is used to conduct a detailed causal analysis of the recommendation process and identify the main sources and propagation paths of the popularity bias. The causal graph contains multiple nodes such as users, projects, popularity, user interests, and directed edges between them, representing the direct causal effects between nodes. This paper models a more complete causal graph, adding node Z to represent popularity through analysis. Through analysis, the three main causal paths of popularity bias through user conformity, project exposure, and recommendation method amplification are determined.

[0026] 3. Multi-task Popularity Deviation Modeling

[0027] According to the above three causal paths, modeling methods for conformity score, exposure score and deviation amplification score are designed respectively:

[0028] Conformity score: By modeling the matching characteristics between user representation and popularity representation, the probability of a user clicking on a highly popular item due to conformity is calculated, formally expressed as P comfority , the calculation method is as follows:

[0029]

[0030] Among them, K(·) represents the matching function for learning the matching features between user representation and popularity representation, which is modeled by vector inner product. Y(·) is the scoring function for calculating the recommendation score, which is calculated by summing the embedding dimensions.

[0031] Exposure score: By modeling the matching characteristics between the item representation and the popularity representation, the probability of an item being overexposed due to high popularity, thus generating additional interactions, formally expressed as P exposure , the calculation method is as follows:

[0032]

[0033] Among them, K(·) represents the matching function between the learning item representation and the popularity representation, which is modeled by the vector inner product. Y(·) is the scoring function, which is calculated by summing the embedding dimensions.

[0034] Deviation amplification score: directly use the direct impact of popularity on the interaction results to model the degree to which the recommendation method amplifies the popularity deviation, formally expressed as P magnify , the calculation method is as follows:

[0035]

[0036] Here, Y(·) is also calculated by summing over the embedding dimensions.

[0037] A multi-task learning model is adopted to design loss functions for the conformity score, exposure score and bias amplification score respectively, and accurate popular bias modeling is achieved through joint optimization.

[0038] The present invention uses a multi-task learning model to supervise and Learning. Since the popular bias modeling problem can be regarded as a binary classification task for the conformity score, exposure score, and bias amplification score, the binary cross-entropy (BCE) loss function is selected to construct the optimization objectives respectively to measure the differences between the three and the actual interaction results, and calculate the differences between them and the actual interaction. Conformity score loss function L U , exposure score loss function L I and the bias amplification score loss function L Z As shown below:

[0039]

[0040] Where D represents the actual interaction data and σ(·) represents the sigmoid function.

[0041] 4. Remove popular bias in recommendations

[0042] On the basis of the mixed recommendation score (i.e. the recommendation score under the influence of user interest and popularity bias), the debiased recommendation score based purely on user interest is obtained by deducting the fusion result of the conformity score, exposure score and bias amplification score. This step effectively eliminates the popularity bias in the recommendation results, thereby improving the accuracy of the recommendation without being affected by the popularity bias.

[0043]

[0044] Among them, c is a hyperparameter used to adjust The product of the three is used to obtain debiased recommendations based purely on real user interests.

[0045] Since the mixed recommendation score modeling can be regarded as a binary classification task for the mixed recommendation score, the mixed recommendation score also uses BCE loss, and the loss function L K With three popular deviation loss functions L U , L I and L Z Fusion obtains the final debiased loss function L debias :

[0046]

[0047] L debias =L k +αL U +βL I +yL Z

[0048] 5. Popularity Embedding Representation

[0049] In order to better capture the characteristics of project popularity, a popularity embedding representation method is proposed. Combining popularity position embedding (representing the position information of popularity in the time series) and original popularity embedding (representing the size of popularity), the ability to represent project popularity is enhanced by fusing the two, providing rich feature information for subsequent prediction and modeling.

[0050] The popularity time position embedding vector is represented by PE and is calculated as follows:

[0051] PE (pos,2i) =sin(pos / 10000 2i / d )

[0052] PE (pos,2i+1) =cos(pos / 10000 2i / d )

[0053] Among them, pos is the sequence position of element i, and d is the dimension of the popularity time position embedding vector PE. The popularity size is extracted from the popularity sequence matrix as the popularity original embedding PP, and the popularity position embedding is fused with the popularity original embedding to obtain the popularity embedding X, as shown below:

[0054] X=PE+PP

[0055] In this way, each popularity embedding vector contains not only the item popularity size information, but also the corresponding time series position information, which greatly enhances the ability to represent item popularity propagation.

[0056] 6. Dynamic span modeling and future popularity prediction

[0057] In order to more accurately predict the future popularity of a project, the concept of dynamic span is proposed. By adaptively adjusting the time window size, the subtle fluctuations in popularity in the short term are captured. At the same time, the dynamic span is integrated into the adaptive attention mechanism to capture the long-term dependence of popularity changes while enhancing attention to short-term fluctuations. Through the combination of dynamic span and adaptive attention mechanism, accurate prediction of future popularity is achieved.

[0058] The popularity feature extraction based on dynamic span includes dynamic span modeling, adaptive attention layer, feedforward layer and residual layer, which are introduced below respectively.

[0059] (1) Dynamic span modeling

[0060] Since the contribution of popularity changes in different time periods to the spread of the project is not evenly distributed in the popularity sequence, some short-term fluctuations may have a more important impact on popular spread. Therefore, this paper adjusts the time range of focus by modeling dynamic spans, thereby enhancing the learning of short-term fluctuations in popularity. The key to dynamic span is to dynamically adjust the span size according to the data. The span of the self-attention head is represented by , and the calculation method is as follows:

[0061] S h =sigmoid(XW h +b h )L

[0062] Among them, W h and b h is a learnable parameter, and L is the length of the popularity sequence. To achieve adaptive attention, each attention head needs to have a different span size and a span parameter S is assigned to each attention head h. h Then we model the mask matrix M(S) based on the dynamic stride matrix S, whose elements M ij (S) is calculated as follows:

[0063]

[0064] This means that if the distance between positions i and j does not exceed the span parameter S h , the corresponding mask value is 1, if the distance exceeds, the corresponding mask value is 0.

[0065] (2) Adaptive Attention Layer

[0066] The input of the adaptive attention layer is the popularity embedding X and the mask matrix M(S), which is obtained by three linear transformations:

[0067] Q=XW Q

[0068] K=XW K

[0069] V=XW V

[0070] Among them, W Q ∈R D×H , W K ∈R D×H and W V ∈R D×H is a matrix with three dimensions of D×H. Since the contribution of popularity changes in different time periods to the project's propagation ability is not evenly distributed in the popularity sequence, some short-term fluctuations may have a more important impact on popular propagation. Therefore, the dynamic span is integrated into the adaptive attention mechanism, and the short-term time range of focus is adjusted by dynamic span, so as to adaptively enhance short-term fluctuation learning. The output result of the adaptive attention layer is O att As shown below:

[0071]

[0072] in, is a scaling factor used to avoid too large inner product values. Since positions beyond the span will not affect the self-attention results, and positions within the span will improve self-attention, it is possible to learn long-term dependencies on popularity changes while paying close attention to short-term fluctuations in popularity.

[0073] (3) Feedforward-Residual Layer

[0074] To avoid overfitting and enhance robustness, a fully connected layer is used as the feedforward layer, and a dropout layer is added after it. The formula is as follows:

[0075] O lin = dropout(O att W O +b O )

[0076] Among them, W O ∈R H×D and b O ∈R C×D is the training weight matrix and bias. To enhance the popular propagation representation, O linCombined with the popularity embedding vector, the output result is obtained that contains both the local popularity information of each time period and the global information of the popularity evolution in the entire time series.

[0077] After layer normalization, we get the future popularity vector O Z , and after dimension transformation, the final output result FP is obtained:

[0078] O Z =LN(O lin +X)

[0079] FP=O Z W Z +b Z

[0080] Among them, LN(·) represents layer normalization, W Z ∈R D×1 and b Z ∈R C×1 Used for dimension transformation.

[0081] 7. Propagation Momentum Modeling

[0082] Drawing on the concept of momentum in physics, a method for measuring communication momentum is proposed. Based on the predicted value of future popularity, combined with the speed of popularity change in the current time period and the previous time period, the communication momentum of the project is calculated. Communication momentum not only takes into account the size of popularity, but also the speed of popularity change, which more comprehensively reflects the popularity and influence of the project.

[0083] The propagation momentum is used as a representation of the popularity of a project. The propagation momentum is modeled based on the future popularity prediction value to replace the future popularity in the final recommendation. Using the speed of change of popularity in the current time period as the propagation speed, the calculation method of the propagation momentum p of a project is as follows:

[0084]

[0085] in, Indicates the predicted value of future popularity of a project in the next time period, which comes from the future popularity vector FP, t n and t n-1 Respectively represent the current time period and the previous time period, and The larger the popularity value or the faster the spread speed, the greater the spread momentum. That is, if a project has a higher user attention or attracts user attention faster, the project has a greater spread momentum.

[0086] 8. Mixed Popular Recommendations

[0087] On the basis of debiased recommendation, the propagation momentum information is integrated to obtain the final hybrid popular recommendation result. This method takes into account both the elimination of popular bias and the active use of popularity, avoiding the problem of recommendation results being too concentrated on highly popular items while ensuring the diversity and accuracy of recommendation results.

[0088] Use the propagation momentum to replace the future popularity prediction value. To achieve hybrid popular recommendation, migrate the recommendation score without popular bias As a pre-recommendation score, it is combined with the communication momentum to achieve popularity utilization and obtain the final hybrid popular recommendation score

[0089]

[0090] Among them, σ(·) is the sigmoid function, which is used to ensure that the recommendation score is between 0 and 1, and q is the popular propagation smoothing hyperparameter, which is used to control the intensity of popular propagation influence. Consistent with the recommendation without popular bias, the BCE loss is also used to construct the optimization target, and the loss function is as follows:

[0091]

[0092] Where D represents the actual interaction data.

[0093] Through the above technical solution, the present invention realizes a hybrid popular recommendation method based on multiple tasks, which effectively improves the performance of the recommendation system and the user experience.

[0094] The above are only preferred specific embodiments of the present invention, but the protection scope is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope should be based on the content of the claims.

Claims

1. A hybrid popular recommendation method based on multi-task, characterized in that: The following steps are involved: S1: Conduct causal recommendation analysis on the popularity bias in recommendation systems, identify and model different causal paths of popularity bias; S2: Adopting a multi-task learning model, modeling the conformity score, exposure score, and bias amplification score respectively, combined with the popularity bias in the approximate recommendation results, to achieve de-popularity biased recommendations, and obtain de-biased recommendation scores generated by user interests; S3: By combining dynamic span and adaptive attention mechanism with popularity position embedding, a popularity feature extraction model based on dynamic span is established to simultaneously capture the long-term dependency and short-term fluctuation of project popularity, thus improving popularity prediction and recommendation performance; S4: Modeling the spreading momentum based on the future popularity prediction value as a representation of the project's popular spreading ability; S5: Fusion of the de-popularity biased recommendation score and the diffusion momentum to obtain a hybrid popular recommendation score result.

2. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific process of the causal recommendation analysis in step S1 is as follows: In the context of recommendation systems, using causal graphs can help us systematically understand which factors affect key indicators such as recommendation effectiveness, accuracy, and user satisfaction. The present invention adds consideration of popularity to the traditional causal graph and models a more complete causal graph, that is, adding node Z to represent popularity. From the perspective of the recommendation system, the interactive behaviors related to popularity bias can be divided into the following three categories: ①Popularity indirectly affects interaction through users, i.e., path Z→U→Y. Corresponding to the popular deviation caused by user conformity behavior, it can be formally expressed as P comfority ; ②Popularity indirectly affects interaction through projects, i.e., path Z→I→Y. Corresponding to the popularity deviation caused by project overexposure, it can be formally expressed as P exposure ; ③Popularity directly affects interaction. That is, path Z→Y, corresponding to the popularity deviation caused by the amplification effect of the recommendation method, can be formally expressed as P magnify . The above three causal paths together constitute and amplify the popularity bias P in the recommendation system. bias , formally expressed as formula: P bias =σ(P comfority +P exposure +P magnify ) 3. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific method of the multi-task learning mode in step S2 is: The conformity score represents the indirect effect of popularity bias on interaction through user conformity behavior, and is calculated as: Among them, K(·) represents the matching function for learning the matching features between user representation and popularity representation. It is modeled by vector inner product, and Y(·) is the scoring function for calculating the recommendation score, which is calculated by summing the embedding dimensions. The exposure score represents the indirect effect of popularity bias on the interaction through the item exposure level, and is calculated as: Among them, K(·) represents the matching function of the matching features between the learning item representation and the popularity representation, which is modeled by the vector inner product, and Y(·) is the scoring function, which is calculated by summing the embedding dimensions. The deviation amplification score represents the direct impact of popularity on the interaction results, and is calculated as: Here, Y(·) is also calculated by summing over the embedding dimensions. To accurately model the conformity score, exposure score, and bias amplification score. The present invention uses a multi-task learning model to supervise and Learning. Conformity score loss function L U , exposure score loss function L I and the bias amplification score loss function L Z They are as follows: Where D represents the actual interaction data and σ(·) represents the sigmoid function. BCE loss is a loss function commonly used for binary classification problems. The predicted score is converted to between 0 and 1 through the sigmoid activation function, and the difference between it and the actual interaction is calculated.

4. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific method for removing the popular deviation recommendation proposed in step S2 is: The fusion result of conformity score, exposure score and deviation amplification score is deducted from the mixed recommendation score, so as to obtain the debiased recommendation score that is not affected by the popular bias and is only generated by user interests, as shown in the formula: Among them, c is a hyperparameter used to adjust The product of the three is used to obtain the bias-free recommendation based purely on the real user interests. The mixed recommendation score also uses BCE loss. Loss function L K , and three popular deviation loss functions L U , L I and L Z Fusion obtains the final debiased loss function L debias As shown below: L debias =L k +αL U +βL I +γL Z Where D represents the actual interaction data, σ(·) represents the sigmoid function, and α, β, and γ are hyperparameters used to control the weights of the conformity score, exposure score, and bias amplification score.

5. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific method of embedding popularity position proposed in step S3 is: The popularity time position embedding vector is represented by PE and is calculated as follows: ON (pos,2i) =sin(pos / 10000 2i / d ) ON (pos,2i+1) =cos(pos / 10000 2i / d ) Among them, pos is the sequence position of element i, d is the dimension of the popularity time position embedding vector PE, the popularity size is extracted from the popularity sequence matrix as the popularity original embedding PP, and the popularity position embedding is fused with the popularity original embedding to obtain the popularity embedding X, the formula is as follows: X=PE+PP Popularity position embedding is used to preserve the positional relationship of popularity in different time periods in the time series to enhance the representation of project popularity propagation.

6. A multi-task based hybrid popular method according to claim 1, characterized in that: The popularity feature extraction model based on dynamic span proposed in step S3 includes: (1) Dynamic span modeling In the popularity sequence, the contribution of popularity changes in different time periods to the project's propagation ability is not evenly distributed. Some short-term fluctuations may have a more important impact on popular propagation. Therefore, this paper enhances the learning of short-term fluctuations in popularity by modeling a dynamic span to adjust the time range of focus. The key to the dynamic span is to dynamically adjust the span size according to the data. The span of the self-attention head is represented by , and the calculation method is as follows: S h =sigmoid(XW h +b h )L Among them, W h and b h is a learnable parameter, L is the length of the popularity sequence, and to achieve adaptive attention, each attention head needs to have a different span size, and a span parameter S is assigned to each attention head h. h , and then model the mask matrix M(S) based on the dynamic stride matrix S, whose elements M ij (S) is calculated as follows: This means that if the distance between positions i and j does not exceed the span parameter S h , the corresponding mask value is 1, if the distance exceeds, the corresponding mask value is 0. (2) Adaptive Attention Layer The input of the adaptive attention layer is the popularity embedding X and the mask matrix M(S), which is obtained by three linear transformations: Q=XW Q K=XW K V=XW V Among them, W Q ∈R D×H , W K ∈R D×H and W V ∈R D×H is a matrix with three dimensions of D×H. Since the contribution of popularity changes in different time periods to the project's propagation ability is not evenly distributed in the popularity sequence, the impact of some short-term fluctuations on popular propagation may be more important. Therefore, the dynamic span is integrated into the adaptive attention mechanism. The dynamic span adjusts the short-term time range of focus to adaptively enhance short-term fluctuation learning. The output result of the adaptive attention layer is O att The calculation method is as follows: in, is a scaling factor used to prevent the inner product value from being too large, because positions beyond the span will not affect the self-attention result, while positions within the span will improve self-attention. Therefore, it is possible to pay close attention to short-term fluctuations in popularity while learning long-term dependencies on popularity changes. (3) Feedforward-Residual Layer To avoid overfitting and enhance robustness, a fully connected layer is used as the feedforward layer, and a dropout layer is added after it. The calculation method is as follows: O lin =dropout(O att W O +b O ) Among them, W O ∈R H×D and b O ∈R C×D is the training weight matrix and bias. To enhance the popular propagation representation, O lin Combined with the popularity embedding vector, the output result is obtained, which contains both the local information of popularity in each time period and the global information of popularity evolution in the entire time series. After layer normalization, the future popularity vector O is obtained. Z After dimension transformation, the final output result FP is obtained. The calculation process is as follows: O Z =LN(O lin +X) FP=O z W z +b z Among them, LN(·) represents layer normalization, W Z ∈R D×1 and b Z ∈R C×1 Used for dimension transformation.

7. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific method of modeling the propagation momentum in step S4 is: modeling the propagation momentum based on the future popularity prediction value to replace the future popularity in the final recommendation. Using the popularity change speed in the current time period as the propagation speed, the propagation momentum p of a certain project is calculated as follows: in, Indicates the predicted value of future popularity of a project in the next time period, which comes from the future popularity vector FP, t n and t n-1 Respectively represent the current time period and the previous time period, and They represent the popularity of a project in the current time period and the previous time period respectively.

8. A multi-task based hybrid popular method according to claim 1, characterized in that: The specific method of step S5 is: The present invention uses the propagation momentum to replace the future popularity prediction value, in order to achieve hybrid popular recommendation, migrate the popular deviation recommendation score As a pre-recommendation score, it is combined with the communication momentum to achieve popularity utilization and obtain the final hybrid popular recommendation score Among them, σ(·) is the sigmoid function, which is used to ensure that the recommendation score is between 0 and 1. q is the popular propagation smoothing hyperparameter, which is used to control the intensity of popular propagation influence. It is consistent with the recommendation without popular deviation. The BCE loss is also used to construct the optimization target. The loss function calculation method is as follows: Where D represents the actual interaction data.

Images