A time- and content-aware popularity debiasing approach for news recommendation systems

By building a TCCM model, combining news timeliness and popularity, using multi-head self-attention and cross-attention networks, the problems of popularity deviation and high computational complexity in the news recommendation system are solved, achieving more accurate recommendations and performance improvements .

CN118797172BActive Publication Date: 2024-12-06HUAQIAO UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411273935.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-06
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

The existing news recommendation system ignores time factors when dealing with popularity deviations, resulting in unreasonable recommendation results, high computational complexity and long time.

Method used

Using the time and content-aware news recommendation system popularity de-bias method, by constructing a TCCM model, calculating news timeliness and popularity, combining user preferences, using a multi-head self-attention network and a cross-attention network to learn news popularity representation, and Causal intervention is performed during the reasoning phase, and the popularity value is controlled to alleviate bias.

Benefits of technology

Achieve more accurate news popularity estimation, reduce popularity bias, improve the performance and user experience of recommendation systems, and reduce computational complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure SMS_51
    Figure SMS_51
  • Figure SMS_52
    Figure SMS_52
  • Figure SMS_53
    Figure SMS_53
Patent Text Reader

Abstract

The present invention discloses a time- and content-aware news recommendation system popularity debiasing method, comprising the following steps: (a) preprocessing a news data set; (b) constructing a TCCM model; (c) training the model; and (d) performing causal intervention in the inference phase to alleviate popularity bias. The present invention considers the impact of news content on popularity, and by combining news content (popularity of entities and words), more accurately estimates news popularity, which is helpful for the development of news recommendation systems.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to algorithm optimization of a news recommendation system, and more specifically, a time- and content-aware popularity debiasing method for a news recommendation system is proposed. Background Art

[0002] With the convenience and real-time update of online news services, many readers have turned from reading paper newspapers to electronic news. However, given the huge number of news reports emerging every day, it is almost impossible for users to select interesting content from the massive amount of news, which leads to the challenge of "information overload" faced by users and websites. Therefore, the use of personalized recommendation systems to filter news based on user preferences has become a solution. For news platforms, this technology can reduce the pressure of information overload on users and optimize their news browsing experience. However, news recommendation systems are affected by the popularity of news, resulting in biased recommendation results and unsatisfactory performance. This is because popular news articles will get more exposure and are often delivered to irrelevant users. Existing methods are not enough to solve the problem of popularity bias in news recommendation, mainly because they ignore the time factor and the impact of news content on popularity. Although they increase the possibility of unpopular news being recommended, outdated news is usually less popular and will also be recommended to users, which may lead to unreasonable recommendations. To achieve more reasonable debiased recommendations, it is necessary to consider the time factor in the process of alleviating popularity bias. Summary of the invention

[0003] The technical problem to be solved by the present invention is to solve the problem that the prior art directly applies clustering algorithms to large-scale data sets, which may lead to high computational complexity and long time consumption.

[0004] The present invention adopts the following technical solution: a time- and content-aware news recommendation system popularity debiasing method, comprising the following steps:

[0005] (a) Preprocessing of news dataset:

[0006] Calculate the timeliness of each piece of news , as follows: News The earliest occurrence in the data set is , then news At a given moment The timeliness is , the click rate of each entity and each word in a news within one hour is taken as their popularity, respectively and Indicates that represents the moment of computation given to each entity, Indicates the time at which each word is calculated. Click-through rate refers to the ratio of the number of clicks on an entity or word to the number of impressions within a specified hour.

[0007] (b) Constructing the TCCM model:

[0008] For the construction of the time module, the time embedding layer is used to first convert the quantitative value representing the timeliness of the news Convert to embedding vector , then, using dense network processing And predict a time score , the conversion formula is: ,in , represents the input features, represents the weight matrix, Indicates the transposition of it. is the offset;

[0009] The construction of the popularity module includes the following steps: S1 converts the popularity of each entity and word into a vector and as the model input; S2 uses a multi-head self-attention network to capture the relevance of entities to learn the popularity representation of each entity; uses a multi-head self-attention network to capture the relevance of words to learn the popularity representation of each word; S3 uses an entity multi-head cross-attention network to learn the popularity representation of each entity from the news context; uses a word multi-head cross-attention network to learn the popularity representation of each word from the news context; S4 formulates the unified representation of the popularity of each entity and word as the sum of its representations learned by the multi-head attention network and the multi-head cross-attention network; S5 uses the attention network to learn the entity-based news popularity representation and the word-based news representation from the unified representation of the popularity of each entity and word described in step S4; S6 obtains a unified news popularity representation by learning a weighted combination of the entity-based news popularity representation and the word-based news representation through the attention network; S7 quantizes the representation of news popularity using a fully connected network and divides it by the quantified news timeliness to obtain the popularity of the entire news;

[0010] For the user-content matching module, the inner product of user and news embeddings is used to calculate the matching score. ,in, and denote the embeddings of users and news respectively, Indicates time;

[0011] The influence of the three factors of time, popularity and user preference are integrated through addition calculation to obtain the TCCM model, which is expressed as follows: ,in represents the user's preference for news timeliness and news popularity, and is a randomly initialized training parameter matrix. Indicates the popularity tendency. Indicates the degree of time tendency, Indicates the degree to which a user clicks on a piece of news, influenced by the timeliness and popularity of the news;

[0012] (c) Perform model training:

[0013] The model is trained using the data set and the BPR loss function is used to improve the model's ability to understand the characteristics of positive samples by maximally distinguishing between positive and negative samples. The calculation method is:

[0014] in, and Respectively represent The interactive scores of the positive and negative samples of the news, represents the training data set, Represents the activation function named sigmoid. The expression of sigmoid activation function is: , is a natural constant, is the input variable;

[0015] (d) Perform causal intervention in the inference stage to alleviate popularity bias: In the inference stage, the popularity values ​​of news entities and words are controlled and set to be lower than the average value to reduce the impact of popularity and alleviate popularity bias.

[0016] Preferably, the popularity module in the TCCM model is constructed using the following formula for calculation:

[0017] , , , , ,

[0018] ,

[0019] ;

[0020] in, , , The expression is:

[0021]

[0022]

[0023]

[0024]

[0025] represents the input features, represents the weight matrix, is the offset, calculate and Similarity; Indicates that the values ​​in the brackets are normalized; is the input vector or hidden layer dimension; Concat is the concatenation function; Q, K, and V represent the query vector, key vector, and value vector, respectively.

[0026] The present invention has the following beneficial effects: The present invention proposes a new model, named Time and Content Aware Causal Model (TCCM). The model models the influence of three factors on user interaction behavior, namely, the timeliness of news, the popularity of news, and the matching degree of news content with user interests. In addition, TCCM applies causal intervention technology to control popularity variables in the inference stage to obtain debiased recommendations. To better alleviate the popularity bias of news recommendation systems, accurate news popularity estimation is crucial. To this end, the present invention proposes a new popularity estimation method. The method takes into account the influence of news content on popularity and more accurately estimates news popularity by combining news content (popularity of entities and words). DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the technical scheme in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0028] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like to indicate directions or positional relationships are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0029] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0031] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0032] Example

[0033] The following are only preferred implementations of the present invention. The protection scope of the present invention is not limited to the following embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention.

[0034] A time- and content-aware popularity debiasing method for a news recommendation system includes the following steps:

[0035] (a) Preprocessing of news dataset:

[0036] Calculate the timeliness of each piece of news , as follows: News The earliest occurrence in the data set is , then news At a given moment The timeliness is , the click rate of each entity and each word in a news within one hour is taken as their popularity, respectively and Indicates that represents the moment of computation given to each entity, Indicates the time at which each word is calculated. Click-through rate refers to the ratio of the number of clicks on an entity or word to the number of impressions within a specified hour.

[0037] (b) Constructing the TCCM model:

[0038] For the construction of the time module, the time embedding layer is used to first convert the quantitative value representing the timeliness of the news Convert to embedding vector , then, using dense network processing And predict a time score , the conversion formula is: ;in , represents the input features, represents the weight matrix, Indicates the transposition of it. is the offset.

[0039] Considering that earlier published news articles are less likely to attract users, the present invention also calculates The inverse of , and use the parameter λ to adjust its intensity. The specific formula is The inverse of , and use the parameter λ to adjust its intensity. The specific process is as follows:

[0040]

[0041] The parameter λ takes a value between 2 and 2.5.

[0042] The construction of the popularity module includes the following steps: S1 Convert the popularity of each entity and word into a vector and as the model input; S2 uses a multi-head self-attention network to capture the relevance of entities to learn the popularity representation of each entity; uses a multi-head self-attention network to capture the relevance of words to learn the popularity representation of each word; S3 uses an entity multi-head cross-attention network to learn the popularity representation of each entity from the news context; uses a word multi-head cross-attention network to learn the popularity representation of each word from the news context; S4 formulates the unified representation of the popularity of each entity and word as the sum of its representations learned by the attention network and the cross-attention network; S5 uses the attention network to learn the entity-based news popularity representation and the word-based news representation from the unified representation of the popularity of each entity and word described in step S4; S6 obtains a unified news popularity representation by learning a weighted combination of the entity-based news popularity representation and the word-based news representation through the attention network; S7 quantizes the representation of news popularity using a fully connected network and divides it by the quantified news timeliness to obtain the popularity of the entire news; the formula is as follows: , , , , ,

[0043] ,

[0044]

[0045] in, , , The specific expression is as follows:

[0046]

[0047]

[0048]

[0049]

[0050] represents the input features, represents the weight matrix, is the offset, calculate and Similarity; Perform normalization operation; It is the input vector or hidden layer dimension. Concat is the concatenation function; Q, K, and V represent the query vector, key vector, and value vector respectively. Query represents the information we want to find. It is a vector that usually represents the representation of an element in the current input sequence. The Query vector is used to calculate the similarity with the Key vector to determine which input information is most important for the current element. Value represents the content of the specific input information. The calculated attention weight (based on the similarity between Query and Key) is applied to the Value vector to generate a weighted output. The Value vector usually has the same dimension as the Key vector, but represents the actual content of the input data.

[0051] A multi-head self-attention network (MHSA) is used to capture the connection between different entities. A multi-head cross-self-attention network (MHCA) is used to learn the impact of the connection between entities and context on popularity. The results obtained by the MHSA and MHCA networks are summed to obtain the popularity representation of each entity after the two influences. The attention network is used to integrate the popularity of all news entities in a news article to obtain an overall entity popularity representation.

[0052] Similarly, MHSA is used to capture the connection between different entities. MHCA is used to learn the influence of the connection between words and context on popularity. The results obtained by MHSA and MHCA networks are summed to obtain the popularity representation of each word after the two influences. The attention network is used to integrate the popularity of all news words in a news article to obtain an overall entity popularity representation.

[0053] For the user-content matching module, the inner product of user and news embeddings is used to calculate the matching score. ,in, and denote the embeddings of users and news respectively, Indicates time;

[0054] The influence of the three factors of time, popularity and user preference are integrated through addition calculation to obtain the TCCM model, which is expressed as follows: ,in Indicates the user's preference for news timeliness and news popularity. Indicates the popularity tendency. Indicates the degree of time tendency, Indicates the degree to which a user clicks on a piece of news, influenced by the timeliness and popularity of the news;

[0055] (c) Perform model training:

[0056] The model is trained using the dataset and the BPR loss function is used. Its purpose is to improve the model's ability to understand the characteristics of positive samples by maximally distinguishing between positive and negative samples. The calculation method is as follows:

[0057]

[0058] in, and Respectively represent The interactive scores of the positive and negative samples of the news, represents the training data set, Represents the activation function named sigmoid; the sigmoid activation function expression is: , is a natural constant, is the input variable;

[0059] (d) Perform causal intervention in the inference stage to alleviate popularity bias: In the inference stage, the popularity values ​​of news entities and words are controlled and set to be lower than the average value to reduce the impact of popularity and alleviate popularity bias.

[0060] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A time- and content-aware news recommendation system popularity debiasing method, characterized in that: The steps include: (a) Preprocessing of news dataset: Calculate the timeliness of each piece of news , as follows: News The earliest occurrence in the data set is , then news At a given moment The timeliness is , the click rate of each entity and each word in a news article within one hour is taken as their popularity, respectively and Indicates that represents the moment of computation given to each entity, Indicates the time at which each word is calculated. Click-through rate refers to the ratio of clicks on an entity or word to impressions within a specified hour. (b) Constructing the TCCM model: For the construction of the time module, the time embedding layer is used to first convert the quantitative value representing the timeliness of the news Convert to embedding vector , then, using dense network processing And predict a time score , the conversion formula is: ,in , represents the input features, represents the weight matrix, Indicates the transposition of it. is the offset; The construction of the popularity module includes the following steps: S1 Convert the popularity of each entity and word into a vector and as module input; S2 uses a multi-head self-attention network to capture the relevance of entities to learn the popularity representation of each entity; uses a multi-head self-attention network to capture the relevance of words to learn the popularity representation of each word; S3 uses an entity multi-head cross-attention network to learn the popularity representation of each entity from the news context; uses a word multi-head cross-attention network to learn the popularity representation of each word from the news context; S4 formulates the unified representation of the popularity of each entity and word as the sum of the popularity representations learned by the multi-head self-attention network and the multi-head cross-attention network; S5 uses the attention network to learn the entity-based news popularity representation and the word-based news representation from the unified representation of the popularity of each entity and word described in step S4; S6 obtains a unified news popularity representation by learning a weighted combination of the entity-based news popularity representation and the word-based news representation through the attention network; S7 quantizes the representation of news popularity using a fully connected network and divides it by the quantified news timeliness to obtain the popularity of the entire news; For the user-content matching module, the inner product of user and news embeddings is used to calculate the matching score. ,in, and denote the embeddings of users and news respectively, Indicates time; The influence of the three factors of time, popularity and user preference are integrated through addition calculation to obtain the TCCM model, which is expressed as: ,in represents the user's preference for news timeliness and news popularity, and is a randomly initialized training parameter matrix. Indicates the popularity tendency. Indicates the degree of time tendency, Indicates the degree to which a user clicks on a piece of news, influenced by the timeliness and popularity of the news; , the parameter λ takes a value between 2 and 2.5; (c) Perform model training: The model is trained using the data set and the BPR loss function is used to improve the model's ability to understand the characteristics of positive samples by maximally distinguishing between positive and negative samples. The calculation method is: in, and Respectively represent The interactive scores of the positive and negative samples of the news, represents the training data set, Represents the activation function named sigmoid. The expression of sigmoid activation function is: , is a natural constant, is the input variable; (d) Perform causal intervention in the inference stage to alleviate popularity bias: Control the popularity values ​​of news entities and words in the inference stage and set the popularity values ​​of the news entities and words to be lower than the average value, thereby reducing the impact of popularity and alleviating popularity bias.

2. The time- and content-aware news recommendation system popularity debiasing method according to claim 1, characterized in that: The popularity module in the TCCM model is calculated using the following formula: , , , , , , ; in, , , The expression is: represents the input features, represents the weight matrix, is the offset, calculate and Similarity; Indicates that the values ​​in the brackets are normalized; is the input vector or hidden layer dimension; Concat is the concatenation function; Q, K, and V represent the query vector, key vector, and value vector, respectively.