A cross-domain news recommendation system and method based on text entailment

Abstract

A cross-domain news recommendation system and method based on text entailment belong to the technical field of information recommendation. In order to solve the problem of cross-domain news recommendation. The present application includes a user domain interest generation module, a BERT-based news embedding module, a user scenario interest generation module based on an attention mechanism, an interest fusion module based on the idea of text entailment, a click rate prediction module, and a loss function calculation module. The BERT-based news embedding module is connected to the user scenario interest generation module based on the attention mechanism and the click rate prediction module. The user domain interest generation module and the user scenario interest generation module based on the attention mechanism are connected to the interest fusion module based on the idea of text entailment. The interest fusion module based on the idea of text entailment is connected to the click rate prediction module. The click rate prediction module is connected to the loss function calculation module. The present application is more effective in the news recommendation task.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of information recommendation, and particularly relates to a cross-domain news recommendation system and method based on text entailment. BACKGROUND

[0002] In today's world, with the popularization and development of Internet technology, everyone can easily obtain news information from the network. With more and more people using the Internet, there is a problem that when people face a large amount of information, they cannot effectively filter and process these information, resulting in waste of time and energy, and seriously affecting the work and life efficiency of people. In order to solve the problem of "information overload" under the mass of news, the personalized news recommendation system emerges as the times require. The personalized news recommendation system has accurate recommendation ability and efficient service, which can help users obtain the news they are interested in more quickly and accurately. In this way, users no longer need to spend a lot of time searching and screening news, thereby improving the efficiency and quality of users obtaining information.

[0003] Although the news recommendation system has been widely used in various news websites and platforms, there are still two long-standing obstacles that greatly limit the performance of the recommendation system, one is the data sparsity problem, and the other is the cold start problem. In the data sparsity problem, because news is a kind of transient information, and users also have constantly changing interests and needs, which leads to relatively less interaction between users and news. The cold start problem refers to when the recommendation system faces new users, due to the lack of historical behavior data of the user, the recommendation algorithm cannot understand the interests and preferences of the user, so as to be unable to make personalized recommendations. Similarly, when some new news articles are added to the recommendation system, due to the lack of enough user interaction data on the news, the recommendation algorithm cannot accurately capture the content and characteristics of the news, so it is difficult to recommend related news articles. Under the constraints of sparsity problem and cold start problem, the traditional news recommendation system can only rely on general recommendation strategies, such as popularity or random recommendation. Such recommendation results may have a large deviation from the interests and needs of users, thereby affecting the user experience and satisfaction.

[0004] With the continuous development of the information age, more and more users begin to interact with multiple domains, and this multi-domain information interaction provides a solution to the above two problems. That is, the cross-domain recommendation which has been very concerned in recent years. The goal of cross-domain recommendation is to use the information collected from other domains to alleviate the recommendation problem in another domain. In the research of cross-domain recommendation, due to different scenarios, there are different methods. Considering the symmetry of users and items, it can be roughly divided into three categories: user non-overlapping item non-overlapping, user partially overlapping item non-overlapping, and user completely overlapping item non-overlapping. The research of the present application faces the recommendation scene of user partially overlapping item non-overlapping.

[0005] In the recommendation scenario of partially overlapping items, there are some users who have interacted in both domains, while some users have only interacted in a single domain. In this recommendation scenario, there are mainly three research methods, including collective matrix factorization, combined overlapping user representation and information mapping.

[0006] In the collective matrix factorization method, the basic method is cross-domain matrix factorization, which uses the information of overlapping users to enhance the matrix factorization process. Some studies have proposed the XPTRANS method, which applies non-negative matrix factorization in two domains, and adds the consistency of overlapping users between domains as a constraint to the matrix factorization process. Some work also uses the similarity of inter-domain clusters and user similarity, and adds the similarity of the two similarity matrices as a regularization term to the matrix factorization process.

[0007] In the combined overlapping user representation method, in order to consider the overlapping user information in different domains at the same time, the fusion embedding is obtained by combining the overlapping user representations in different domains, and the unified embedding is used in different domains to realize the information transfer. Some studies have proposed a dual-target optimization framework DTCDR, which generates user and item embeddings in the source and target domains respectively, and uses three operations such as max-pooling to generate the representation of overlapping users. Some work additionally considers the topic distribution of the model in the user embedding, and obtains the representation of the overlapping user by mapping and fusing the user topic preference.

[0008] In the information mapping method, different domains are divided into source and target domains, and the core is to map the source domain overlapping user information to the target domain corresponding user embedding through a suitable mapping function, and use fusion method to enhance the user representation in the target domain. Some studies have proposed the EMCDR model and first applied this idea, which generates user and item embeddings based on matrix factorization and Bayesian personalized ranking method, and uses a multi-layer perception to learn the mapping function of overlapping users in the source and target domains to enhance the representation of cold-start users in the target domain. Some work has proposed a cross-domain latent feature mapping model (CDLFM), which considers three kinds of user similarity between domains as constraints in the matrix factorization process.

[0009] Overall, in the three methods of collective matrix factorization, combined overlapping user representation and information mapping, although the information of other domains can be used to enhance the representation of users in the current domain, the representation of users is relatively single, which cannot well reflect the differentiated feature representation of users in different domains, whether the same overlapping user embedding representation is used or the user representation obtained by the mapping function is used. SUMMARY

[0010] The present application solves the problem of cross-domain news recommendation reflecting the user's characteristic representation in different fields, and proposes a cross-domain news recommendation system and method based on text entailment.

[0011] To achieve the above-mentioned purpose, the present application realizes the following technical solutions:

[0012] A cross-domain news recommendation system based on text entailment includes a user domain interest generation module, a BERT-based news embedding module, a user scenario interest generation module based on an attention mechanism, an interest fusion module based on the idea of text entailment, a click rate prediction module, and a loss function calculation module.

[0013] The BERT-based news embedding module is connected to the user scenario interest generation module based on an attention mechanism and the click rate prediction module, the user domain interest generation module and the user scenario interest generation module based on an attention mechanism are connected to the interest fusion module based on the idea of text entailment, the interest fusion module based on the idea of text entailment is connected to the click rate prediction module, and the click rate prediction module is connected to the loss function calculation module.

[0014] The user domain interest generation module is used to fuse the user abstract interest vector and the domain adaptive feature vector to obtain the user's domain interest vector.

[0015] The BERT-based news embedding module is used to fuse the collaborative information vector and the semantic information vector of the news to obtain the fused news embedding vector.

[0016] The user scenario interest generation module based on an attention mechanism is used to induce the user interaction news embedding vector through the candidate news embedding vector to obtain the user scenario interest vector.

[0017] The interest fusion module based on the idea of text entailment fuses the obtained user's domain interest vector and user scenario interest vector to obtain the user multi-level interest vector.

[0018] The click rate prediction module is used to calculate the similarity between the user multi-level interest vector and the candidate news embedding vector to obtain the prediction value of the user's click rate on the candidate news.

[0019] The loss function calculation module uses the cross-entropy loss function to construct the loss function of the cross-domain news recommendation system based on text entailment, which is used to update the parameters of the cross-domain news recommendation system based on text entailment for system optimization.

[0020] A cross-domain news recommendation method based on text entailment is realized by relying on the cross-domain news recommendation system based on text entailment, which includes the following steps:

[0021] S1, the user domain interest generation module fuses the user abstract interest vector and the domain adaptive feature vector to obtain a user domain interest vector;

[0022] S2, the BERT-based news embedding module fuses the collaborative information vector and the semantic information vector of the news to obtain a fused news embedding vector;

[0023] S3, the method of step S2 processes the user interaction news and the candidate news to obtain a user interaction news embedding vector and a candidate news embedding vector, and a user scenario interest generation module based on an attention mechanism induces the user interaction news embedding vector through the candidate news embedding vector to obtain a user scenario interest vector;

[0024] S4, an interest fusion module based on text entailment idea fuses the user domain interest vector obtained in step S1 and the user scenario interest vector obtained in step S3 to obtain a user multi-level interest vector;

[0025] S5, a click rate prediction module is used to calculate the similarity between the user multi-level interest vector obtained in step S4 and the candidate news embedding vector obtained in step S3 to obtain a prediction value of the click rate of the user on the candidate news;

[0026] S6, a loss function calculation module constructs a loss function of the cross-domain news recommendation system based on text entailment by using a cross-entropy loss function, which is used to update the parameters of the cross-domain news recommendation system based on text entailment for system optimization.

[0027] Further, the specific implementation method of step S1 includes the following steps:

[0028] S1.1, collect news data to construct a cross-domain news recommendation data set, including a source domain news data set and a target domain news data set, for the users of the source domain news data set and the target domain news data set, first construct a user target domain adaptive feature embedding matrix User source domain adaptive feature embedding matrix User target domain abstract interest embedding matrix User source domain abstract interest embedding matrix Wherein The user set of the target domain news data set and the user set of the source domain news data set, is the user set of the source domain news data set and the target domain news data set, and d is the dimension of the hidden vector;

[0029] S1.2, obtain the user target domain adaptive feature vector in a lookup table manner User source domain adaptive feature vector User target domain abstract interest embedding vector User source domain abstract interest embedding vector

[0030] Then the user abstract interest vector and the adaptive feature vector are fused, and the expression is calculated as:

[0031]

[0032]

[0033] Wherein, is the user's source domain field interest vector, is the user's target domain field interest vector, e a is the user's field interest vector.

[0034] Further, in step S1, by introducing the loss function, the Euclidean distance of the user target domain abstract interest embedding vector and the user source domain abstract interest embedding vector is calculated, so as to minimize the difference between the two, and the negative number of the Euclidean distance of the user target domain adaptive feature vector and the user source domain adaptive feature vector is calculated, so as to maximize the difference between the two, and the expression is calculated as:

[0035]

[0036]

[0037] L con =L close +L diff

[0038] Wherein, L close is the loss function for making user interest close, L diff is the loss function for making field adaptive feature far away, and L con is the sum of the loss function for making user interest close and the loss function for making field adaptive feature far away.

[0039] Further, the specific implementation method of step S2 includes the following steps:

[0040] S2.1, initialize news embedding matrix, and obtain the news collaborative information embedding vector e n by table lookup method;

[0041] S2.2, data cleaning is performed on the multi-domain news text information, and then the multi-domain news text information after data cleaning is processed by using the text content embedding method of Chinese pre-training language model BERT, each news text is intercepted to a fixed length and added with padding operation, and input into Chinese BERT model for semantic understanding of text content, and the semantic embedding vector of news is obtained

[0042] S2.3, splice the semantic embedding vector of the news obtained in step S2.2 into the collaborative information embedding vector of the news obtained in step S2.1, and then input into a multi-layer full connection network to obtain a fusion news embedding vector e f The calculation expression is:

[0043]

[0044] wherein e f is the fusion news embedding vector, [;] is a splicing operation, MLP is a multi-layer full connection network, and the activation function uses Relu.

[0045] Further, the specific implementation method of step S3 includes the following steps:

[0046] S3.1, set the interactive news sequence of the user as H={N1, N2,...N i ,...,N k}, wherein N i is the i-th interactive news of the user, k is the length of the interactive news sequence of the user, set the candidate news to be predicted as N c , initialize the news embedding matrix of the source domain and the target domain wherein is the set of all news of the source domain and the target domain;

[0047] S3.2, process the user interactive news and the candidate news in step S3.1 by using the method of step S2 to obtain a user interactive news embedding sequence and a candidate news embedding vector wherein is the i-th interactive news embedding vector of the user;

[0048] S3.3, the user scenario interest generation module based on the attention mechanism induces the user interactive news embedding vector through the candidate news embedding vector, and obtains a user scenario interest vector e d The calculation expression is:

[0049]

[0050] wherein a i is the contribution degree of the i-th interactive news to the user scenario interest;

[0051] The calculation expression of a i is:

[0052]

[0053] a = softmax(Mask(a1, a2,..., an) k ))

[0054] where W e R d×d is a learnable parameter matrix, b e R d is a learnable parameter vector, Mask is a mask operation for padding in user history, a is a weight vector of each interactive news of the user;

[0055] Suppose the mask vector m is m e {0, 1} d , where 0 represents the news without user interaction, and the calculation expression of the Mask operation is:

[0056] m = (1 - m) x (-inf)

[0057]

[0058] where ε represents a minimum constant to prevent division by zero error for the user without interactive news.

[0059] Further, the specific implementation method of step S4 includes the following steps:

[0060] S4.1, the interest fusion module based on the idea of text entailment fuses the user's domain interest vector obtained in step S1 and the user's scenario interest vector obtained in step S3 to obtain a user multi-level interest vector e b , and the calculation expression is:

[0061] e b = e a + e d

[0062] Or the interest fusion module based on the idea of text entailment fuses the user's domain interest vector obtained in step S1 and the user's scenario interest vector obtained in step S3 by using a splicing operation to obtain a user multi-level interest vector e b , and the calculation expression is:

[0063] e b = [e a + e d ; e a - e d ; e a * e d ];

[0064] S4.2, the text inclusion relationship between the user's field interest vector obtained by step S4.1 and the user's scene interest vector, establish a text inclusion judgment method, take the label of the user's click on the candidate news as the label of whether the text includes, judge the inclusion relationship between the user's field interest vector and the user's scene interest vector through a multi-layer fully connected network, take the subtraction operation as the default text inclusion combination method, and the calculation expression is:

[0065] e b =e a -e d

[0066]

[0067]

[0068] Wherein, L entailment is the text inclusion loss, S is the source domain or target domain training sample, is the predicted label of the text inclusion relationship, u is the user, v i is the news clicked by the user, is the news not clicked by the user.

[0069] Further, the specific implementation method of step S5 is to use a multi-layer feedforward network to calculate the similarity between the user multi-level interest vector obtained by step S4 and the candidate news embedding vector obtained by step S3, and the calculation expression is:

[0070] V=Sigmoid(MLP([e b ;e fc ]))

[0071] Wherein, V is the predicted value of the user's click rate on the current candidate news, MLP represents a multi-layer feedforward network, the activation function uses Relu, and Sigmoid is an activation function.

[0072] Further, the specific implementation method of step S6 includes the following steps:

[0073] S6.1, the loss function calculation module uses the cross entropy loss function to construct the loss function of the click rate prediction module, and the calculation expression is:

[0074]

[0075]

[0076]

[0077] Wherein, S is the training set, including the positive and negative example interaction records of the user on the news, u is the current user, v jpositive example candidate news that has been interacted by a user, negative example candidate news that has not been interacted by a user, a loss function calculation for a target domain, a loss calculation for a source domain, Lpred is a loss function calculation common to the source domain and the target domain;

[0078] S6.2, constructing an L2 regularization loss L nom The calculation expression is:

[0079]

[0080] wherein, theta is all learnable parameters of the model;

[0081] S6.3, based on the loss functions of steps S6.1 and S6.2, obtaining the loss function L of the cross-domain news recommendation system based on text entailment, and the calculation expression is:

[0082] L=L pred +lambda norm L norm +lambda con L con +lambda entailment L entailment

[0083] wherein, lambda norm is a penalty coefficient of the L2 regularization loss, lambda con is a loss coefficient of L con , and lambda entailment is a loss coefficient of L entailment .

[0084] Advantages of the present application:

[0085] The cross-domain news recommendation method based on text entailment proposes a hierarchical representation method of user interest. The method divides the user interest into three levels: the abstract interest at the bottom, the domain interest generated by the abstract interest through domain adaptation (using domain adaptation features), and the scenario interest generated by the candidate news induced history record. This hierarchical representation method can better depict the user interest.

[0086] The cross-domain news recommendation method based on text entailment proposes a cross-domain user interest modeling method based on text entailment. The domain interest and the scenario interest in the first point constitute an entailment relationship, the information fusion method of text entailment is introduced into the interest fusion process, and the determination of the entailment relationship is taken as an auxiliary task of the cross-domain recommendation system. Through experiments, a suitable fusion method is selected, and the recommendation effect is improved.

[0087] The cross-domain news recommendation method based on text entailment provided in the application proposes a cross-domain news data set construction method based on a social network.

[0088] The cross-domain news recommendation method based on text entailment provided in the application proposes a cross-domain news recommendation model based on text entailment, through the introduction of multi-level interests such as user abstract interest, field interest and scene interest, and the interest fusion by using the idea of text entailment, the recommendation effect of target domain news can be well enhanced by source domain news information. The experimental results show that, compared with the effect of single-domain news recommendation, the cross-domain news recommendation model based on text entailment can achieve good effect on the target domain data set, and the effect is also more obviously improved in the case of introducing a news content understanding module in the news recommendation task. BRIEF DESCRIPTION OF DRAWINGS

[0089] Figure 1 The structure schematic diagram of the cross-domain news recommendation system based on text entailment provided in the application is shown in the figure.

[0090] Figure 2 The flowchart of the cross-domain news recommendation method based on text entailment provided in the application is shown in the figure. DETAILED DESCRIPTION

[0091] In order to make the purpose, technical scheme and advantages of the application more clear, the application is further described in detail below in combination with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the application, and are not used to limit the application, that is, the described specific embodiments are only a part of the embodiments of the application, but not all the specific embodiments. The components of the specific embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations, and the application can also have other embodiments.

[0092] Therefore, the detailed description of the specific embodiments of the application provided in the drawings below is not intended to limit the scope of the claimed application, but only represents selected specific embodiments of the application. Based on the specific embodiments of the application, all other specific embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0093] In order to further understand the invention content, characteristics and effects of the application, the following specific embodiments are exemplified, and the drawings are Figure 1 and the drawings Figure 2 are described as follows: Specific embodiment one:

[0095] A cross-domain news recommendation system based on text entailment, comprising a user field interest generation module 1, a BERT-based news embedding module 2, a user scenario interest generation module 3 based on an attention mechanism, an interest fusion module 4 based on the idea of text entailment, a click rate prediction module 5, and a loss function calculation module 6.

[0096] The BERT-based news embedding module 2 is connected to the user scenario interest generation module 3 based on an attention mechanism and the click rate prediction module 5, the user field interest generation module 1 and the user scenario interest generation module 3 based on an attention mechanism are connected to the interest fusion module 4 based on the idea of text entailment, the interest fusion module 4 based on the idea of text entailment is connected to the click rate prediction module 5, and the click rate prediction module 5 is connected to the loss function calculation module 6.

[0097] The user field interest generation module 1 is used for fusing a user abstract interest vector and a field adaptation feature vector to obtain a user field interest vector.

[0098] The BERT-based news embedding module 2 is used for fusing a news collaborative information vector and a semantic information vector to obtain a fused news embedding vector.

[0099] The user scenario interest generation module 3 based on an attention mechanism is used for inducing a user interactive news embedding vector through a candidate news embedding vector to obtain a user scenario interest vector.

[0100] The interest fusion module 4 based on the idea of text entailment fuses the obtained user field interest vector and user scenario interest vector to obtain a user multi-level interest vector.

[0101] The click rate prediction module 5 is used for calculating the similarity between the user multi-level interest vector and the candidate news embedding vector to obtain a prediction value of the click rate of the user to the candidate news.

[0102] The loss function calculation module 6 constructs a loss function of the cross-domain news recommendation system based on text entailment by using a cross-entropy loss function, and is used for updating the parameters of the cross-domain news recommendation system based on text entailment for system optimization. Specific implementation method two:

[0104] A cross-domain news recommendation method based on text entailment is implemented by relying on the cross-domain news recommendation system based on text entailment in the specific implementation method one, and comprises the following steps:

[0105] S1, the user field interest generation module fuses a user abstract interest vector and a field adaptation feature vector to obtain a user field interest vector.

[0106] In the recommendation process, the interest preferences of users in different fields often need to be considered. Specifically, when users browse news in the source domain and the target domain, the clicked news is often influenced by different fields, showing different emphases. Therefore, it can be considered that the same user has different field preference interests in the source domain and the target domain. However, in addition to the field preference, the user also has an interest preference for news, which is not affected by the field and reflects the user's personal constant news interest. Therefore, when making news recommendations, the user's constant news tendency and reading preferences in different fields need to be considered comprehensively to more accurately model the user's interest.

[0107] The following example illustrates the user's interest preferences in different fields. Suppose user A is interested in political news, which can be considered as the user's own interest preference. However, when user A browses different news domains, he may exhibit different click interests. For example, when browsing Renmin Daily news, since Renmin Daily pays more attention to domestic news, the user may expect to learn more about domestic political events such as government policies. While browsing Voice of America news, since Voice of America reports more American news, the user may want to learn more about American political events such as presidential elections and foreign relations. This shows that the user's interest preferences in different fields are different, and recommendations need to be made for different fields to better meet the user's needs.

[0108] Based on the above assumptions, the user's relatively stable interest preference in field news recommendation, i.e., the user's abstract interest, is introduced. This interest does not change with the change of news field and reflects the user's constant news reading interest. At the same time, in order to capture the user's unique interest in different fields, a differentiated field embedding vector, i.e., a field adaptation feature, is additionally set in the source domain and the target domain to reflect the user's interest preferences in different news fields. Based on the user's abstract interest and the field adaptation feature, the user's specific field interest in the source domain or the target domain can be obtained by combining the two.

[0109] The user's abstract interest reflects the user's personal interest preference that is not affected by the field, so the user's abstract interest in the source domain and the target domain is as close as possible, while the user's field adaptation feature reflects the interest preference in different news fields. On the basis of the abstract interest, the field adaptation feature should as much as possible embody the information specific to the field, so the field adaptation features in the source domain and the target domain can as much as possible be far away to contain field differentiated information.

[0110] Further, the specific implementation method of step S1 includes the following steps:

[0111] S1.1, collect news data to build cross-domain news recommendation data set, including source domain news data set, target domain news data set, for the user of the source domain news data set and the target domain news data set, first, construct the user target domain adaptive feature embedding matrix User source domain adaptive feature embedding matrix User target domain abstract interest embedding matrix User source domain abstract interest embedding matrix Wherein User set of target domain news data set, user set of source domain news data set, All user sets of source domain news data set and target domain news data set, d is the dimension of the hidden vector;

[0112] S1.2, get the user target domain adaptive feature vector by lookup table User source domain adaptive feature vector User target domain abstract interest embedding vector User source domain abstract interest embedding vector

[0113] Then, the abstract interest vector and the adaptive feature vector of the user are fused, and the expression is calculated as:

[0114]

[0115]

[0116] Wherein, User source domain interest vector, User target domain interest vector, e a User domain interest vector;

[0117] Further, in step S1, by introducing the loss function, the Euclidean distance of the user target domain abstract interest embedding vector and the user source domain abstract interest embedding vector is calculated, so as to minimize the difference between the two, and the negative number of the Euclidean distance of the user target domain adaptive feature vector and the user source domain adaptive feature vector is calculated, so as to maximize the difference between the two, and the expression is calculated as:

[0118]

[0119]

[0120] L con =L close +L diff

[0121] Wherein, L close Loss function for making user interest close, Ldiff Ldis is a loss function for moving the domain adaptation features away con Ldis is a loss function for moving the domain adaptation features away

[0122] S2, a BERT-based news embedding module fuses the collaborative information vector and the semantic information vector of the news to obtain a fused news embedding vector;

[0123] In news recommendation, the news itself contains rich text, and such semantic information can well explore the semantic association between news, thereby improving the recommendation effect. Therefore, in this part, in addition to the news collaborative information, the news semantic information is additionally added, and the model can better perform news modeling by fusing the news collaborative information and the news semantic information.

[0124] Further, the specific implementation method of step S2 includes the following steps:

[0125] S2.1, initializing a news embedding matrix to obtain a collaborative information embedding vector e n of the news through table lookup;

[0126] S2.2, performing data cleaning on the multi-domain news text information, and then processing the data cleaned multi-domain news text information by using a text content embedding method of a Chinese pre-training language model BERT, performing fixed length cutting and padding operation on each news text, and inputting the Chinese BERT model to perform semantic understanding of the text content, to obtain a semantic embedding vector

[0127] S2.3, splicing the corresponding semantic embedding vector of the news obtained in step S2.2 into the collaborative information embedding vector of the news obtained in step S2.1, and then inputting to a multi-layer fully connected network to obtain a fused news embedding vector e f , and the calculation expression is:

[0128]

[0129] wherein, e f is the fused news embedding vector, [;] is the splicing operation, MLP is the multi-layer fully connected network, and the activation function uses Relu;

[0130] S3, the method of step S2 processes the user interaction news and the candidate news to obtain a user interaction news embedding vector and a candidate news embedding vector, and a user scenario interest generation module based on an attention mechanism induces the user interaction news embedding vector through the candidate news embedding vector to obtain a user scenario interest vector;

[0131] In the previous implementation, user-specific abstract interests and domain-adaptive features were generated, and the current domain interest vector was obtained by combining the two. Although both of these interest vectors can effectively characterize user interests, more granular user interest modeling can still be performed at the news level. This means incorporating user-interactive news into the user interest modeling process to obtain the user's contextual interests in the current recommendation scenario. In this process, an attention mechanism is employed. The user's historical interactive news and candidate news to be recommended are used as input. The attention mechanism calculates the importance weight of each interactive news item to the user's current contextual interests. Then, all user-interactive news vectors are weighted and summed according to these weights to obtain the user's contextual interest vector in the current recommendation scenario.

[0132] Furthermore, the specific implementation method of step S3 includes the following steps:

[0133] S3.1 Set the user's interactive news sequence as H = {N1, N2, ... N} i , ..., N k}, where N i Let N be the i-th interactive news item for the user, k be the length of the user's interactive news sequence, and N be the candidate news items to be predicted. c Initialize the embedding matrices of all news articles in the source and target domains. in This is a collection of all news items from both the source and target domains.

[0134] Furthermore, the source domain and the target domain do not have the same news items;

[0135] S3.2. Using the method in step S2, process the user interaction news and candidate news from step S3.1 to obtain the user interaction news embedding sequence. and candidate news embedding vectors in Embed a vector for the user's i-th interactive news article;

[0136] S3.3 The user context interest generation module based on the attention mechanism induces the user interaction news embedding vector through the candidate news embedding vector, and obtains the user context interest vector e by weighted averaging the user interaction news embedding vectors. d The calculation expression is:

[0137]

[0138] Where, α i The degree to which the i-th interactive news article contributes to the user's contextual interest;

[0139] α i The calculation expression is:

[0140]

[0141] a = softmax(Mask(a1, a2,..., a k ))

[0142] where W e R d×d is a learnable parameter matrix, b e R d is a learnable parameter vector, Mask is a mask operation for padding in user history, a is the weight vector of each interactive news of the user; the purpose is to eliminate the influence of padding on user scenario interest modeling;

[0143] Assume that the mask vector m is m e {0, 1} d , where 0 represents news without user interaction, and the calculation expression of the Mask operation is:

[0144] m = (1 - m) x (-inf)

[0145]

[0146] where e represents a small constant to prevent division by zero error for news without user interaction;

[0147] It can be seen that the scenario interest not only considers the interactive news of the user, but also considers the current candidate news, which can well reflect the dynamic interest of the user. Specifically, the abstract interest of the user reflects the personal news reading tendency of the user, the domain interest reflects the interest preference of the user in a certain specific domain, and the scenario interest reflects the interest degree of the user in the current recommended scenario for the candidate news. If the candidate news has high similarity with some news that the user has interacted with, it can be considered that the candidate news is likely to attract the user's attention to the information, so as to accurately predict the click rate. Therefore, compared with the fixed interest and the domain-specific interest of the user, the scenario interest can better reflect the dynamic interest of the user in different recommended scenarios, thereby better modeling the user interest.

[0148] S4, the interest fusion module based on the idea of text entailment fuses the domain interest vector of the user obtained in step S1 and the scenario interest vector of the user obtained in step S3 to obtain a multi-level interest vector of the user;

[0149] The user domain interest and the scenario interest are obtained through the user domain interest generation module and the user scenario interest generation module based on the attention mechanism, respectively. The scenario interest is the interest of the user in the interactive news induced from the candidate news, which represents the interest of the user in the current scenario and is a kind of tendency of the user interest in the current domain. From the perspective of text entailment, the scenario interest can be considered to have an entailment relationship with the domain interest.

[0150] Further, the specific implementation method of step S4 includes the following steps:

[0151] S4.1, the interest fusion module based on the text implication idea fuses the user's field interest vector obtained in step S1 and the user's scenario interest vector obtained in step S3 to obtain a user multi-level interest vector e b The calculation expression is:

[0152] e b = e a + e d

[0153] Or the interest fusion module based on the text implication idea fuses the user's field interest vector obtained in step S1 and the user's scenario interest vector obtained in step S3 using splicing operation to obtain a user multi-level interest vector e b The calculation expression is:

[0154] e b = [e a + e d ; e a - e d ; e a * e d ];

[0155] Further, a plurality of interest fusion methods based on text implication are explored, and the appropriate interest fusion operation is selected through experimental results. The experimental results show that among all the fusion operations, the best effect is obtained by using the summation method. Therefore, the summation operation can be used as the default interest fusion method to improve the accuracy and effect of interest recommendation.

[0156] Further, the scenario interest induced by the candidate news has a text implication relationship with the field interest. If the current candidate news is the news clicked by the user, the field interest should contain the information of the scenario interest, and the two constitute an implication relationship. If the current candidate news is not clicked by the user, the scenario interest induced by the candidate news does not have an implication relationship with the field interest. Based on this idea, the label of the candidate news clicked by the user is regarded as the label of whether the text implies, and the field interest and the scenario interest are judged for the implication relationship through a multi-layer fully connected network.

[0157] S4.2, a text entailment relationship between the user's domain interest vector obtained by step S4.1 and the user's scenario interest vector is established, a text entailment determination method is established, a label of a user's click on a candidate news is regarded as a label of whether a text is entailed, the user's domain interest vector and the user's scenario interest vector are judged by a multi-layer full connection network for an entailment relationship, a subtraction operation is taken as a default text entailment combination mode, and the expression is calculated as follows:

[0158] e b a d

[0159]

[0160]

[0161] wherein L entailment is a text entailment loss, S is a source domain or target domain training sample, is a predicted label of a text entailment relationship, u is a user, v i is news clicked by the user, is news not clicked by the user;

[0162] S5, a click rate prediction module is used to calculate a similarity between the user multi-level interest vector obtained by step S4 and the candidate news embedding vector obtained by step S3, and a prediction value of a user's click rate on a candidate news is obtained.

[0163] For the obtained user multi-level interest vector e b and the candidate news vector e c , the similarity between the candidate news and the user multi-level interest is used to judge whether the user will click the current candidate news, that is, the prediction of the user's click rate on the candidate news is needed. In this part, a multi-layer feedforward network is used to comprehensively consider the relationship between the user multi-level interest and the candidate news to calculate the click rate.

[0164] Further, the specific implementation method of step S5 is to use a multi-layer feedforward network to calculate the similarity between the user multi-level interest vector obtained by step S4 and the candidate news embedding vector obtained by step S3, and the calculation expression is as follows:

[0165] V=Sigmoid(MLP([e b ;e fc ])

[0166] wherein V is a prediction value of a user's click rate on a current candidate news, MLP represents a multi-layer feedforward network, an activation function uses Relu, and Sigmoid is an activation function.

[0167] ​​Further, in predicting the click rate of the source domain and the target domain, the multi-layer feedforward network shares the same parameters. By this operation, the source domain recommendation is regarded as an auxiliary task of the target domain recommendation, so that the source domain and the target domain can learn the parameters in the network together, so that the auxiliary task (i.e. the source domain recommendation) can provide additional information to enhance the effect of the target domain recommendation.

[0168] S6, the loss function calculation module uses the cross-entropy loss function to construct the loss function of the cross-domain news recommendation system based on text entailment, which is used to update the parameters of the cross-domain news recommendation system based on text entailment for system optimization;

[0169] Further, the specific implementation method of step S6 includes the following steps:

[0170] S6.1, the loss function calculation module uses the cross-entropy loss function to construct the loss function of the click rate prediction module, and the calculation expression is:

[0171]

[0172]

[0173]

[0174] Wherein, S is the training set, including the positive and negative example interaction records of users to news, u is the current user, v i is the positive example candidate news interacted by the user, is the negative example candidate news not interacted by the user, is the loss function calculation for the target domain, is the loss calculation for the source domain, Lpred is the loss function calculation common to the source domain and the target domain;

[0175] S6.2, the L2 regularization loss L norm is constructed, and the calculation expression is:

[0176]

[0177] Wherein, θ is all the learnable parameters of the model; the model tends to use smaller weight values, which can prevent the weight values from being too large, thereby alleviating the overfitting phenomenon;

[0178] S6.3, based on the loss functions of steps S6.1 and S6.2, the loss function L of the cross-domain news recommendation system based on text entailment is obtained, and the calculation expression is:

[0179] L=L pred +λ norm L norm +λcon L con +λ entailment L entailmment

[0180] wherein λ norm is a penalty coefficient of the L2 regular loss, λ con is a loss coefficient of the L con loss, λ entailmment is a loss coefficient of the L entailmment loss.

[0181] Based on the first and second embodiments, specific experimental effects are verified and compared, and the specific method is as follows:

[0182] I. Constructing experimental data set: In the published data set, the number of cross-domain news recommendation data set is very small, and the cross-domain news recommendation data set in Chinese is also difficult to obtain. In many cross-domain recommendation works, more data is crawled from douban, twitter and other websites to construct cross-domain recommendation data set. Therefore, in this part of the work, the twitter data set is innovatively used to construct the cross-domain news recommendation data set.

[0183] Many news official media are registered in twitter. These news official media will publish news on the official website of twitter for users to read. Users will also express their preferences for the news through three interactions of like, forward and comment in twitter. In the specific construction of cross-domain news recommendation data set, the news official media in twitter data set is regarded as the news platform, and the like, comment and forward behaviors of users are regarded as the interaction of news, so as to construct the news data set corresponding to the official media platform. Through the cross-domain interaction of multiple different official media platform data sets, the cross-domain recommendation data set is constructed.

[0184] In order to obtain the twitter official media user information, the crawler is used to automatically crawl the relevant tweet information from the twitter official website, including the text content and publishing time of the tweet, the statistical information and user information of like, comment and forward, the specific time of like, comment and forward of users, the basic information of official media users and the like. The data between various official media users is relatively dense and of good quality. Taking the data of people's daily and xinhuanet as an example, as shown in table 1:

[0185] Table 1 comparison of people's daily-new xinhua data

[0186]

[0187] The People's Daily dataset contains 25,000 users and 3,200 tweets, with an average of 8.7 news interactions per user. The Xinhua News Agency dataset contains 7,850 users and 2,000 tweets, with an average of 6.4 news interactions per user. The overlap between the two datasets is 3,500 users, with an average of 33 interactions in the People's Daily and 11 interactions in the Xinhua News Agency. This suggests that the interaction information is more dense in the cross-domain context.

[0188] II. Parameter setting for a cross-domain news recommendation system based on textual entailment: The TECDR (Textual Entailment Based Cross Domain News Recommendation) system is implemented using a deep learning framework based on Pytorch. For the implemented model, the learning rate is set to 0.001, the batch size is 512, and the dimensions of the user embedding matrix and the news embedding matrix are both 128. During training, for each positive example interaction, four news articles are randomly selected as negative example interactions.

[0189] III. Evaluation metrics: In terms of evaluation, the leave-one-out method is used, which means that for each positive example in the validation set, 99 news articles are randomly selected as negative examples. The HR@10 and NDCG@10 metrics are used to evaluate the recommendation results.

[0190] IV. Comparison methods:

[0191] MLP_MF: MlpMF is a recommendation method based on matrix factorization. It combines the user and item latent vectors and then uses a fully connected layer to predict the click rate. This method can be applied to single-domain recommendation. To adapt the model to cross-domain recommendation, the interaction matrices of the source and target domains are combined to form a dual-domain interaction matrix, which can then be used with the single-domain recommendation model.

[0192] DTCDR: DTCDR is a dual-target optimization cross-domain recommendation model. It combines the user interests of the source and target domains through max-pooling and then optimizes the source and target domains separately to enhance the recommendation results of both domains. In the implementation process, the user review modeling module is ignored due to the lack of user review information.

[0193] V. Experimental Effect Comparison:

[0194] Tables 2 and 3 show the effects of each model on the single-domain Xinhua News data set and the cross-domain People's Daily-Xinhua News data set, where People's Daily is the source domain and Xinhua News is the target domain. The comparison effect is on the Xinhua News data set in the target domain. In Table 2, the MLP_MF method tests its effect on single-domain and cross-domain, while the DTCDR and TECDR models are tested on the cross-domain data set. The first column in the table is the model name, and +BERT represents the additional introduction of BERT-based news content embedding in the news embedding layer. The evaluation indicators use HR@10 and NDCG@10. Table 2 shows the experimental results of the models in HR@10, and Table 3 shows the experimental results of the models in NDCG@10. The best effect is marked in bold in the table. Based on the current experimental results, the following three conclusions can be drawn:

[0195] Table 2 Comparison of effects of each recommendation model under the index HR@10

[0196]

[0197]

[0198] Table 3 Comparison of effects of each recommendation model under the index NDCG@10

[0199]

[0200] 1) The effect of cross-domain recommendation is better than that of single-domain recommendation: On the Xinhua News single-domain data set, the recommendation effect of the matrix factorization-based method MLP_MF is HR 0.375, NDCG 0.191, but after introducing the People's Daily data set, the MLP_MF model effect reaches HR 0.394, NDCG 0.205, with a significant improvement. This is because after introducing the source domain data set, on the one hand, it brings more user and item information to the recommendation data set, to some extent, alleviating the cold start problem; on the other hand, through the shared user embedding, the information of the source domain and the target domain can be connected, making it easier for the model to more accurately model users and news, thereby improving the recommendation effect of the model.

[0201] 2) TECDR model is better than DTCDR model: Compared with the DTCDR model, the TECDR model introduces a user interest decomposition module, a user scenario interest module, and an interest fusion module based on text entailment. Compared with the DTCDR model, the TECDR model can more accurately depict the user's steady interest, domain-specific interest, and domain dynamic interest. Through modeling and fusion of multi-level interests, the user's interest is better depicted. In terms of experimental results, the DTCDR model has an effect of HR 0.405, NDCG 0.213 on cross-domain, while the TECDR reaches HR 0.440, NDCG 0.245, which has a very obvious improvement in experimental results.

[0202] 3) The model performs better after introducing text information: After adding text information as the pre-embedded representation of news in the TECDR model, the effect (HR 0.475, NDCG 0.272) is significantly improved compared to the TECDR model without using text information (HR 0.440, NDCG 0.245). This may be because the text information of the news contains rich semantic information, which is more conducive to the modeling and expression of news, and further improves the effect of the model through accurate characterization of news.

[0203] Table 4 shows the experimental results of the model under different text entailment methods. The fusion methods using addition, subtraction, and dot multiplication alone are tested, and the fusion methods combining addition and subtraction, addition and subtraction, and dot multiplication are also tried. The best effect is marked in bold in the table.

[0204] Table 4 Effect comparison of different text entailment methods

[0205]

[0206] As can be seen from the results in Table 4, the text entailment method using only addition has the best HR and NDCG index effect, which is significantly improved compared to the subtraction and dot multiplication methods. At the same time, adding additional subtraction and dot multiplication fusion operations based on the addition operation cannot well improve the effect of the model. These additional operations may make the model too complex and prone to overfitting and other problems. In comparison, the addition operation can better capture the similarity and correlation between texts, and therefore achieve better results.

[0207] It has to be noted that the terms "first", "second", and the like in connection with an entity or action refer to this entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without further constraints, exclude the presence of additional elements of the process, method, article, or apparatus.

[0208] While the application has been described with reference to specific implementations thereof, it should be understood that various modifications and substitutions can be made by those skilled in the art without departing from the scope of the present application. In particular, any one of the features of the present application disclosed above can be utilized independently of any other and the scope of the application should not be limited by the specific embodiments disclosed herein, but should be given the widest coverage possible in its true scope.

Claims

1. A cross-domain news recommendation system based on text entailment, characterized in that, The system comprises a user field interest generation module (1), a BERT-based news embedding module (2), a user scenario interest generation module based on an attention mechanism (3), an interest fusion module based on a text entailment idea (4), a click rate prediction module (5), and a loss function calculation module (6). The BERT-based news embedding module (2) is connected to the user scenario interest generation module based on an attention mechanism (3) and the click rate prediction module (5), the user field interest generation module (1) and the user scenario interest generation module based on an attention mechanism (3) are connected to the interest fusion module based on a text entailment idea (4), the interest fusion module based on a text entailment idea (4) is connected to the click rate prediction module (5), and the click rate prediction module (5) is connected to the loss function calculation module (6). The user field interest generation module (1) is used for fusing a user abstract interest vector and a field adaptive feature vector to obtain a user field interest vector. The BERT-based news embedding module (2) is used for fusing a collaborative information vector and a semantic information vector of news to obtain a fused news embedding vector. The user scenario interest generation module based on an attention mechanism (3) is used for inducing a user interactive news embedding vector through a candidate news embedding vector to obtain a user scenario interest vector. The interest fusion module based on a text entailment idea (4) fuses the obtained user field interest vector and user scenario interest vector to obtain a user multi-level interest vector. The interest fusion module based on the text entailment idea fuses the obtained user domain interest vector and the obtained user scenario interest vector to obtain a user multi-level interest vector The calculation expression is: ； wherein, is a user's domain interest vector, is a user's context interest vector; Or the interest fusion module based on the text implication idea fuses the obtained user's domain interest vector and the obtained user's scenario interest vector by splicing operation to obtain a user multi-level interest vector The calculation expression is: ； A text entailment judgment method is established by using the text entailment relationship between the obtained user field interest vector and user scenario interest vector, a label of a user's click on a candidate news is regarded as a label of whether a text entails, the user field interest vector and the user scenario interest vector are judged for an entailment relationship through a multi-layer full connection network, a subtraction operation is taken as a default text entailment combination mode, and a calculation expression is: y = f (x) - f (x') = 0 ； ； ； wherein, is a text entailment loss, S is a source domain or target domain training sample, is a predicted label for a text entailment relationship, u is a user, is news clicked by the user, is news not clicked by the user; The click rate prediction module (5) is used for calculating a similarity between a user multi-level interest vector and a candidate news embedding vector to obtain a prediction value of a user's click rate on a candidate news. The loss function calculation module (6) constructs a loss function of the text entailment-based cross-domain news recommendation system by using a cross-entropy loss function, and is used for updating parameters of the text entailment-based cross-domain news recommendation system to optimize the system.

2. A cross-domain news recommendation method based on text entailment, relying on the cross-domain news recommendation system based on text entailment of claim 1, characterized in that, The method comprises the following steps: S1, the user field interest generation module fuses a user abstract interest vector and a field adaptive feature vector to obtain a user field interest vector; S2, the BERT-based news embedding module fuses a collaborative information vector and a semantic information vector of news to obtain a fused news embedding vector; S3, the method based on step S2 processes user interactive news and candidate news to obtain a user interactive news embedding vector and a candidate news embedding vector, and the user scenario interest generation module based on an attention mechanism induces the user interactive news embedding vector through the candidate news embedding vector to obtain a user scenario interest vector. S4, the interest fusion module based on the text entailment idea fuses the user's domain interest vector obtained in step S1 and the user's scenario interest vector obtained in step S3 to obtain a user multi-level interest vector; S5, the click rate prediction module is used to calculate the similarity between the user multi-level interest vector obtained in step S4 and the candidate news embedding vector obtained in step S3 to obtain a prediction value of the user's click rate on the candidate news; S6, the loss function calculation module uses a cross-entropy loss function to construct a loss function of the cross-domain news recommendation system based on text entailment, which is used to update the parameters of the cross-domain news recommendation system based on text entailment for system optimization.

3. The cross-domain news recommendation method based on text entailment according to claim 2, characterized in that, The specific implementation method of step S1 includes the following steps: S1.1, collect news data to construct cross-domain news recommendation data set, including source domain news data set, target domain news data set, for the user of the source domain news data set, the target domain news data set, first construct the user target domain adaptive feature embedding matrix , the user source domain adaptive feature embedding matrix , the user target domain abstract interest embedding matrix , the user source domain abstract interest embedding matrix , wherein , user set of target domain news data set, user set of source domain news data set, all user set of source domain news data set and target domain news data set, d is the dimension of hidden vector; S1.2, obtaining the user target domain adaptation feature vector in a lookup table manner , the user source domain adaptation feature vector , the user target domain abstract interest embedding vector , the user source domain abstract interest embedding vector ; Then, the user abstract interest vector and the adaptive feature vector are fused to calculate the expression: ； ； wherein, is a source domain topical interest vector for the user, is a target domain topical interest vector for the user, is a topical interest vector for the user.

4. The cross-domain news recommendation method based on text entailment according to claim 3, characterized in that, In step S1, the loss function is introduced to calculate the Euclidean distance between the user target domain abstract interest embedding vector and the user source domain abstract interest embedding vector to minimize the difference between them, and the negative number of the Euclidean distance between the user target domain adaptive feature vector and the user source domain adaptive feature vector is calculated to maximize the difference between them. The calculation expression is: ； ； ； wherein, a loss function to bring user interests closer, a loss function to move domain adaptation features further apart, a sum of the loss function to bring user interests closer and the loss function to move domain adaptation features further apart.

5. The cross-domain news recommendation method based on text entailment according to claim 4, characterized in that, The specific implementation method of step S2 includes the following steps: S2.1, initialize the news embedding matrix, get the collaborative information embedding vector of the news by table lookup ; S2.2, data cleaning is performed on the multi-domain news text information, and then the text content embedding method of the Chinese pre-training language model BERT is used to process the multi-domain news text information after data cleaning, each news text is intercepted to a fixed length and padding operation is added, and the Chinese BERT model is input to understand the semantic content of the text, and the semantic embedding vector of the news is obtained ; S2.3, the step S2.1 obtained by embedding the news of the corresponding step S2.2 obtained by splicing the semantic embedding vector of the news into the vector, and then inputting into the multi-layer fully connected network to obtain the fusion news embedding vector The calculation expression is: MLP([ ]) ; where, is the fused news embedding vector, [;] is the concatenation operation, and MLP is a multi-layer fully connected network with a ReLU activation function.

6. The cross-domain news recommendation method based on text entailment according to claim 5, characterized in that, The specific implementation method of step S3 includes the following steps: S3.1, set the interactive news sequence of the user as wherein is the i-th interactive news of the user, k is the length of the interactive news sequence of the user, and the candidate news to be predicted is set as , initialize the source domain and target domain all-news embedding matrix wherein is the source domain and target domain all-news set; S3.2, processing the user interactive news and the candidate news in step S3.1 by the method of step S2 to obtain a user interactive news embedding sequence H={ } and a candidate news embedding vector , wherein is the i-th interactive news embedding vector of the user. S3.3, the user scenario interest generation module based on the attention mechanism induces the user interaction news embedding vector through the candidate news embedding vector, and obtains the user scenario interest vector by weighted average of the user interaction news embedding vector The calculation expression is: ； wherein, is the degree of contribution of the ith interactive news to the user's situational interest; The computational expression is: ； )； wherein, is a learnable parameter matrix, is a learnable parameter vector, and Mask is a mask operation for padding in the user history, is a weight vector for each interaction news of the user. Assume mask vector For Where 0 represents no user interaction with the news, the calculation expression for the Mask operation is: ； ； wherein represents a constant to prevent a division by zero error for users who have no interacted news.

7. The cross-domain news recommendation method based on text entailment according to claim 6, characterized in that, The specific implementation method of step S5 is to use a multi-layer feedforward network to calculate the similarity between the user multi-level interest vector obtained in step S4 and the candidate news embedding vector obtained in step S3, and the calculation expression is: ))； where V is the predicted value of the user's click rate on the current candidate news, MLP represents a multi-layer feedforward network, and the activation function uses Relu, is the activation function.

8. The cross-domain news recommendation method based on text entailment according to claim 7, characterized in that, The specific implementation method of step S6 includes the following steps: S6.1, the loss function calculation module uses a cross-entropy loss function to construct a loss function of the click rate prediction module, and the calculation expression is: ； ； ； wherein S is a training set including positive and negative example interaction records of users on news, u is a current user, is a positive example candidate news interacted by the user, is a negative example candidate news not interacted by the user, is a loss function calculation for a target domain, is a loss calculation for a source domain, is a loss function calculation common to the source domain and the target domain; S6.2, Constructing L2 regularization loss The calculation expression is: ； wherein, are all the learnable parameters of the model; S6.3、based on the loss function of step S6.1 and step S6.2, obtaining the loss function of the text entailment-based cross-domain news recommendation system The calculation expression is: ； wherein, is a penalty coefficient for the L2 regular loss, is is a loss coefficient, is is a loss coefficient.

Images