News recommendation method and system based on hierarchical granularity semantic embedding and self-attention

Abstract

The present disclosure provides a news recommendation method and system based on hierarchical granularity semantic embedding and self-attention, including: obtaining the click news history data and candidate news data of the user to be recommended, and performing encoding processing to obtain the corresponding news feature representation; the encoding processing is specifically: for each news data, obtaining the semantic tensors of the news data at different granularities based on a series of bidirectional long short-term memory network models and splicing them; using a hierarchical granularity attention mechanism to denoise the spliced semantic tensors to obtain a vector representation of the news data; based on the news feature representation of the click news history data, using a threshold-based scaled dot product attention mechanism to obtain a new feature representation, and based on the new feature representation, obtaining the vector representation of the user to be recommended through weighted processing; based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, obtaining the predicted click probability of the candidate news data for news recommendation.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of news recommendation, and in particular relates to a news recommendation method and system based on hierarchical granularity semantic embedding and self-attention. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Personalized news recommendation analyzes a user's historical news browsing history and news popularity to infer user preferences and recommend news of interest. User interests are diverse, and candidate news items only match a small portion of a user's previously clicked news items. Therefore, existing news recommendation methods often incorporate candidate news items into user interest modeling to accurately match candidate news items with specific user interests. For example, Wu et al. proposed a candidate-aware self-attention network that uses candidate news items as cues to model users' short-term and long-term interests. Wu et al. utilized dilated convolutions to construct a multi-level representation for each news item and obtain matching information between each previously clicked news item and the candidate news item at a fine-grained level. Qi et al. proposed an attention-based multi-domain matching framework for news recommendation that captures matching representations between each viewed news item and the candidate news item in terms of title, summary, and category. However, existing methods interact candidate news items with each historical news item, which results in the retention of noise information generated by historical news items unrelated to the candidate news item and interactions with the candidate news item, negatively impacting model performance. In addition, news feature extraction was previously based on neural network learning representation, but as the number of network layers deepened and the number of tensor mappings increased, the final output only retained coarse-grained information and lost the original fine-grained information.

[0004] The inventors found that to accurately match the user's interests with candidate news, it is usually only necessary for the candidate news to interact with those important historical click news. Figure 1 It's not difficult to see that each candidate news item may only match a portion of a user's interests. For example, the first candidate news item only matches the user's interest in sports and has little relevance to other interests (such as politics and travel). If traditional methods were used to incorporate candidate news items into user interest modeling, namely, interacting with each historical news item, this approach would retain noise information generated by the interaction between unimportant historical news and candidate news items. This noise information would be mixed into the subsequent user modeling process, negatively impacting the accuracy of user interest modeling. Summary of the Invention

[0005] In order to solve the above problems, the present disclosure provides a news recommendation method and system based on hierarchical granularity semantic embedding and self-attention. The scheme is based on the proposed threshold attention module. Before the candidate news interacts with the historical news, the historical news that is irrelevant to the current candidate news is screened out to avoid the introduction of noise by its interaction with the candidate news, thereby forming a more accurate user representation. At the same time, in order to more effectively understand the semantics of the news, a news feature encoding module is proposed to combine the coarse-grained information and fine-grained information of the news features to avoid the loss of fine-grained information of the news as the neural network deepens. This allows the model to learn the coarse-grained information of the global semantics of the news while paying attention to the original fine-grained information, thereby effectively ensuring the accuracy of news recommendation.

[0006] According to a first aspect of an embodiment of the present disclosure, a news recommendation method based on hierarchical granularity semantic embedding and self-attention is provided, comprising:

[0007] Obtain the click history data of the user to be recommended and the candidate news data;

[0008] The historical click news data and candidate news data are encoded separately to obtain corresponding news feature representations. Specifically, the encoding process includes: for each piece of news data, semantic tensors of the news data at different granularities are obtained based on a series-connected bidirectional long short-term memory network model and then concatenated; the concatenated semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data.

[0009] Based on the news feature representation of click-through news history data, a threshold-based scaled dot-product attention mechanism is used to obtain a new feature representation. Based on the new feature representation, a weighted processing is performed to obtain the vector representation of the user to be recommended.

[0010] Based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, the predicted click probability of the candidate news data is obtained, and news recommendation is performed based on the predicted click probability.

[0011] Furthermore, for each piece of news data, the semantic tensors of the news data at different granularities are obtained and spliced based on the serial bidirectional long short-term memory network model. Specifically, the serial bidirectional long short-term memory network model includes a first bidirectional long short-term memory network and a second bidirectional long short-term memory network connected in series, with each piece of news data being used as the input of the first bidirectional long short-term memory network, and the output of the first bidirectional long short-term memory network being a fine-grained semantic tensor; the fine-grained semantic tensor being used as the input of the second bidirectional long short-term memory network, and the output of the second bidirectional long short-term memory network being a coarse-grained semantic tensor, and the fine-grained semantic tensor and the coarse-grained semantic tensor are spliced.

[0012] Furthermore, the spliced semantic tensor is denoised using a hierarchical granular attention mechanism to obtain a vector representation of the news data. Specifically, based on the spliced semantic tensor, the attention mechanism is used to calculate the weight value corresponding to the vector representation of the news data at each time step; based on the obtained weight value, the vector representation of each time step in the spliced semantic tensor is weightedly summed to obtain the vector representation of the news data.

[0013] Furthermore, each piece of news data includes a news title, a category, and a subcategory, and the vector representation of each piece of news data is obtained by weighted concatenation of the encoding results of the news title, category, and subcategory.

[0014] Furthermore, based on the news feature representation of the click news history data, a threshold-based scaled dot product attention mechanism is used to obtain a new feature representation. Specifically: based on the news feature representation of the click news history data, the corresponding initial attention score matrix and initial attention weight are calculated. When the initial attention weight is less than the preset threshold, the elements in the initial attention score matrix are assigned to negative infinity to obtain a new attention score matrix, and new attention weights are obtained based on the new attention score matrix; based on the product of the new attention weight and the content vector in the attention mechanism, a new feature representation is obtained.

[0015] Furthermore, the threshold-based scaled dot product attention mechanism adopts a multi-head attention mechanism, wherein the output results of each attention head are spliced together to obtain a new feature representation.

[0016] Furthermore, the vector representation of the user to be recommended is obtained through weighted processing based on the new feature representation, specifically: based on the click news history data and the news feature representation of the candidate news, a threshold attention mechanism guided by the candidate news is adopted, and the weight corresponding to each news data in the click news history data is calculated according to the candidate news, and then compared with the preset threshold to obtain a new attention weight; and based on the news feature representation of the click news history data and the weight, a weighted calculation is performed to obtain the vector representation of the user to be recommended.

[0017] According to a second aspect of an embodiment of the present disclosure, a news recommendation system based on hierarchical granularity semantic embedding and self-attention is provided, comprising:

[0018] A data acquisition module is used to obtain the click history data of the user to be recommended and the candidate news data;

[0019] The news encoding module is used to encode the click news history data and candidate news data to obtain corresponding news feature representations. The encoding process specifically includes: for each piece of news data, semantic tensors at different granularities are obtained based on a series of bidirectional long short-term memory network models and then spliced; the spliced semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data;

[0020] The user encoding module is used to obtain new feature representations based on the news feature representation of click-through news history data using a threshold-based scaled dot-product attention mechanism, and then obtain the vector representation of the user to be recommended based on the new feature representation through weighted processing;

[0021] The click prediction module is used to obtain the predicted click probability of candidate news data based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, and to recommend news based on the predicted click probability.

[0022] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, a news recommendation method based on hierarchical granularity semantic embedding and self-attention is implemented.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the news recommendation method based on hierarchical granularity semantic embedding and self-attention is implemented.

[0024] Compared with the prior art, the present invention has the following advantages:

[0025] (1) The present disclosure provides a news recommendation method and system based on hierarchical granularity semantic embedding and self-attention. The scheme is based on the proposed threshold attention module, which removes historical news that is irrelevant to the candidate news by setting a threshold to avoid noise information generated by the interaction between the irrelevant historical news and the candidate news; at the same time, in order to more effectively understand the semantics of the news, a news feature encoding module is proposed to combine the coarse-grained information and fine-grained information of the news features to avoid the loss of fine-grained information of the news as the neural network deepens, so that the model can learn the coarse-grained information of the global semantics of the news and pay attention to the original fine-grained information, thereby effectively ensuring the accuracy of news recommendation.

[0026] (2) Through a large number of experiments on the large real-world dataset Microsoft News Dataset (MIND), the scheme described in this disclosure has been proven to be superior to existing methods and effectively improve the performance of news recommendation.

[0027] Advantages of additional aspects of the present disclosure will be given in part in the following description and in part will become apparent from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings, which constitute a part of the present disclosure, are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure.

[0029] Figure 1 This is an example of a user's behavior of reading news as described in the background technology of this disclosure;

[0030] Figure 2 This is an overall flow chart of a news recommendation method based on hierarchical granularity semantic embedding and self-attention described in an embodiment of the present disclosure;

[0031] Figure 3 Schematic diagram of the calculation process of the threshold-based scaled dot product attention mechanism described in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0032] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0033] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0034] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0035] In the absence of conflict, the embodiments of the present disclosure and the features thereof may be combined with each other.

[0036] Example 1:

[0037] The purpose of this embodiment is to provide a news recommendation method based on hierarchical granularity semantic embedding and self-attention.

[0038] A news recommendation method based on hierarchical granular semantic embedding and self-attention, including:

[0039] Obtain the click history data of the user to be recommended and the candidate news data;

[0040] The historical click news data and candidate news data are encoded separately to obtain corresponding news feature representations. Specifically, the encoding process includes: for each piece of news data, semantic tensors of the news data at different granularities are obtained based on a series-connected bidirectional long short-term memory network model and then concatenated; the concatenated semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data.

[0041] Based on the news feature representation of click-through news history data, a threshold-based scaled dot-product attention mechanism is used to obtain a new feature representation. Based on the new feature representation, a weighted processing is performed to obtain the vector representation of the user to be recommended.

[0042] Based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, the predicted click probability of the candidate news data is obtained, and news recommendation is performed based on the predicted click probability.

[0043] In a specific implementation, for each piece of news data, the semantic tensors of the news data at different granularities are obtained and spliced based on the serial bidirectional long short-term memory network model. Specifically, the serial bidirectional long short-term memory network model includes a first bidirectional long short-term memory network and a second bidirectional long short-term memory network connected in series, with each piece of news data being used as the input of the first bidirectional long short-term memory network, and the output of the first bidirectional long short-term memory network being a fine-grained semantic tensor; the fine-grained semantic tensor being used as the input of the second bidirectional long short-term memory network, and the output of the second bidirectional long short-term memory network being a coarse-grained semantic tensor, and the fine-grained semantic tensor and the coarse-grained semantic tensor are spliced.

[0044] In a specific implementation, the spliced semantic tensor is denoised using a hierarchical granular attention mechanism to obtain a vector representation of the news data. Specifically, based on the spliced semantic tensor, the attention mechanism is used to calculate the weight value corresponding to the vector representation of the news data at each time step; based on the obtained weight value, the vector representation of each time step in the spliced semantic tensor is weightedly summed to obtain the vector representation of the news data.

[0045] In a specific implementation, each piece of news data includes a news title, a category, and a subcategory, and the vector representation of each piece of news data is obtained by weighted concatenation of the encoding results of the news title, category, and subcategory.

[0046] In the specific implementation, based on the news feature representation of the click news history data, a threshold-based scaled dot product attention mechanism is used to obtain a new feature representation, specifically: based on the news feature representation of the click news history data, the corresponding initial attention score matrix and initial attention weight are calculated. When the initial attention weight is less than the preset threshold, the elements in the initial attention score matrix are assigned to negative infinity to obtain a new attention score matrix, and new attention weights are obtained based on the new attention score matrix; based on the product of the new attention weight and the content vector in the attention mechanism, a new feature representation is obtained.

[0047] In a specific implementation, the threshold-based scaled dot product attention mechanism adopts a multi-head attention mechanism, wherein the output results of each attention head are spliced together to obtain a new feature representation.

[0048] In a specific implementation, the vector representation of the user to be recommended is obtained through weighted processing based on the new feature representation, specifically: based on the click news history data and the news feature representation of the candidate news, a threshold attention mechanism guided by the candidate news is adopted, and the weight corresponding to each news data in the click news history data is calculated according to the candidate news, and then compared with the preset threshold to obtain a new attention weight; and based on the news feature representation of the click news history data and the weight, a weighted calculation is performed to obtain the vector representation of the user to be recommended.

[0049] For ease of understanding, the solution described in this embodiment is described in detail below with reference to the accompanying drawings:

[0050] like Figure 2 As shown, this embodiment provides a news recommendation method based on hierarchical granular semantic embedding and self-attention. Its specific technical concept is: a threshold attention module is proposed to obtain a pure historical news representation without noise by setting a threshold, so as to avoid the interaction between historical news that is irrelevant to the candidate and the candidate news to generate noise information. In order to understand the semantics of news more effectively, the scheme described in this embodiment designs a hierarchical granular information extraction module, which combines the coarse-grained information and fine-grained information of news features to avoid the loss of fine-grained information of news as the neural network deepens, so that the model can learn the coarse-grained information of the global semantics of the news and pay attention to the original fine-grained information; at the same time, in the process of model training, negative sampling technology is applied to K+1 news to predict click scores. That is, the news consists of a positive sample of a user and a negative sample of a randomly selected user. Then the positive news is predicted together. and Negative news.

[0051] The overall architecture of the method described in this embodiment mainly includes three parts: a news encoder, a candidate user encoder, and a click prediction module. Each part is described in detail below:

[0052] (1) Problem definition

[0053] Given a user u, it is composed of a set of historical click news browsed by the user, denoted as D u =[D1,D2,...,D N ]. Where N is the number of news clicked by the user in history. Our goal is to calculate the number of candidate news clicked by user u. C Probability Among them D C =[D c1 ,D c2 ,...,D cM Then, we sort the candidate news according to their click scores to recommend the best news.

[0054] (2) Solution

[0055] (1) News coding module

[0056] In order to understand the information representation of each news item, we design a novel news encoder. Figure 3 As shown, for each news item, title, category, and subcategory are input into the encoder, and a unified news representation is output.

[0057] Now we briefly introduce how to encode news titles. Given a news title D, the tag is represented as D i =[w1,w2,...,w M ], First, we use Glove to initialize the embedding matrix E i =[e1,e2,...,e M ], where e m ∈W e , W e ∈R V×D , V and D are the vocabulary size and embedding dimension, respectively. Secondly, we input the obtained low-dimensional vector representation of the news headline into the hierarchical granularity information extraction module. To enable the model to better obtain coarse-grained and fine-grained information, the model later combines the coarse-grained information with the fine-grained information from the previous moment and simultaneously applies it to downstream tasks. This allows the model to understand both the coarse-grained global semantic information of the news and the fine-grained semantic information in downstream tasks. Here, we use two Bi-LSTM models in series to obtain semantic tensors at different representation levels. The calculation formula is as follows:

[0058] B1=Bi-LSTM(E w ) (1)

[0059] B2=Bi-LSTM(B1) (2)

[0060] B1 represents the representation tensor learned at the first level, and B2 represents the representation tensor learned at the second level. Then, the two tensors of different granularities are concatenated, and the calculation formula is as follows:

[0061] B rep =add(B1,B2) (3)

[0062] add(·) means adding two pieces of tensor information together.

[0063] Finally, the concatenation of information at different levels of granularity will inevitably introduce noise. Not all information is important, and different information will inevitably have different degrees of importance. The HGTS model adds a hierarchical granular attention with a "selection mechanism" behind the tensor of coarse-grained and fine-grained concatenation. It assigns different weights to the concatenated tensor by calculating the importance of each moment in each granularity, allowing the model's later aggregation and judgment to pay attention to and make choices about overly long input information. The calculation formula for hierarchical granular attention is as follows:

[0064]

[0065] in, Represents the vector representation of the i-th time step of the input text, W i represents the learnable parameters, σ(·) represents the sigmoid activation function, Represents the weight value of the i-th time step. Then use the weight value to perform weighted processing on the original text:

[0066] r t =Q att ·B rep (5)

[0067] At the same time, we use a linear network to generate representations of categories and subcategories, labeled r c ,r sc .

[0068] We apply attention networks to learn the weights of various information in news and then construct the final news representation. For example, the title attention weight α t The evaluation method is described as follows:

[0069]

[0070]

[0071] where σ is tanh, q t 、V t and v t are learnable parameters.c and a sc , which are obtained using equations similar to Equ.(6). The attention weights of categories and subcategories are evaluated using a procedure similar to that of title weights, i.e., Equ.(7), and are expressed as α and c and α sc .

[0072] Finally, the news feature embedding r is described by the following equation:

[0073] r=[α t r t ; α c r c ; α sc r sc ] (8)

[0074] (2) User coding module

[0075] The user modeling module in this invention aims to learn the user’s representation from the representation of the news the user clicks on, such as Figure 2 shown.

[0076] The news feature matrix we input is expressed as R = [r1, r2, ... r N ]. Given the matrix:

[0077] Q=W q *R (9)

[0078] K=W K *R (10)

[0079] V=W v *R (11)

[0080] Among them, W q 、W k 、W v is a trainable parameter matrix.

[0081] The traditional attention mechanism enables the model to have a strong noise filtering ability. This effect is achieved by assigning smaller weights to irrelevant features and weakly related features. The specific calculation process can be described by the following formula:

[0082]

[0083] W=softmax(S) (13)

[0084] F=WV (14)

[0085] where k is the size of K. Divided by The purpose of this is to prevent the size of the dot product from being too large and to ensure that this method produces useful gradients.

[0086] However, although the traditional scaled dot product attention can enhance the significant historical click news features to a certain extent, other irrelevant historical news will still participate in the interaction, that is, the noise is not completely eliminated. The noise information in these unimportant features will affect the final prediction of the model and have a negative impact on the performance of the model. Therefore, we use the threshold-based scaled dot product self-attention, such as Figure 3 As shown in Figure 2, by setting the attention score threshold to filter out the most helpful historical news, the model’s attention is focused, ultimately improving the overall performance of the model.

[0087] Specifically, we compare the value of the attention weight matrix W with the threshold T. When the value of the weight is less than T, we set the value of the corresponding position in the attention score S to -∞. Then, we input the new attention score S′ into the softmax function to obtain the new attention weight W′. The calculation process of our proposed threshold attention mechanism can be described by the following formula:

[0088]

[0089] W′=softmax(S′) (16)

[0090] F=W′V (17)

[0091] By doing so, the new attention weights of features with lower attention scores will be infinitely close to 0, that is, irrelevant historical news will not participate in the calculation of the attention feature F. Figure 2 Figure 3 shows the computation of our threshold-based scaled dot-product attention, where Q, K, and V are the inputs. S is the first calculated attention score, S′ is the last calculated attention score, W is the first calculated attention weight, and W′ is the last calculated attention weight. T is the threshold, and F is the output attention feature.

[0092] Multi-head attention assigns features from different representation spaces to different attention heads, enabling these attention heads to independently learn information from different positions. We will perform threshold-based scaled dot product attention in parallel in different heads and concatenate their outputs as the output of our threshold-based multi-head scaled dot product attention. This is expressed as follows:

[0093]

[0094] S news =MultiHead(Q,K,V)

[0095] =concat(F head1 ,F head2 ,...,F headH)W q (19)

[0096] Record the results as:

[0097] S news =[s1,s2,...,s N ] (20)

[0098] in, and is the linear projection parameter matrix, F headh is the output of the hth threshold-based scaled dot-product attention head.

[0099] Next, in order to filter out historical news that is weakly related to the current candidate news and avoid user modeling of interactive participation between candidate news and irrelevant historical news, we feed the obtained historical news into the threshold attention guided by the candidate news:

[0100] Q=W q *r c 、K=W k *S, V=W v *S (21)

[0101] u=threshold_based_attention(QW Q ,KW K ,VW V ) (twenty two)

[0102] (3) Click prediction module

[0103] Get candidate news c After the representation of and the representation of user u, we can predict the click probability score by the inner product

[0104] We use negative sampling technology for the following training. In the same session, each news browsed by the user is considered a positive sample, and then we randomly sample K exposed news that were not clicked by the user as negative samples. The click probability score of the i-th positive news in the same period is expressed as The click-through rate score of the jth piece of K negative news is expressed as The news click prediction problem can be considered as a pseudo K+1-way classification task. We apply the softmax function to normalize these click probability scores to the posterior click probability of the positive samples.

[0105]

[0106] The loss function L is the negative log-likelihood of all positive samples S:

[0107]

[0108] Example 2:

[0109] The purpose of this embodiment is to provide a news recommendation system based on hierarchical granularity semantic embedding and self-attention.

[0110] A news recommendation system based on hierarchical granular semantic embedding and self-attention, including:

[0111] A data acquisition module is used to obtain the click history data of the user to be recommended and the candidate news data;

[0112] The news encoding module is used to encode the click news history data and candidate news data to obtain corresponding news feature representations. The encoding process specifically includes: for each piece of news data, semantic tensors at different granularities are obtained based on a series of bidirectional long short-term memory network models and then spliced; the spliced semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data;

[0113] The user encoding module is used to obtain new feature representations based on the news feature representation of click-through news history data using a threshold-based scaled dot-product attention mechanism, and then obtain the vector representation of the user to be recommended based on the new feature representation through weighted processing;

[0114] The click prediction module is used to obtain the predicted click probability of candidate news data based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, and to recommend news based on the predicted click probability.

[0115] Furthermore, the system described in this embodiment corresponds to the method described in Example 1, and its technical details are described in detail in Example 1, so they are not repeated here.

[0116] In further embodiments, there is also provided:

[0117] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor. When the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0118] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0119] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0120] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in embodiment 1 is performed.

[0121] The method in Example 1 can be directly implemented as being executed by a hardware processor, or by a combination of hardware and software modules within the processor. The software module can be located in a storage medium well-established in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, a detailed description is not given here.

[0122] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in this embodiment can be implemented using electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0123] The above embodiment provides a news recommendation method and system based on hierarchical granularity semantic embedding and self-attention, which can be implemented and has broad application prospects.

[0124] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A news recommendation method based on hierarchical granularity semantic embedding and self-attention, characterized by: include: Obtain the click history data of the user to be recommended and the candidate news data; The historical click news data and candidate news data are encoded separately to obtain corresponding news feature representations. Specifically, the encoding process includes: for each piece of news data, semantic tensors of the news data at different granularities are obtained based on a series-connected bidirectional long short-term memory network model and then concatenated; the concatenated semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data. Based on the news feature representation of click-through news history data, a threshold-based scaled dot-product attention mechanism is used to obtain a new feature representation. Based on the new feature representation, a weighted processing is performed to obtain the vector representation of the user to be recommended. Based on the inner product of the news feature representation of the candidate news data and the vector representation of the user to be recommended, the predicted click probability of the candidate news data is obtained, and news recommendations are made based on the predicted click probability; The method adopts a threshold-based scaled dot product attention mechanism to obtain a new feature representation, specifically: based on the news feature representation of the click news history data, the corresponding initial attention score matrix and initial attention weight are calculated; when the initial attention weight is less than a preset threshold, the elements in the initial attention score matrix are assigned a value of negative infinity to obtain a new attention score matrix, and new attention weights are obtained based on the new attention score matrix; the new feature representation is obtained based on the product of the new attention weight and the content vector in the attention mechanism; The method obtains the vector representation of the user to be recommended through weighted processing based on the new feature representation, specifically: based on the click news history data and the news feature representation of the candidate news, adopts a threshold attention mechanism guided by the candidate news, calculates the weight corresponding to each news data in the click news history data according to the candidate news, and then compares it with the preset threshold to obtain a new attention weight; and based on the news feature representation of the click news history data and the weight, a weighted calculation is performed to obtain the vector representation of the user to be recommended.

2. A news recommendation method based on hierarchical granularity semantic embedding and self-attention as claimed in claim 1, characterized in that: For each piece of news data, the semantic tensors of the news data at different granularities are obtained and spliced based on the serial bidirectional long short-term memory network model. Specifically, the serial bidirectional long short-term memory network model includes a first bidirectional long short-term memory network and a second bidirectional long short-term memory network connected in series, and each piece of news data is used as the input of the first bidirectional long short-term memory network, and the output of the first bidirectional long short-term memory network is a fine-grained semantic tensor; the fine-grained semantic tensor is used as the input of the second bidirectional long short-term memory network, and the output of the second bidirectional long short-term memory network is a coarse-grained semantic tensor, and the fine-grained semantic tensor and the coarse-grained semantic tensor are spliced.

3. The news recommendation method based on hierarchical granularity semantic embedding and self-attention according to claim 1, characterized in that: The spliced semantic tensor is denoised using a hierarchical granular attention mechanism to obtain a vector representation of the news data. Specifically, based on the spliced semantic tensor, the attention mechanism is used to calculate the weight value corresponding to the vector representation of the news data at each time step; based on the obtained weight value, the vector representation of each time step in the spliced semantic tensor is weightedly summed to obtain the vector representation of the news data.

4. The news recommendation method based on hierarchical granularity semantic embedding and self-attention according to claim 1, characterized in that: Each piece of news data includes a news title, category, and subcategory. The vector representation of each piece of news data is obtained by weighted concatenation of the encoding results of the news title, category, and subcategory.

5. The news recommendation method based on hierarchical granularity semantic embedding and self-attention according to claim 1, characterized in that: The threshold-based scaled dot product attention mechanism adopts a multi-head attention mechanism, wherein the output results of each attention head are spliced together to obtain a new feature representation.

6. A news recommendation system based on hierarchical granularity semantic embedding and self-attention, characterized by: The system is used to execute the news recommendation method based on hierarchical granularity semantic embedding and self-attention as described in claim 1, and the system includes: A data acquisition module is used to obtain the click history data of the user to be recommended and the candidate news data; The news encoding module is used to encode the click news history data and candidate news data to obtain corresponding news feature representations. The encoding process specifically includes: for each piece of news data, semantic tensors at different granularities are obtained based on a series of bidirectional long short-term memory network models and then spliced; the spliced semantic tensors are denoised using a hierarchical granularity attention mechanism to obtain a vector representation of the news data; The user encoding module is used to obtain new feature representations based on the news feature representation of click-through news history data using a threshold-based scaled dot-product attention mechanism, and then obtain the vector representation of the user to be recommended based on the new feature representation through weighted processing; The click prediction module is used to obtain the predicted click probability of candidate news data based on the news feature representation of the candidate news data and the inner product of the vector representation of the user to be recommended, and to recommend news based on the predicted click probability.

7. An electronic device comprising a memory, a processor, and a computer program stored and running on the memory, characterized in that: When the processor executes the program, it implements a news recommendation method based on hierarchical granularity semantic embedding and self-attention as described in any one of claims 1-5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements a news recommendation method based on hierarchical granularity semantic embedding and self-attention as described in any one of claims 1 to 5.

Images