A scientific and technological information recommendation method based on long-term and short-term preferences

Abstract

The present invention relates to a method for recommending scientific and technological intelligence based on long-term and short-term preferences, which belongs to the field of recommendation systems and can be used to provide users with personalized scientific and technological intelligence recommendations. The method includes: using a multi-head self-attention mechanism to extract representations of intelligence titles and summaries respectively, and then combining the intelligence source representation to obtain a more accurate intelligence representation; using a long-term and short-term memory network and a user ID embedding method to learn long-term and short-term preference representations; and combining the long-term and short-term preference representations with the intelligence representation to calculate the recommendation score, thereby realizing personalized scientific and technological intelligence recommendations. The present invention solves the shortcomings of existing recommendation systems in intelligence representation and the problem of inaccuracy in capturing users' long-term and short-term interests. It can capture novel and sudden scientific and technological intelligence during the recommendation process, and is suitable for intelligence recommendation scenarios in the fields of national defense, scientific research, education, etc. It helps to improve the work efficiency of intelligence personnel and promote knowledge dissemination and technological innovation.

Description

Technical Field

[0001] The present invention relates to a scientific and technological information recommendation method, in particular to a scientific and technological information recommendation method based on long-term and short-term preferences, and belongs to the field of recommendation systems. Background Art

[0002] With the rapid development of the internet and information technology, the generation and dissemination of scientific and technological intelligence is accelerating. Recommender systems have become an important solution to help users quickly find interesting scientific and technological information from this vast amount of information. These systems analyze user preferences and behaviors to recommend potentially interesting information. However, existing recommendation systems still have limitations when it comes to recommending scientific and technological information.

[0003] First, existing recommendation systems primarily focus on users' short-term behavior, ignoring their long-term interests. Users' interests are diverse, ranging from long-term, stable interests to short-term, volatile interests. Excessive focus on short-term behavior can cause recommendations to deviate from users' long-term interests, thereby reducing recommendation quality and user satisfaction.

[0004] Secondly, existing recommendation systems face challenges in processing multi-source, heterogeneous data. Scientific and technological intelligence comes from a wide variety of sources, including academic papers, news reports, policy documents, industry reports, and more. These data types and structures vary, making it a pressing challenge to effectively integrate this multi-source, heterogeneous data into recommendation systems. Furthermore, assigning appropriate weights to different types of scientific and technological intelligence data to improve recommendation quality is also a significant concern.

[0005] Furthermore, traditional recommendation methods face the problem of excessive computational complexity when processing massive amounts of data. With the continued growth of scientific and technological intelligence data and the increasing number of users, traditional recommendation methods may not be able to meet the needs of real-time recommendations. Therefore, an efficient scientific and technological intelligence recommendation system should be able to reduce computational complexity while ensuring recommendation quality and achieve efficient real-time recommendations.

[0006] Furthermore, existing recommendation systems mostly focus on recommending existing information and perform poorly in recommending novel and emerging scientific and technological information. This novel and emerging information is highly valuable to users, so how to capture this novel and emerging information in the recommendation process and provide users with timely recommendations has become an important research direction.

[0007] In summary, the current scientific and technological intelligence recommendation system still has shortcomings in capturing users' long-term and short-term interests, processing multi-source heterogeneous data, reducing computational complexity, and recommending novel and sudden intelligence. Therefore, it is urgent to develop an effective scientific and technological intelligence recommendation method to improve the accuracy and timeliness of recommendations. Summary of the Invention

[0008] The purpose of this invention is to solve the shortcomings of existing recommendation systems in terms of information representation and the inaccuracy in capturing users' long-term and short-term interests, and to creatively propose a scientific and technological information recommendation method based on long-term and short-term preferences.

[0009] The innovations of this invention are as follows: First, it proposes an intelligence encoding method that combines intelligence titles, intelligence sources, and intelligence summaries, effectively addressing the shortcomings of existing research in intelligence representation, thereby improving the accuracy of intelligence recommendations. Second, based on the above-mentioned intelligence encoding method, the present invention proposes a method for representing users' long-term and short-term preferences. By utilizing a long short-term memory (LSTM) network to learn users' short-term preference representations from their recent compilation records, while simultaneously using an ID embedding-based method to learn users' long-term preference representations, and using the long-term preference representations as the initial state of the LSTM, a user's long-term and short-term preference representations are obtained.

[0010] The present invention is achieved through the following technical solutions.

[0011] First, we define an intelligence encoder. For each piece of intelligence, we use the title, source, and summary of the intelligence to learn the intelligence representation. The intelligence encoder encodes the intelligence into e = [r s ,r t ,r a ], where r s According to intelligence sources, t For information title, r a Next, the user ID embedding method is used to represent the user's long-term preference, and the user's long-term preference is represented as u l =W u [u]. Then, the LSTM network is used to learn the user’s long-term and short-term representation u from the user’s intelligence compilation history, where u l As the initial state of the LSTM network. Finally, the dot product of the candidate intelligence representation vector and the user representation vector is calculated as the recommendation score.

[0012] A method for recommending scientific and technological intelligence based on long-term and short-term preferences includes the following steps:

[0013] Step 1: Define the intelligence encoder, including the following steps;

[0014] Step 1.1: Encode the intelligence headlines, which includes 3 sub-steps;

[0015] Step 1.1.1: Convert the intelligence title from a word sequence to a low-dimensional semantic vector sequence. Assume that the word sequence of the intelligence title is [w1,w2,...,wN ], where N is the length of the title, through a word embedding lookup table The word sequence is converted into a word vector sequence [e1,e2,...,e N ], where V and D are the vocabulary size and word embedding dimension respectively;

[0016] Step 1.1.2: Use a multi-head self-attention network to capture the interactions between words to learn the contextual representation of words. The representation of the i-th word learned by the k-th attention head is calculated as:

[0017]

[0018]

[0019] in, and is the projection parameter in the k-th self-attention head, Indicates the relative importance of the interaction between the i-th and j-th words. Multi-head representation of the i-th word is the concatenation of the representations produced by h independent self-attention heads, i.e.

[0020] Step 1.1.3: Use the attention mechanism to select important words in the intelligence title. The attention weight of the i-th word in the intelligence title The calculation method is:

[0021]

[0022] Among them, V w and v w is the projection parameter, q w is the query vector. The final representation of the intelligence title is the weighted sum of the context representations of the words, i.e.

[0023] Step 1.2: Encode the intelligence summary, which includes 3 sub-steps

[0024] Step 1.2.1: Represent the word sequence of the intelligence summary as [w1,w2,...,w M ], where M is the length of the summary. Through the word embedding layer, it is converted into a word vector sequence [e1,e2,...,e M ];

[0025] Step 1.2.2: Use a multi-head self-attention network to take the word vector sequence as the input of the network and learn the contextual word representation by capturing the interaction between words

[0026] Step 1.2.3: Use the attention network to calculate the attention weight of each word, and finally obtain the intelligence summary representation by weighted summing the context representation of the summary word, i.e.

[0027] Step 1.3: Encode the intelligence source, which includes two sub-steps;

[0028] Step 1.3.1: Use ID embedding method to embed the discrete representation of intelligence source v s Convert to low-dimensional dense representation e s ;

[0029] Step 1.3.2: Use a fully connected layer to learn the hidden source representation from the source embedding, which is calculated as r s =ReLu(V s ×e s +v s ), where V s and v s are the parameters of the fully connected layer;

[0030] Step 1.4: Concatenate the intelligence title representation, intelligence summary representation, and intelligence source representation to obtain the intelligence representation, i.e., e = [r s , r t , r a ];

[0031] Step 2: Learning the user's long-term and short-term preference representation, including the following steps:

[0032] Step 2.1: Use the method based on user ID embedding to calculate the user's long-term preference representation, with u representing the user's ID and W representing the user's long-term preference representation. u The query table represents the user's long-term preference, and the user's long-term preference is represented by u l =W u [u];

[0033] Step 2.2: Use the LSTM network to calculate the user's short-term preference representation from the user's recently compiled intelligence history;

[0034] Step 2.2.1: Express the intelligence compilation sequence sorted by user in ascending order of timestamp as , where k is the sequence length, and the intelligence encoder is used to obtain the representation of these intelligences {e1, e2, ..., e k};

[0035] Step 2.2.2: Calculate the user's short-term preference representation as follows:

[0036] i t =σ(W i [ht-1 , e t ]+b i ), (1)

[0037] f t =σ(W f [h t-1 , e t ]+b f ), (2)

[0038] o t =σ(W o [h t-1 , e t ]+b o ), (3)

[0039] c t =f t ⊙c t-1 +i t ⊙tanh(W c [h t-1 , e t ]+b c ), (4)

[0040] h t =o t ⊙tanh(c t ) (5)

[0041] Among them, σ is the sigmoid activation function, ⊙ is the term-by-term multiplication, W i 、W f and W c is the weight matrix parameter of the LSTM network, b i 、b f and b c is the corresponding bias parameter, i t 、f t and o t Represents the activation values ​​of the input gate, forget gate and output gate at the current moment, c t Indicates the cell state at the current moment, h t Represents the hidden state at the current moment. The user's short-term preference representation is the last hidden state of the LSTM network, i.e. u s =h k ;

[0042] Step 2.3: Denote the user’s long-term preference as u l As the initial state of the LSTM network, the last hidden state of the LSTM network is used as the user's long-term and short-term preference representation u;

[0043] Step 3: For a given candidate intelligence, calculate the recommendation score, denote the user's long-term and short-term preferences as u, and denote the candidate intelligence as e x , then the predicted score of the user clicking on the information is s(u,c x )=u T e x , where T is the transpose operation.

[0044] Beneficial effects

[0045] Compared with the existing technology, the method of the present invention has the following advantages: 1. More accurate intelligence representation: The present invention combines multiple information such as intelligence title, intelligence source and intelligence summary, and adopts multi-head self-attention network and ID embedding method to obtain corresponding representation according to their respective characteristics, which can more accurately represent the intelligence content, thereby improving the accuracy of recommendation results; 2. More accurate user representation: The present invention adopts a method for representing user long-term and short-term preferences, which fully considers the diversity and variability of user interests, helps to generate recommendation results that are more in line with user needs and improve user satisfaction; and 3. Stronger adaptability: Using LSTM network and ID embedding-based method, the recommendation method proposed by the present invention can maintain high recommendation performance under different scenarios and changes in user behavior, thereby enhancing the adaptability and robustness of the recommendation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is the overall model diagram of the method of the present invention;

[0047] Figure 2 Schematic diagram of an intelligence encoder according to the method of the present invention. DETAILED DESCRIPTION

[0048] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0049] Example

[0050] This embodiment recommends scientific and technological intelligence to a user based on the user's intelligence compilation history.

[0051] When the method of the present invention is implemented, the data set used includes an intelligence data table and an interaction data table. The intelligence data table is shown in Table 1 and includes the intelligence number, source, title and summary, which serve as input information for the intelligence encoder.

[0052] Table 1 Intelligence data table

[0053]

[0054]

[0055] The interactive data table is shown in Table 2, which includes the user ID, the user's historical compilation record, and the actual click results on the candidate intelligence list under the compilation record.

[0056] Table 2 Interaction data table

[0057]

[0058] This example first encodes all the intelligence in the user's compiled records. We use pre-trained Glove embeddings as the initial word embeddings, with a word embedding dimension of 300. We set the intelligence title length to 25 words and the intelligence summary length to 50 words. This yields a table of intelligence data after processing through the embedding layer, as shown in Table 3.

[0059] Table 3 Intelligence data table after embedding processing

[0060]

[0061] The intelligence encoder proposed in this method obtains 300-dimensional word vectors from word IDs, which are then used to obtain vector representations of the intelligence title and summary. A multi-headed self-attention network is then used to obtain contextual representation vectors for the words in the title and summary. This is then weighted summed using the attention network to obtain a 300-dimensional intelligence title vector and a 300-dimensional intelligence summary vector. Furthermore, the intelligence encoder obtains a 100-dimensional vector representation of the intelligence source from the intelligence source ID, followed by a fully connected layer to calculate a 300-dimensional intelligence source vector. These three vectors are then concatenated to form a 900-dimensional intelligence representation.

[0062] This method transforms the recommendation problem into a 2+1 pseudo-classification problem by embedding candidate intelligence from the interaction data table, with a 2:1 ratio of positive to negative samples. A 900-dimensional representation of the user's long-term preferences is obtained through user ID embedding, which serves as the initial state of the LSTM network. An intelligence encoder is then used to calculate the representation of the intelligence in the compiled records and input this into the LSTM network, ultimately yielding the user's long-term and short-term preference representations. The intelligence encoder then calculates a vector representation of the candidate intelligence. The user representation and the vector representation are then combined to calculate a prediction score. The highest prediction score is used as the predicted positive sample, and the others are used as the predicted negative samples. During training, the sum of the negative log-likelihoods of all positive samples is minimized to improve model accuracy.

[0063] The results of intelligence recommendation on the test set were evaluated using the following evaluation metrics: AUC, MRR, nDCG@5, and nDCG@10. The baseline used the NAML model. The evaluation comparison between this embodiment and the baseline is shown in Table 4. As can be seen from Table 4, this method improves on all evaluation metrics compared to the baseline.

[0064] Table 4 Comparison of experimental results

[0065]

[0066]

[0067] AUC represents the model's classification ability, MRR represents the average ranking of relevant items in the recommendation list, and nDCG@5 and nDCG@10 represent the relevance of the top five and top ten items in the recommendation list, respectively. Larger values ​​are preferred for these evaluation metrics. The baseline model, NAML, stands for "multi-view learning-based approach."

[0068] The above is only a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the embodiment and the drawings. Any equivalent replacement or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method for recommending scientific and technological information based on long-term and short-term preferences, characterized by: The method comprises step S1, defining an encoder; and converting the intelligence title from a word sequence into a low-dimensional semantic vector sequence, specifically by: using a word embedding lookup table The word sequence of intelligence title [w1,w2,...,w N ], converted to a word vector sequence [e1,e2,...,e N ], where N is the length of the title, V and D are the vocabulary size and word embedding dimension respectively; a multi-head self-attention network is used to capture the interaction between words to learn the context representation of words, and the representation of the i-th word learned by the k-th attention head is calculated as: in, and is the projection parameter in the k-th self-attention head, Indicates the relative importance of the interaction between the i-th and j-th words, and the multi-head representation of the i-th word is the concatenation of the representations produced by h independent self-attention heads, i.e. Use the attention mechanism to select important words in intelligence titles, and the attention weight of the i-th word in the intelligence title The calculation method is: Among them, V w and v w is the projection parameter, q w is the query vector, and the final representation of the intelligence title is the weighted sum of the context representations of the words, i.e. Step S2, encoding the information into e=[r s ,r t ,r a ], where r s According to intelligence sources, t For information title, r a It is represented by intelligence summary; Step S2 includes encoding the intelligence summary by representing the word sequence of the intelligence summary as [w1, w2, ..., w M ], where M is the length of the summary, and then through the word embedding layer, it is converted into a word vector sequence [e1,e2,...,e M ]; Use a multi-head self-attention network, take the word vector sequence as the input of the network, and learn contextual word representation by capturing the interaction between words The attention network is used to calculate the attention weight of each word, and the intelligence summary representation is obtained by weighting the context representation of the summary word, that is, Encode the intelligence source by using the ID embedding method to encode the discrete representation v s Convert to low-dimensional dense representation e s ; Use fully connected layers to learn hidden source representations from source embeddings, calculated as r s =ReLu(V s ×e s +v s ), where V s and v s is the parameter of the fully connected layer; and the intelligence title representation, intelligence summary representation and intelligence source representation are spliced ​​together to obtain the intelligence representation, that is, e = [r s ,r t ,r a ]; Step S3: Use the user ID embedding method to represent the user's long-term preference. The user's long-term preference is represented as u l =W u [u], where u l As the initial state of the LSTM network, the LSTM network is used to learn the user's long-term and short-term representation u from the user's intelligence compilation history; Step S4: Calculate the dot product of the candidate intelligence representation vector and the user representation vector as the recommendation score.

2. The method for recommending scientific and technological information based on long-term and short-term preferences according to claim 1, characterized in that: The step S3 includes calculating the user's long-term preference representation using a method based on user ID embedding: u represents the user's ID, and W represents the user's long-term preference representation. u The query table represents the user's long-term preference, and the user's long-term preference is represented by u l =W u [u]; Use LSTM network to calculate the user's short-term preference representation from the user's recently compiled intelligence history: the user's intelligence compilation sequence sorted in ascending order by timestamp is represented as Where k is the sequence length; the intelligence encoder is used to obtain the representation of these intelligences {e1,e2,...,e k }; Calculate the user's short-term preference representation as follows: I t =σ(W i [h t-1 ,e t ]+b i ), f t =σ(W f [h t-1 ,e t ]+b f ), the t =σ(W o [h t-1 ,e t ]+b o ), c t =f t ☉c t-1 +i t ☉tanh(W c [h t-1 ,e t ]+b c ), h t =o t ⊙tanh(c t ) Among them, σ is the sigmoid activation function, ⊙ is the term-by-term multiplication, W i 、W f 、W o and W c is the weight matrix parameter of the LSTM network, b i 、b f , and b c is the corresponding bias parameter, i t 、f t and o t Represents the activation values ​​of the input gate, forget gate and output gate at the current moment, c t Indicates the cell state at the current moment, h t Represents the hidden state at the current moment. The user's short-term preference representation is the last hidden state of the LSTM network, i.e. u s =h k ; and represent the user's long-term preference as u l As the initial state of the LSTM network, the last hidden state of the LSTM network is used as the user's long-term and short-term preference representation u.

3. The method for recommending scientific and technological information based on long-term and short-term preferences according to claim 1, characterized in that: The step S4 includes calculating the recommendation score for the given candidate intelligence, denoting the user's long-term and short-term preferences as u, and the candidate intelligence as e. x , then the predicted score of the user clicking on the information is s(u,c x )=u T e x , where T is the transpose operation.

Images