An online question and answer community expert recommendation method based on a multi-head self-attention mechanism
By combining multi-head self-attention mechanism and convolutional neural network, the method dynamically captures changes in user interests, solving the problem of insufficient interest change capture in existing expert recommendation methods, and realizing accurate and personalized expert recommendations in online question-and-answer communities.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MINJIANG UNIVERSITY
- Filing Date
- 2022-07-27
- Publication Date
- 2026-04-21
AI Technical Summary
Existing expert recommendation methods struggle to effectively capture dynamic changes in users' interests, resulting in insufficient recommendation accuracy and an inability to fully integrate heterogeneous information from multiple sources, thus affecting recommendation quality.
A user encoder is constructed using a multi-head self-attention mechanism. By encoding features from the user's historical question answer sequences and topics of interest, and combining convolutional neural networks and attention mechanisms to extract question features, the encoder dynamically captures changes in user interests and uses a predictor to calculate similarity for recommendations.
It improves the accuracy and personalization of expert recommendations, can respond to changes in user interests in real time, and increases the question answering rate.
Smart Images
Figure CN115408603B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent recommendation technology, and in particular to an expert recommendation method for online question-answering communities based on a multi-head self-attention mechanism. Background Technology
[0002] With the popularization of the Internet, online Q&A communities have become important knowledge-sharing platforms. However, with the explosive growth of platform data, how to effectively recommend massive amounts of questions to expert users for answers is a serious challenge facing the platforms.
[0003] Commonly used expert recommendation methods mainly include link analysis and text analysis. Link analysis discovers experts through the question-and-answer relationships of community users; representative methods include webpage ranking and hyperlink topic search. Text analysis models user answer records, mines user interests, and calculates the matching degree between interest tags and question tags; representative methods include probabilistic latent semantic analysis models and latent Dirichlet distribution topic models. Additionally, some works transform the expert recommendation problem into a classification problem, using decision trees and support vector machines from machine learning to solve the problem, allowing for the application of multifaceted features to expert recommendation. The disadvantages of these methods are: 1. They rely on the quality of manually constructed complex features, hindering the scalability of recommendations. 2. They struggle to learn abstract, high-order feature interaction information. 3. They cannot incorporate heterogeneous multi-source information such as images and text, resulting in insufficient data mining.
[0004] In recent years, deep learning technology has continued to develop. Its advantage lies in enabling machines to learn and process complex problems like humans, extracting high-order feature interaction information through deep network structures without the need for manually constructing complex features. In expert recommendation applications using deep learning methods, the most popular are Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). CNNs excel at learning contextual features from input text, embedding user interests and given question words into feature representations, and then outputting the results through soft magnetic layers to predict whether a user is an expert. RNNs, on the other hand, excel at handling time-series features, learning dependencies between sequences. By abstracting users' historical answer information into sequence information and inputting it into the RNN, they capture dynamic changes in user interests. Representative models of this method include Long Short-Term Memory (LSTM) networks and Gated Recurrent Networks (GRNs). The disadvantages of these methods are: 1. They ignore short-term user interest drift in real-world scenarios and do not fully consider users' personalized needs, which affects recommendation quality. 2. Due to their unique sequential structure, RNNs can only be computed one by one, not in parallel, and are prone to losing information from the beginning of the sequence, affecting recommendation accuracy. Therefore, existing expert recommendation methods often ignore the dynamic changes in users' interests and cannot effectively extract dynamic interest features, thus reducing the accuracy of recommendations. Summary of the Invention
[0005] The technical problem to be solved by this invention is to provide an expert recommendation method for online question-and-answer communities based on a multi-head self-attention mechanism. This method extracts the dynamic interest representation of users through the multi-head self-attention mechanism and dynamically captures the short-term interest changes of users based on their historical question-answering sequences, thereby providing accurate, personalized, and real-time expert recommendation services for online question-and-answer communities and improving the question answering rate.
[0006] To solve the above-mentioned technical problems, the present invention is implemented as follows:
[0007] An expert recommendation method for online question answering communities based on multi-head self-attention mechanism includes: question encoder construction process, user encoder construction process, predictor construction process, deep learning model training process, and prediction process;
[0008] The problem encoder construction process includes: feature encoding of the problem, extracting information from the problem title and problem topic as problem labels, matching them with the interest labels of expert users; training all problem records in the dataset to construct the problem encoder;
[0009] The user encoder construction process includes: performing feature encoding on the user's historical question answer sequence and the user's topics of interest; extracting the user's dynamic interest feature information from the historical question answer sequence using a multi-head self-attention mechanism, then extracting the user's static interest feature information from the user's topics of interest, and concatenating the user's dynamic interest representation vector and static interest representation vector to obtain the user's comprehensive representation vector;
[0010] The predictor construction process includes: for a given question and an inviting user, determining whether the current user will accept the invitation to the question by calculating the similarity between the representation vector of the target question and the combined representation vector of the inviting user;
[0011] The deep learning model training process includes: collecting training data from the question-and-answer community; constructing training samples by combining user-question invitation records with user profiles, user answer records, and question description information; labeling the samples as 0 or 1; transforming expert identification questions into classification questions; and training the model using the training samples to obtain the trained model.
[0012] The prediction process includes: for a given question and an inviting user, calculating the similarity between the representation vector of the target question and the representation vector of the inviting user using a trained model, determining whether the current user will accept the invitation to the question, and thus generating an expert recommendation result.
[0013] Furthermore, the problem encoder construction process specifically includes:
[0014] Step A1: Through the embedding layer, the title of the question is segmented into words and word embedding representation is performed, which is then converted into word vector representation in the latent semantic space;
[0015] Suppose the words in the question title w are w = [w1, w2, ... w l ], where l represents the length of w, and the transformed word vector is represented as:
[0016] W=embedding(w)=[W1,W2,…,W l ]
[0017] Step A2: Use CNN to capture local semantic information and learn the representation of context words;
[0018] Assume the title word vector represented by the implicit context words is denoted as c. i The calculation formula is as follows:
[0019] c i =relu(C×W [i-M,i+M] +b)
[0020] The formula for the nonlinear activation function relu(·) is relu(x) = max(0,x), W [i-M,i+M] This refers to the word embedding connection located between iM and i+M, where C and b are the filter parameters of the convolutional neural network, and M is 1;
[0021] Step A3: Allocate the weight of title keywords through an attention mechanism;
[0022] Assume the attention weight for the i-th word is a. i The calculation formula is as follows:
[0023] α i =tanh(v×c i +v b )
[0024]
[0025] The formula for the tanh(·) function is: a i These are intermediate variables generated by the attention mechanism, v and v b These are trainable parameters;
[0026] The representation vector of the question title is obtained by weighting the context words from the CNN through attention weights, and its calculation formula is as follows:
[0027]
[0028] Step A4: Segment the topic to which the question belongs and perform word embedding representation, converting it into word vector representation in the latent semantic space;
[0029] Suppose the words in the topic t of the problem are t = [t1, t2, ... t l ], where l represents the length of t, and the transformed word vector is represented as:
[0030] T=embedding(t)=[T1,T2,…,T l ]
[0031] Step A5: Perform global average pooling on the word vectors of the topic to which the question belongs, and obtain the average word vector e. t The formula for representing the topic information of this question is as follows:
[0032] e t =V GAP T
[0033] Step A6: For each question, perform the above steps to generate vector representations of the question's title and its associated topic. Then, concatenate these vectors to generate the final question vector representation e, as shown in the following formula:
[0034] e = [e w ,e t ].
[0035] Furthermore, the user encoder construction process specifically includes:
[0036] Step B1: Mine the user's historical question answer sequence information. First, arrange the historical questions in chronological order; then, use the question encoder from the previous section to process each question in the sequence and obtain their question vector representations; finally, obtain the sequence E = [e1, e2, ..., e L ] T L is the sequence length;
[0037] Step B2: The multi-head self-attention mechanism adds a position vector P = [p (pos,0) p (pos,1) , ..., p (pos,2i) p (pos,2i+1) ] T This involves adding temporal information to the problem representation vector in the sequence, resulting in the sequence E' = [e1', e'2, ..., e']. L ] T The calculation formula is as follows:
[0038]
[0039] E' = E + P
[0040] Where pos is the position of the problem in the sequence, and its value ranges from [0,L]. d model That is, the dimension of the position vector;
[0041] Step B3: Input sequence E' into a multi-head self-attention mechanism network structure to capture the user's dynamic interest changes, and obtain a new sequence Z = [z1, z2, ..., z L ] T Each output element z i All are input elements e i User dynamic interest representation vector u learned through a multi-head self-attention mechanism d ;
[0042] Step B4: Learn the user's long-term interest representation vector through the topics the user follows;
[0043] Extract user-focused topics d = [d1, d2, ..., d2] from user information. J [J] represents the number of topics followed. First, word embeddings are obtained, and then global average pooling is performed to obtain the user's static interest vector u. s The calculation formula is as follows:
[0044]
[0045] Step B5: Concatenate the user's short-term dynamic interest representation vector and long-term static interest representation vector to obtain the user's comprehensive representation vector, as shown in the following formula:
[0046] u s+d =[u s ,u d ].
[0047] Further, in step B3, the multi-head self-attention mechanism network includes a multi-head self-attention network layer, a first residual connection and a layer normalization operation layer, a feedforward neural network layer, a second residual connection and a layer normalization operation layer, and a global average pooling layer; the multi-head self-attention network layer includes h parallel self-attention networks; the obtained sequence matrix is globally average pooled to obtain the user's dynamic interest representation vector u. d .
[0048] Furthermore, the predictor, given a question and an inviting user, calculates the target question q. x The representation vector e target and invited user u x The comprehensive representation vector u s+d The similarity score is used to determine whether the current user will accept the invitation to answer the question. The similarity calculation formula is:
[0049] score(u x ,q x ) = sigmoid(u s+d T e target )
[0050] The formula for the sigmoid(·) function is: The result is a probability score with values in the range [0,1].
[0051] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:
[0052] 1. Feature extraction of problem description information is performed using convolutional neural networks and attention mechanisms. Convolutional neural networks can accurately capture the contextual information of words, while attention mechanisms assign different degrees of attention to the extracted features. Combining the two methods to construct a problem encoder not only supplements the forgotten contextual information in the pre-trained model, but also emphasizes the important information of the problem, suppresses irrelevant information, and improves the accuracy of the algorithm.
[0053] 2. A multi-head self-attention mechanism is employed to process the user's historical question-answering sequences, dynamically capturing the feature information of interest changes inherent in the user's historical behavior. This structural characteristic avoids the loss of information at the beginning of the sequence while learning rich feature information. In addition, by combining the embedding representations corresponding to the user's long-term interest topics as a static interest representation, a user encoder that can effectively extract the user's dynamic and static interests is constructed.
[0054] 3. Use a pre-trained word embedding model to obtain word embedding representations of the question topic and question title. Perform a dot product operation on the user's comprehensive interest features and the target question features, and then obtain the similarity through a fully connected layer to generate recommendation results.
[0055] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0057] Figure 1 This is a schematic flowchart of the method of the present invention;
[0058] Figure 2 This is a schematic diagram of the overall framework of the algorithm of this invention;
[0059] Figure 3 This is a schematic diagram of the structure of the encoder of the present invention;
[0060] Figure 4 This is a schematic diagram of the structure of the user encoder of the present invention;
[0061] Figure 5 This is a schematic diagram of the network structure of the multi-head self-attention mechanism of the present invention;
[0062] Figure 6 This is a schematic diagram of the parallel structure of the multi-head self-attention layer of the present invention. Detailed Implementation
[0063] This invention provides an expert recommendation method for online question-and-answer communities based on a multi-head self-attention mechanism. By extracting dynamic interest representations of users through the multi-head self-attention mechanism, and dynamically capturing short-term interest changes of users based on their historical question-answering sequences, it provides accurate, personalized, and real-time expert recommendation services for online question-and-answer communities, thereby improving the question-answering rate.
[0064] The overall concept of the technical solutions in the embodiments of the present invention is as follows:
[0065] To address the issue of insufficient accuracy in expert recommendations within question-answering communities due to inadequate extraction of users' dynamic interest representations, this invention proposes a question-answering community expert recommendation algorithm that integrates multi-head self-attention. Its overall framework is as follows: Figure 2 As shown, the algorithm comprises three modules: a question encoder, a user encoder, and a predictor. On the question side, a convolutional neural network (CNN) combined with an attention mechanism is used to construct the question encoder, which processes the target question and the user's historical answers, extracting corresponding question representations from the question title and topic. On the user side, a user encoder is constructed that can simultaneously extract both static and dynamic interest representations. Its core network layer is a multi-head self-attention mechanism layer capable of capturing dynamic user interests. Based on the user's historical answer sequence, the multi-head self-attention mechanism learns the dynamic interest representations inherent in the sequence, and then combines them with the user's static interest representations to obtain the user's comprehensive interest representation. Finally, the predictor performs similarity calculations between the target question representation and the user's comprehensive representation to generate recommendation results.
[0066] By defining expert recommendation questions in Q&A communities as question-user matching problems, the goal is to obtain the question-user matching degree through a model built using a training set, and to determine whether a user will accept an invitation to answer a particular question. The specific problem description is as follows: Assume Q represents the set of questions, U represents the set of users, and a sample question is represented as q. x User samples are represented as u x Given the question - invite user pair [q] x ,u x ], tagged as y x∈{0,1}, where 0 indicates that the user has not received the question invitation, and 1 indicates that the user has received the invitation and answered it. There are N training samples [q]. x ,u x ,y x This constitutes the training dataset. A model f is built using the training dataset, and a loss function is defined to optimize the model. The model determines whether a user will accept an invitation to answer a certain question, i.e., the label y. x It can be 0 or 1.
[0067] like Figure 1 As shown, an expert recommendation method for online question answering communities based on multi-head self-attention mechanism includes: question encoder construction process, user encoder construction process, predictor construction process, deep learning model training process, and prediction process;
[0068] The problem encoder construction process includes: feature encoding of the problem, extracting information from the problem title and problem topic as problem labels, matching them with the interest labels of expert users; training all problem records in the dataset to construct the problem encoder;
[0069] The user encoder construction process includes: performing feature encoding on the user's historical question answer sequence and the user's topics of interest; extracting the user's dynamic interest feature information from the historical question answer sequence using a multi-head self-attention mechanism, then extracting the user's static interest feature information from the user's topics of interest, and concatenating the user's dynamic interest representation vector and static interest representation vector to obtain the user's comprehensive representation vector;
[0070] The predictor construction process includes: for a given question and an inviting user, determining whether the current user will accept the invitation to the question by calculating the similarity between the representation vector of the target question and the combined representation vector of the inviting user;
[0071] The deep learning model training process includes: collecting training data from the question-and-answer community; constructing training samples by combining user-question invitation records with user profiles, user answer records, and question description information; labeling the samples as 0 or 1; transforming expert identification questions into classification questions; and training the model using the training samples to obtain the trained model.
[0072] The prediction process includes: for a given question and an inviting user, calculating the similarity between the representation vector of the target question and the representation vector of the inviting user using a trained model, determining whether the current user will accept the invitation to the question, and thus generating an expert recommendation result.
[0073] In one specific embodiment, the specific implementation of each process is as follows.
[0074] I. Problem Encoder Construction Phase
[0075] 1) The title of the question is segmented into words using an embedding layer, and then represented by word embeddings. This is then converted into a word vector representation in the latent semantic space. Assume the words in the question title w are w = [w1, w2, ... w...]. l ], where l represents the length of w, and the transformed word vector is represented as:
[0076] W=embedding(w)=[W1,W2,…,W l (1)
[0077] 2) Utilize CNNs to capture local semantic information and learn representations of context words. Let c be the title word vector representing the implicit context words. i The calculation formula is as follows:
[0078] c i =relu(C×W [i-M,i+M] +b) (2)
[0079] The formula for the nonlinear activation function relu(·) is relu(x) = max(0,x), W [i-M,i+M] This refers to the word embedding connection located between iM and i+M, where C and b are the filter parameters of the convolutional neural network, and M is 1.
[0080] 3) Assigning weights to title words using an attention mechanism. In a question title, different words convey different amounts of information to represent the question. Let the attention weight of the i-th word be a. i The calculation formula is as follows:
[0081] α i =tanh(v×c i +v b (3)
[0082]
[0083] The formula for the tanh(·) function is: a i These are intermediate variables generated by the attention mechanism, v and v b These are trainable parameters.
[0084] The representation vector of the question title is the context word representation c obtained from the CNN. i via attention weight a i weighted e w The calculation formula is as follows:
[0085]
[0086] 4) Segment the topic to which the question belongs and perform word embedding representation, converting it into a word vector representation in the latent semantic space. Assume the words in the topic t are t = [t1, t2, ... t]. l ], where l represents the length of t, and the transformed word vector is represented as:
[0087] T=embedding(t)=[T1,T2,...,T l (6)
[0088] 5) Perform global average pooling on the word vectors of the topic to which a question belongs, and obtain the average word vector e. t The formula for representing the topic information of this question is as follows:
[0089] e t =V GAP T (7)
[0090] 6) For each question, perform the above steps to generate a vector representation of the question's title and its associated topic. Then, concatenate the vectors to generate the final question vector representation e, as shown in formula (8):
[0091] e = [e w ,e t (8)
[0092] The overall process of building a problem encoder is as follows: Figure 3 As shown.
[0093] II. User Encoder Construction Phase
[0094] 1) Mining the user's historical question-answering sequence information: First, arrange the historical questions in chronological order; then, use the question encoder from the previous section to process each question in the sequence and obtain their question vector representations; finally, obtain the sequence E = [e1, e2, ..., e L ] T L is the sequence length.
[0095] 2) The multi-head self-attention mechanism is achieved by adding P = [p (pos,0) p (pos,1) , ..., p (pos,2i) p (pos,2i+1) ] T This position vector representation adds temporal information to the problem representation vector in the sequence, resulting in the sequence E'=[e1',e'2,...,e' L ] T The calculation formula is as follows:
[0096]
[0097] E' = E + P (10)
[0098] Where pos is the position of the problem in the sequence, and its value ranges from [0,L]. d model That is, the dimension of the position vector.
[0099] 3) Input the sequence E' into the multi-head self-attention mechanism network structure to capture the user's dynamic interest changes, and obtain a new sequence Z = [z1, z2, ..., z L ] T Each output element z i All are input elements e i User dynamic interest representations learned through a multi-head self-attention mechanism. For example... Figure 5 As shown, the structure of the multi-head self-attention mechanism network embodiment includes a multi-head self-attention network layer (MHA), a first residual connection and layer normalization operation layer, a feed forward neural network layer (FFN), a second residual connection and layer normalization operation layer, and a global average pooling layer; wherein, the residual connection and layer normalization operation layer is abbreviated as Add&Norm layer.
[0100] 1. Multi-head self-attention network layer (MHA)
[0101] The MHA layer consists of h parallel self-attention networks, such as... Figure 6 As shown.
[0102] a. First, initialize the three weight matrices W. q W k W v As the model is trained, optimized, and updated, the weight matrices are multiplied by matrix E' to calculate matrices Q, K, and V, using the following formulas:
[0103] Q = E'W q (11)
[0104] K = E'W k (12)
[0105] V = E'W v (13)
[0106] Where Q represents the query matrix, K represents the key matrix, and V represents the value matrix. Each row of the matrix corresponds to the "query", "key", and "value" vectors of each question representation vector in matrix E'. Q and K have the same dimension.
[0107] b. Calculate the self-attention output of each head. i The calculation formula is as follows:
[0108]
[0109] The softmax() function normalizes the matrix. The scaling factor is used to make the normalization result more stable, and its size is K in dimension.
[0110] c. Perform h self-attention calculations in parallel, where h represents the number of heads. Concatenate the self-attention matrices obtained from each head together and then multiply by the weight matrix W. Y The final matrix Y is obtained, and its calculation formula is as follows:
[0111] Y=MHA(E')=concat(head1,head2,…,head h W Y (15)
[0112] The `concat()` function concatenates all self-attention output matrices horizontally. Y It is a linear transformation matrix.
[0113] 2 First Add & Norm Layer
[0114] Add operation:
[0115] E'+MHA(E') (16)
[0116] Layer Normalization operation:
[0117] Y'=LayerNorm(E'+MHA(E')) (17)
[0118] 3. Feedforward Neural Network Layer (FFN)
[0119] After the first fully connected layer, the ReLU(·) activation function is used, and its calculation formula is:
[0120] F1 = max(0, Y' * W) Y' +b Y' (18)
[0121] After the second fully connected layer, without using an activation function, the calculation formula is as follows:
[0122]
[0123] Among them, W Y' and b Y' Let be the weight matrix and bias of the first fully connected layer, and be the trainable parameters. and These are the weight matrix and bias of the second fully connected layer, and are also trainable parameters.
[0124] 4. Second Add & Norm layer
[0125] Add operation:
[0126] E'+FFN(Y') (20)
[0127] Layer Normalization operation:
[0128] Z=LayerNorm(E'+FFN(Y')) (21)
[0129] Finally, the resulting sequence matrix Z is globally averaged and pooled to obtain the user's dynamic interest representation vector u. d The calculation formula is as follows:
[0130]
[0131] 4) The topics users follow can be used to learn their long-term interest representation. The topics users follow, d = [d1, d2, ..., d...], can be extracted from user information. J ], J represents the number of topics of interest. First, use formula (6) to obtain the word embedding representation D = [D1, D2, ..., D J Then, using formula (7) for global average pooling, the user's static interest representation u is obtained. s The calculation formula is as follows:
[0132]
[0133] 5) Concatenate the user's short-term dynamic interest representation vector and long-term static interest representation vector to obtain the final representation vector on the user side, as shown in formula (24).
[0134] u s+d =[u s ,u d ] (twenty four)
[0135] The overall process of building a user encoder is as follows: Figure 4 As shown.
[0136] III. Predictor Construction Phase
[0137] Given a question and invited users, calculate the target question q. x The representation vector e target and invited user u x The representation vector u s+d The similarity score is used to determine whether the current user will accept the invitation to answer the question. The similarity calculation formula is:
[0138] score(u x ,q x ) = sigmoid(u s+d T e target (25)
[0139] The formula for the sigmoid(·) function is: The result is a probability score with values in the range [0,1].
[0140] IV. Deep Learning Model Training Phase
[0141] 1) Collect training data from the Q&A community. Construct training samples by combining user-question invitation records with user profiles, user answer records, and question descriptions. The sample labels are marked as 0 or 1. Transform expert identification questions into classification questions and process them intelligently using the proposed CQA expert recommendation method based on multi-head self-attention mechanism.
[0142] 2) The model was trained on the training samples to obtain the trained model. The model parameters were set as follows: word vector embedding dimension was 64. The CNN had 64 filters, a window stride of 1, and a window size of 3. The multi-head self-attention network was set to a single-layer structure with 2 heads, and the embedding matrix E was 6×64. In the final predictor module, there were 4 fully connected layers with progressively smaller neuron node sizes of 1024, 512, 256, and 128. The batch size was set to 2048, the loss function was binary cross-entropy, and the learning rate was 0.00001.
[0143] V. Forecasting Phase
[0144] Given a question and an inviting user, the similarity between the representation vector of the target question and the representation vector of the inviting user is calculated using the trained model. This determines whether the current user will accept the invitation to the question, thus generating expert recommendation results.
[0145] This invention utilizes convolutional neural networks (CNNs) and attention mechanisms to extract features from question descriptions. CNNs accurately capture the contextual information of words, while attention mechanisms assign different levels of attention to the extracted features. Combining these two methods to construct a question encoder not only supplements forgotten contextual information in the pre-trained model but also emphasizes important question information, suppresses irrelevant information, and improves the algorithm's accuracy. A multi-head self-attention mechanism is employed to process the user's historical question-answering sequences, dynamically capturing the feature information of interest changes inherent in the user's historical behavior. This structural characteristic avoids the loss of information at the beginning of the sequence while learning rich feature information. Furthermore, by combining the embedding representations corresponding to the user's long-term interest topics as a static interest representation, a user encoder that effectively extracts the user's dynamic and static interests is constructed. A pre-trained word embedding model is used to obtain word embedding representations of the question topic and question title. The user's comprehensive interest features and target question features are then multiplied by a dot product, and similarity is obtained through a fully connected layer to generate recommendation results.
[0146] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. An online question and answer community expert recommendation method based on a multi-head self-attention mechanism, characterized in that, include: The process of building the problem encoder, the process of building the user encoder, the process of building the predictor, the process of training the deep learning model, and the prediction process; The problem encoder construction process includes: feature encoding of the problem, extracting information from the problem title and problem topic as problem labels, matching them with the interest labels of expert users; training all problem records in the dataset to construct the problem encoder; The user encoder construction process includes: performing feature encoding on the user's historical question answer sequence and the user's topics of interest; extracting the user's dynamic interest feature information from the historical question answer sequence using a multi-head self-attention mechanism, then extracting the user's static interest feature information from the user's topics of interest, and concatenating the user's dynamic interest representation vector and static interest representation vector to obtain the user's comprehensive representation vector; The user encoder construction process specifically includes: Step B1, excavate the user's historical answer question sequence information, first arrange the historical answer questions according to time; then use the question encoder of the last section to process each question in the sequence to obtain their question vector representation; finally, obtain the sequence , L is the sequence length; Step B2, multi-head self-attention mechanism by adding position vector representing the sequence representation vector adding timing information to the sequence , the calculation formula is: ; ; wherein, pos is the position of the problem in the sequence, taking values in the range , , i.e. the position vector dimension; Step B3, put the sequence A multi-head self-attention mechanism network structure is used to capture dynamic changes in user interest and obtain new sequences. Each output element z i All are input elements e i User dynamic interest representation vector learned through a multi-head self-attention mechanism u d ; Step B4: Learn the user's long-term interest representation vector through the topics the user follows; Extracting user attention topics from user information , J The number of topics of interest is represented, the word embedding representation is obtained first, and then global average pooling is performed to obtain the static interest representation vector of the user u s The calculation formula is: ; wherein, is a parameter matrix for word embedding representation D global average pooling; Step B5, the short-term dynamic interest representation vector and the long-term static interest representation vector of the user are spliced to obtain a comprehensive representation vector of the user, as follows: ; The predictor construction process includes: for a given question and an inviting user, determining whether the current user will accept the invitation to the question by calculating the similarity between the representation vector of the target question and the combined representation vector of the inviting user; The deep learning model training process includes: collecting training data from the question-and-answer community; constructing training samples by combining user-question invitation records with user profiles, user answer records, and question description information; labeling the samples as 0 or 1; transforming expert identification questions into classification questions; and training the model using the training samples to obtain the trained model. The prediction process includes: for a given question and an inviting user, calculating the similarity between the representation vector of the target question and the representation vector of the inviting user using a trained model, determining whether the current user will accept the invitation to the question, and thus generating an expert recommendation result.
2. The method of claim 1, wherein: The problem encoder construction process specifically includes: Step A1: Through the embedding layer, the title of the question is segmented into words and word embedding representation is performed, which is then converted into word vector representation in the latent semantic space; Assuming the question title The words in the text are , represent The length of , the transformed word vector is represented as: ; Step A2: Use CNN to capture local semantic information and learn the representation of context words; Let the title vector represented by the implicit contextual words be denoted as The calculation formula is as follows: ; where the non-linear activation function is given by , denotes the concatenation of word embeddings between and , C and b are filter parameters of a convolutional neural network, M is 1; Step A3: Allocate the weight of title keywords through an attention mechanism; Assume the attention weight of the i-th word is i a i The calculation formula is: ; wherein, The function formula is , a i is an intermediate variable produced by the attention mechanism, v and v b are trainable parameters; The representation vector of the question title is the context word representation obtained by the CNN, weighted by the attention weight, and the calculation formula is ; Step A4: Segment the topic to which the question belongs and perform word embedding representation, converting it into word vector representation in the latent semantic space; Assuming the topic of the problem t The words in the text are , l represent t The length of , the transformed word vector is represented as: ; Step A5, global average pooling is performed on the word vectors of the subject of the question to obtain an average word vector e t to represent the subject information of the question, and the calculation formula is: ; wherein, is a parameter matrix of global average pooling; Step A6. For each question, perform the above steps to produce a vector representation of the title of the question and the subject matter to which it belongs, and then produce a final vector representation of the question by vector concatenation e as follows: .
3. The method of claim 1, wherein, In step B3, the multi-head self-attention mechanism network includes a multi-head self-attention network layer, a first residual connection and a layer normalization operation layer, a feedforward neural network layer, a second residual connection and a layer normalization operation layer, and a global average pooling layer; the multi-head self-attention network layer includes h A parallel self-attention network is used to perform global average pooling of the resulting sequence matrix to obtain the user's dynamic interest representation vector. u d .
4. The method of claim 1, wherein: The predictor calculates the target question based on a given question and inviting user. Representation vector and invited users The comprehensive representation vector The similarity score is used to determine whether the current user will accept the invitation to answer the question. The similarity calculation formula is: ; wherein, The formula for the function is with the result being a probability score with a value range in [0, 1].