Decoupling representation microblog topic detection method for enhancing interpretability
By building a user-level dialogue network and a multi-grained attention framework, combining the decoupled representation module and the generation network, the problems of difficult to interpret and data sparseness in Weibo topic mining are solved, and the topic reasoning is higher interpretability and Continuity improves the effectiveness of Weibo topic detection.
Patent Information
- Application Number
- CN202410927400.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-12
AI Technical Summary
Existing Weibo topic mining technology is difficult to generate potential representations with strong interpretation, and faces data sparsity, which makes it difficult to achieve effective detection of coherent topics in Weibo text processing.
Build a user-level dialogue network based on forwarding and comment relationships, adopting a multi-grained attention framework and a two-layer graph convolutional neural network, combining the decoupled representation module and the generation network, extracting potential generation factors and adding sparse constraints to generate more in line with real social interactions Distribution of topics about media features.
The interpretability and coherence of topic reasoning are improved, and by more comprehensively considering social network structural information and text content, the generated Weibo topics perform better than the existing model in terms of correlation evaluation indicators.
Smart Images

Figure CN118940794A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing and social media data mining, and in particular to a decoupled representation microblog topic detection method with enhanced interpretability. Background Art
[0002] The emergence of social media websites (such as Sina Weibo) has greatly changed the form of content on the Internet. Weibo allows users to publish and browse information on it, and has strong social attribute functions such as forwarding and commenting. There is a huge amount of text data stored on the Weibo platform, and it is growing at an alarming rate every day. Weibo text content contains a lot of information, and mining topic information from it can be used for topic recommendation, emergency detection, precision marketing, etc. At present, text topic mining technology has achieved good results in long text data such as news and articles. However, the length of Weibo text is short, usually limited to less than 140 words. The sparseness of Weibo information and the randomness of Weibo words have greatly increased the difficulty of Weibo text processing. Therefore, topic mining technology for Weibo needs to adopt a different method from traditional topic mining.
[0003] At present, the research on microblog topic mining mainly includes: (1) Based on aggregation strategy. This method aggregates short texts into a pseudo-long document according to certain rules, reducing the data sparsity of short texts. For example, the microblogs published by the same author are first merged into a long document, and then the three-layer Bayesian structure topic model is used to mine potential topics; or the generation of word pairs in the entire corpus is directly modeled, which reduces the data sparsity of short texts; (2) Using word embedding. This method uses the characteristics of word embeddings that contain rich semantic information to model topics, which to a certain extent solves the problem of short text lacking semantic information. In order to deeply understand the semantics of short texts, this method regards short texts as a set of word embeddings, and assumes that the topic-word distribution is a multi-dimensional Gaussian distribution, and then uses a hierarchical Bayesian model to infer topics; (3) Using social network context information. This type of method introduces the structural characteristics of social networks, such as user attention networks and user comment forwarding relationship networks, to supplement static and dynamic context information for microblog text content, thereby inferring topics.
[0004] Although the above methods have achieved good performance, in most cases, the latent representations learned by standard VAE are difficult to interpret and cannot be flexibly applied to specific topic reasoning tasks. Therefore, improving the control strength of the model over the latent representation in the topic reasoning stage and improving the interpretability of the representation are critical to improving topic models in specific fields. In addition, the Weibo dataset has a sparsity problem and can provide little information, which requires that the amount of data required for model learning be as small as possible under the premise of generating coherent topics. Summary of the invention
[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a decoupled representation microblog topic detection method with enhanced interpretability. The method constructs a user-level conversation network based on forwarding and commenting relationships, where nodes represent users and edges represent forwarding or commenting relationships between pairs of users.
[0006] Secondly, we designed a multi-granularity text representation method and built a multi-granularity attention framework. [1] Among them, word-level attention alleviates the noise impact caused by colloquialism, user-level attention models the influence of different users on topics, obtains hierarchical text representation, and fuses structural information through a two-layer graph convolutional neural network to generate user node representation; the decoupled representation module is used to extract the potential generative factors that affect the topic from the topic vector, and the generative network is used to reconstruct the node embedding of the decoupled generative factors. [2] , obtain the topic word distribution, and add sparse constraints to the topic word distribution to make it more consistent with the characteristics of real social media data, and finally generate coherent topics. Compared with the existing model, the microblog topics mined by this method are the best in the coherence score evaluation index [3].
[0007] The objective of the present invention is achieved through the following technical solutions:
[0008] A decoupled representation microblog topic detection method with enhanced explainability includes the following steps:
[0009] (1) Build a user-level dialogue network;
[0010] (2) Build a multi-granular attention framework, in which word-level attention is used to alleviate the noise impact caused by colloquialism, and user-level attention is used to model the influence of different users on the topic, obtaining a hierarchical text representation, which is then fused with the adjacency matrix that stores structural information to generate a user node representation;
[0011] (3) The user node representation is used as the input of neural variational inference, and the variational autoencoder is used as the inference framework. The inference network is used to encode the node information to generate a more coherent topic vector;
[0012] (4) The decoupled representation module is used to extract the potential generative factors that influence the topic from the topic vector. The generative network is used to reconstruct the node embedding of the decoupled generative factors to obtain the topic word distribution. Sparse constraints are added to the topic word distribution to make it more consistent with the characteristics of real social media data.
[0013] Furthermore, step (1) specifically includes:
[0014] The user is regarded as a node in the dialogue network. At the same time, all microblogs related to the corresponding user, including source microblogs, comment microblogs, and forwarded microblogs, are aggregated into documents and regarded as the text information of the node. If there are microblog forwarding or comments between users in the dialogue network, the nodes representing the users are connected. This operation constructs a user-level dialogue network G = (V, E, T), where V is the set of nodes in the dialogue network G, and v i It means V={v i |1≤v i ≤L}; is the set of edges in the dialogue network G, e(i,j)∈E, representing the node v i and node v j There is a comment forwarding relationship between them; T = {t x1 ,t x2 ,...,t xL} is a collection of text information attached to the node, each t xi (i=1,2…L) contains M words; L is the number of users.
[0015] Furthermore, step (2) specifically includes:
[0016] In the user-level dialogue network, the attribute matrix X∈R L×C , R is a matrix, L is the number of users, and C is the dimension of the user node representation; replace each word with its corresponding low-dimensional, continuous and real-valued embedding, and make a weighted average of X, the adjacency matrix A∈R L×L Contains e(i,j), e(i,j)∈E, is the set of edges in the dialogue network G, V is the set of nodes in the dialogue network G; D is the degree matrix of the adjacency matrix A; and They are the degree matrix and adjacency matrix with self-loop added, see formula (1):
[0017]
[0018] Among them, I n is a self-connection matrix, S is a symmetric transfer matrix, simulating random walk, see formula (2):
[0019]
[0020] For user v i , word embedding matrix W i ∈R M×L , M represents the Mth word; W q , W k , W v are different weight matrices, the query matrix Q of the word-level attention layer wi , key matrix Kwi Sum value matrix V wi The calculation is shown in formula (3):
[0021] Q wi =W q ·W i K wi =W k ·W i V wi =W v ·W i (3)
[0022] The word-level attention mechanism reduces the noise impact of colloquial vocabulary from a fine-grained level, uses softmax(·) for normalization, highlights the weights of important elements, and calculates the weight representation X of each word. wi , see formula (4):
[0023]
[0024] C is the dimension of user node representation. The user-level attention mechanism is introduced to learn the user's contribution to the topic distribution. The influence of different users on the topic is converted into the corresponding hierarchical text representation X calculated using the attention mechanism. r :
[0025] Q s =W q ·X s K s =W k ·X s V s =W v ·X s (5)
[0026]
[0027] In formula (5) and (6), input X s ={X w1 ,X w2 ,...,X wL}, first embed the user into X s Perform nonlinear transformation to obtain the user-level key matrix K s , followed by the user attention vector Q s Calculate the similarity and further normalize it through the softmax function to get user v i The importance of the topic and the user's own semantics V s Multiply to get the hierarchical text representation X r ;
[0028] Introducing a two-layer graph convolutional network to aggregate neighbor node representations containing relationship information and hierarchical text representation X r , learning node representation:
[0029]
[0030]
[0031]
[0032] In formulas (7), (8), and (9), W1 and W2 are the parameters of the first and second layers of the graph convolutional network, respectively; f(·) takes ReLU(·) as the nonlinear activation function; the neighbor node representation is updated The graph convolutional network aggregates the information sent by each node from its neighbors; H1 represents the user node embedding representation obtained after the first layer of the graph convolutional network GCN; H2 combines the hierarchical text representation and the neighbor node representation; the loss function of the graph convolutional network GCN is shown in (10), which minimizes the following objective function so that adjacent nodes contain more common information:
[0033]
[0034] Among them, v i represents the fusion representation of the i-th node, v j represents the fusion representation of the jth node, v j The selection range is v i The conditional probability p(v j |v i ) is formalized as shown in formula (11):
[0035]
[0036] Among them, h j is taken from H2, user v i Neighbor v j ∈N i , cos(·,·) represents the cosine similarity.
[0037] Furthermore, step (3) specifically includes:
[0038] The neural variational inference framework is used to infer the user-topic distribution θ = (p(t1|e), p(t2|e), …, p(t K |e)) and topic-word distribution φ w =(p(w|t1),p(w|t2),…,p(w|t K )), where e represents the user, t i represents the i-th topic, K represents the number of topics, and w represents words; p(t i|e)(i=1,2,..,K) represents the probability that user e belongs to the i-th topic, p(w|t i )(i=1,2,..,K) represents the probability that word w belongs to the i-th topic;
[0039] User-topic distribution: Given a user embedding h i , first map it into the nonlinear latent space e:
[0040] e=ReLU(W e ·h i +b e ) (12)
[0041] Among them, W e and b e is the parameter of the encoder, ReLU is the nonlinear activation function; assuming that the user embeds h i The prior distribution and posterior distribution of are both Gaussian distributions, then the mean μ and variance σ of the posterior Gaussian distribution are 2 Obtained through linear transformation, see formula (13)(14):
[0042] μ=W μ ·e+b μ (13)
[0043] log(σ 2 )=W σ ·e+b σ (14)
[0044] Among them, W μ , b μ , W σ , b σ is the parameter of the encoder; the latent semantic vector z∈R is further obtained by using the reparameterization technique K , the formalization is shown in formula (15):
[0045] z=μ+∈×σ (15)
[0046] Among them, ∈ is sampled from the Gaussian distribution N(0,I), and the latent semantic vector z is normalized and calculated using the sparsemax(·) function w , to achieve its sparse effect;
[0047] Topic-word distribution: Topic-word distribution in the text φ w It can be regarded as the parameter of the decoder, see formula (16) for details:
[0048] d = sparsemax(φ w ×(θ) T ) (16)
[0049] Afterwards, h is reconstructed by the decoder i , and obtain the reconstructed user embedding h′ i , see formula (17):
[0050] h′ i =ReLU(W d d+b d ) (17)
[0051] Where W d 、b d These are all decoder parameters;
[0052] For topic generation, the objective function of this part is shown in formula (18):
[0053]
[0054] In formula (18), L V It is expressed as the loss function formula for learning user-topic distribution and topic-word distribution, and uses KL divergence to measure the prior distribution q(z) and the true posterior distribution p(z|h′ i ), where q(z) satisfies the prior Gaussian distribution N(0,I);
[0055] By combining formulas (10) and (18), the overall objective function is defined to mine potential topics on Weibo, see formula (19), where λ is the trade-off L G and L V Hyperparameters:
[0056] L=(1-λ)L G +λL V (19).
[0057] Furthermore, step (4) specifically includes:
[0058] To extract the generative factor, we first make each dimension in the topic vector z independent of each other and add a control parameter β to the KL term, as shown in (20):
[0059]
[0060] The increase of β reduces the KL term, making p(z|h′ i ) is closer to q(z), encouraging the encoder to extract more relevant information and achieve dimension separation. After separation, each dimension in the topic vector corresponds to an independent potential generation factor. According to the information bottleneck theory, this objective function maximizes the information bottleneck z and the output h′ of the downstream task. i The mutual information between them, and filter out h′ i may exist in the input h iThe irrelevant information in is as shown in formula (21):
[0061] max[I(z;h′ i )-β·I(h i ;z)] (21)
[0062] Where I(·;· represents mutual information and β is the Lagrange multiplier.
[0063] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the decoupled representation microblog topic detection method with enhanced interpretability are implemented;
[0064] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the decoupled representation microblog topic detection method for enhancing interpretability.
[0065] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:
[0066] (1) In order to alleviate the problem of short posts and informal expressions in social media, the method of the present invention simultaneously considers the post content and social network structure information to model a more complete social context.
[0067] (2) In order to overcome the problem that topics generated by traditional topic models are logically coherent but do not meet users' expectations and are difficult to understand, the present invention comprehensively considers the coherence score and the interpretability of the topic, and proposes a decoupled representation microblog topic detection method with enhanced interpretability to more comprehensively evaluate the quality of the topic model.
[0068] (3) In order to separate the multiple potential generative factors that drive the generation of topic representation from each other, the present invention adjusts the weight of the regularization loss in the topic reasoning stage to achieve the decoupling effect of topic representation.
[0069] (4) Experimental results on a real Sina Weibo dataset demonstrate the effectiveness of the proposed method and prove the effectiveness of decoupled representation for Weibo topic detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0070] Figure 1 A framework diagram of the decoupled representation microblog topic detection method for enhancing interpretability provided by the present invention; wherein Figure 1 It consists of two parts: user node representation learning module and topic reasoning module. The topic reasoning module also includes reasoning network, generation network and decoupled representation module.
[0071] Figure 2This is the variation of the topic coherence score with the Lagrangian coefficient β on the June data set in a specific implementation method. DETAILED DESCRIPTION
[0072] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0073] Taking three real microblog data sets as examples, the specific implementation method of the present invention is given. The overall framework of the method is as follows: Figure 1 The entire system algorithm flow includes four steps: user-level dialogue network input, multi-granular text representation and fusion, topic generation based on neural variational reasoning, and topic reasoning with decoupled representation.
[0074] The specific steps are as follows:
[0075] (1) User-level dialogue network input
[0076] Previous researchers used the hashtag-search API to collect relevant microblogs covering 50 hot topics in May, June and July 2014 on the Sina Weibo platform. [4] The present invention uses these three commonly used real microblog datasets as the original corpus and performs preprocessing according to the following steps to construct a user-level dialogue network: 1) filtering users who have no forwarding or commenting relationship; 2) aggregating all microblogs related to the same user, including source microblogs, forwarded microblogs, and commented microblogs, into a document as the text information of the node referred to by the user.
[0077] Table 1 shows the detailed statistical information of the three datasets, as follows: the May dataset includes a total of 44,395 users, of which there are 64,292 pairs of users with forwarding or commenting relationships, totaling 70,893 microblogs; the June dataset includes a total of 89,979 users, of which there are 151,427 pairs of users with forwarding or commenting relationships, totaling 163,420 microblogs; the July dataset includes a total of 119,269 users, of which there are 178,154 pairs of users with forwarding or commenting relationships, totaling 188,657 microblogs.
[0078] Table 1 Statistics of Weibo dataset
[0079]
[0080]
[0081] (2) Multi-granularity text representation and fusion
[0082] In the user-level dialogue network, the attribute matrix X∈R L×C, R is a matrix, L is the number of users, and C is the dimension of the user node representation; replace each word with its corresponding low-dimensional, continuous and real-valued embedding, and make a weighted average of X, the adjacency matrix A∈R L×L Contains e(i,j), e(i,j)∈E, is the set of edges in the dialogue network G, V is the set of nodes in the dialogue network G; D is the degree matrix of the adjacency matrix A; and They are the degree matrix and adjacency matrix with self-loop added, see formula (1):
[0083]
[0084] Among them, I n is a self-connection matrix, S is a symmetric transfer matrix, simulating random walk, see formula (2):
[0085]
[0086] For user v i , word embedding matrix W i ∈R M×L , M represents the Mth word; W q , W k , W v are different weight matrices, the query matrix Q of the word-level attention layer wi , key matrix K wi Sum value matrix V wi The calculation is shown in formula (3):
[0087] Q wi =W q ·W i K wi =W k ·W i V wi =W v ·W i (3)
[0088] The word-level attention mechanism reduces the noise impact of colloquial vocabulary from a fine-grained level, uses softmax(·) for normalization, highlights the weights of important elements, and calculates the weight representation X of each word. wi , see formula (4):
[0089]
[0090] C is the dimension of user node representation. The user-level attention mechanism is introduced to learn the user's contribution to the topic distribution. The influence of different users on the topic is converted into the corresponding hierarchical text representation X calculated using the attention mechanism. r :
[0091] Q s =W q ·X s K s =W k ·X s V s =W v ·X s (5)
[0092]
[0093] In formula (5) and (6), input X s ={X w1 ,X w2 ,...,X wL}, first embed the user into X s Perform nonlinear transformation to obtain the user-level key matrix K s , followed by the user attention vector Q s Calculate the similarity and further normalize it through the softmax function to get user v i The importance of the topic and the user's own semantics V s Multiply to get the hierarchical text representation X r ;
[0094] Introducing a two-layer graph convolutional network [5] Aggregate neighbor node representation containing relationship information and hierarchical text representation X r , learning node representation:
[0095]
[0096]
[0097]
[0098] In formulas (7), (8), and (9), W1 and W2 are the parameters of the first and second layers of the graph convolutional network, respectively; f(·) takes ReLU(·) as the nonlinear activation function; the neighbor node representation is updated The graph convolutional network aggregates the information sent by each node from its neighbors; H1 represents the user node embedding representation obtained by the first layer of GCN; H2 combines the hierarchical text representation and the neighbor node representation; the loss function of the graph convolutional network GCN is shown in (10), which minimizes the following objective function so that adjacent nodes contain more common information:
[0099]
[0100] Among them, v irepresents the fusion representation of the i-th node, v j represents the fusion representation of the jth node, v j The selection range is v i The conditional probability p(v j |v i ) is formalized as shown in formula (11):
[0101]
[0102] Among them, h j is taken from H2, user v i Neighbor v j ∈N i , cos(·,·) represents the cosine similarity.
[0103] (3) Topic Generation Based on Neural Variational Inference
[0104] Using Neural Variational Inference [6] The framework infers the user-topic distribution θ = (p(t1|e), p(t2|e), …, p(t K |e)) and topic-word distribution φ w =(p(w|t1),p(w|t2),…,p(w|t K )), where e represents the user, t i represents the i-th topic, K represents the number of topics, and w represents words; p(t i |e)(i=1,2,..,K) represents the probability that user e belongs to the i-th topic, p(w|t i )(i=1,2,..,K) represents the probability that word w belongs to the i-th topic;
[0105] User-topic distribution: Given a user embedding h i , first map it into the nonlinear latent space e:
[0106] e=ReLU(W e ·h i +b e ) (12)
[0107] Among them, W e and b e is the parameter of the encoder, ReLU is the nonlinear activation function; assuming that the user embeds h i The prior distribution and posterior distribution of are both Gaussian distributions, then the mean μ and variance σ of the posterior Gaussian distribution are 2 Obtained through linear transformation, see formula (13)(14):
[0108] μ=W μ ·e+bμ (13)
[0109] log(σ 2 )=W σ ·e+b σ (14)
[0110] Among them, W μ , b μ , W σ , b σ is the parameter of the encoder; the latent semantic vector z∈R is further obtained by using the reparameterization technique K , the formalization is shown in formula (15):
[0111] z=μ+∈×σ (15)
[0112] Among them, ∈ is sampled from the Gaussian distribution N(0,I), and the latent semantic vector z is normalized and calculated using the sparsemax(·) function w , to achieve its sparse effect;
[0113] Topic-word distribution: Topic-word distribution in the text φ w It can be regarded as the parameter of the decoder, see formula (16) for details:
[0114] d = sparsemax(φ w ×(θ) T ) (16)
[0115] Afterwards, h is reconstructed by the decoder i , and obtain the reconstructed user embedding h′ i , see formula (17):
[0116] h′ i =ReLU(W d d+b d ) (17)
[0117] Where W d , b d These are all decoder parameters;
[0118] For topic generation, the objective function of this part is shown in formula (18):
[0119]
[0120] In formula (18), L V It is expressed as the loss function formula for learning user-topic distribution and topic-word distribution, and uses KL divergence to measure the prior distribution q(z) and the true posterior distribution p(z|h′ i ), where q(z) satisfies the prior Gaussian distribution N(0,I);
[0121] By combining formulas (10) and (18), the overall objective function is defined to mine potential topics on Weibo, see formula (19), where λ is the trade-off L G and L V Hyperparameters:
[0122] L=(1-λ)L G +λL V (19).
[0123] (4) Topic Reasoning with Decoupled Representations
[0124] To extract the generative factor, we first make each dimension in the topic vector z independent of each other and add a control parameter β to the KL term, as shown in (20):
[0125]
[0126] The increase of β reduces the KL term, making p(z|h′ i ) is closer to q(z), encouraging the encoder to extract more relevant information and achieve dimension separation. After separation, each dimension in the topic vector corresponds to an independent potential generation factor. According to the information bottleneck theory, this objective function maximizes the information bottleneck z and the output h′ of the downstream task. i The mutual information between them, and filter out h′ i may exist in the input h i The irrelevant information in is as shown in formula (21):
[0127] max[I(z;h′ i )-β·I(h i ;z)] (21)
[0128] Where I(·;· represents mutual information and β is the Lagrange multiplier.
[0129] In the specific implementation process, by setting various hyperparameters in advance, that is, when the embedding dimension is 200, the weight coefficient λ is 0.5, and β is 4, the topic coherence is the highest. Figure 2 As shown in the figure, when β is less than 4, as β increases, the encoder learns dimensionally decoupled topic representations, and the coherence score improves; when β is greater than 4, the reconstruction loss is weakened, which widens the gap between the topic representation and the original input, and the coherence score decreases.
[0130] In order to verify the effectiveness of the method of the present invention, the method of the present invention (DRTM) is compared with the current advanced and representative models (LCTM, LeadLDA, IATM, DGTM) and two variants of the method of the present invention (DRTM (-disentangled representation), CRTM (-sparsity constraints)).
[0131] LCTM (Latent Concept Topic Model) [7] In order to solve the data sparsity problem of short texts, word embedding is introduced to enhance the understanding of the semantics of short texts. The model also introduces a new latent variable - concept to capture the semantic similarity of words, and assumes that topics are distributed on concepts and concepts are distributed on word embeddings.
[0132] LeadLDA [8] A dialogue tree is constructed based on the forwarding and replying relationships between microblogs, and then hidden topics are inferred based on the topic dependency relationship between leader messages and follower messages on the dialogue tree.
[0133] IATM (InteractionAware Topic Model) [9] It considers both text content and dynamic user behaviors in social networks by modeling dynamic user interactions and different user concerns, and then uses neural variational reasoning to mine topics.
[0134] DGTM (Topic Model with Dual-streams Graph Convolution Networks)
[10] At the same time, the characteristics of wide and deep dissemination of topics in social media are taken into account.
[0135] DRTM (-disentangled representation) only adds sparse constraints to the distribution of topic words, making it more consistent with real social media data.
[0136] DRTM (-sparsity constraints) learns to decouple topic embedding, improves the interpretability of representation, and further controls the weights of the generated factors in the network, thereby enhancing controllability.
[0137] The evaluation index of experimental performance is the topic coherence score, and the formula is as follows:
[0138]
[0139] Tables 2, 3, and 4 show the topic coherence results of the model and all the comparison methods on three microblog datasets. For each dataset, the coherence scores of the first 10 (N=10), 15 (N=15), and 20 (N=20) words of the inferred topic are recorded when the number of topics K=50 and 100 respectively. The higher the topic coherence score, the better the performance of the model.
[0140] Table 2 Performance comparison of the proposed method and other methods on the May dataset
[0141]
[0142]
[0143] Table 3 Performance comparison of the proposed method and other methods on the June dataset
[0144]
[0145] Table 4 Performance comparison of the proposed method and other methods on the July dataset
[0146]
[0147] From the topic coherence results in Tables 2, 3, and 4, it can be seen that the decoupled representation module has learned the separation dimension. During the node reconstruction process, it automatically selected the generation factors that have a significant impact on the results, and fundamentally identified the generation factors that control the spread of topics. The average number of interactions per user in the June dataset is the highest, the network connection is closer, and the structural characteristics have a greater impact on the spread of topics, so the decoupled representation is more sufficient.
[0148] Preferably, an embodiment of the present application also provides a specific implementation of an electronic device capable of implementing all steps in a decoupled representation microblog topic detection method with enhanced interpretability in the above embodiment, and the electronic device specifically includes the following contents:
[0149] Processor, memory, communications interface and bus;
[0150] Among them, the processor, memory, and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as server-side devices, metering devices, and user-side devices.
[0151] The processor is used to call the computer program in the memory, and when the processor executes the computer program, all the steps in the decoupled representation microblog topic detection method with enhanced interpretability in the above embodiment are implemented.
[0152] An embodiment of the present application also provides a computer-readable storage medium capable of implementing all the steps in a decoupled representation microblog topic detection method with enhanced interpretability in the above-mentioned embodiment. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it implements all the steps in a decoupled representation microblog topic detection method with enhanced interpretability in the above-mentioned embodiment.
[0153] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0154] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0155] Although the present application provides method operation steps such as embodiments or flow charts, more or fewer operation steps may be included based on conventional or non-creative labor. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment).
[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.
[0158] The above contents are intended to illustrate the technical solutions of the present invention schematically, and the present invention is not limited to the embodiments described above. Without departing from the scope of the present invention and the scope protected by the claims, a person skilled in the art may make many specific changes under the guidance of the present invention, which are all within the protection scope of the present invention.
[0159] References:
[0160] [1]Liu D, Greene D, Dong RA Novel Perspective to Look At Attention: Bi-level Attention-based Explainable Topic Modeling for News Classification[J].arXiv preprint arXiv:2203.07216,2022.
[0161] [2]Higgins I, Matthey L, Pal A, et al. beta-vae: Learning basic visual concepts with a constrained variational framework [J]. ICLR (Poster), 2017, 3.
[0162] [3]Chang J, Gerrish S, Wang C, et al. Reading tea leaves: How humans interpret topic models [J]. Advances in neural information processing systems, 2009, 22.
[0163] [4]Li J, Liao M, Gao W, et al. Topic extraction from microblog posts using conversation structures [M] / / Social Media Content Analysis: NaturalLanguage Processing and Beyond. 2018: 419-437.
[0164] [5]Kipf T N,Welling M.Semi-supervised classification with graphconvolutional networks[J].arXiv preprint arXiv:1609.02907,2016.
[0165] [6]Srivastava A,Sutton C.Autoencoding Variational Inference for TopicModels.In:Proc.of the 5th International Conference on LearningRepresentations,2017.
[0166] [7]Hu W,Tsujii J.A latent concept topic model for robust topicinference using word embeddings[C] / / Proceedings of the 54th Annual Meeting ofthe Association for Computational Linguistics(Volume 2:Short Papers).2016:380-386.
[0167] [8]Li J,Liao M,Gao W,et al.Topic Extraction from Microblog PostsUsing Conversation Structures.In:Proc.of the 54th Annual Meeting of theAssociation for Computational Linguistics.2016:1722–1731.
[0168] [9]He R,Zhang X,Jin D,et al.Interaction-Aware Topic Model forMicroblog Conversations through Network Embedding and User Attention.In:Proc.of the 27th International Conference on Computational Linguistics.2018:1398-1409.
[0169]
[10] Wang H,He R,Liu H,et al.Topic model on microblog with dual-streams graph convolution networks[C] / / 2022International Joint Conference onNeural Networks(IJCNN).IEEE,2022:1-8.
Claims
1. A decoupled representation microblog topic detection method with enhanced interpretability, It is characterized in that The following steps are involved: (1) Build a user-level dialogue network; (2) Build a multi-granular attention framework, in which word-level attention is used to alleviate the noise impact caused by colloquialism, and user-level attention is used to model the influence of different users on the topic, obtaining a hierarchical text representation, which is then fused with the adjacency matrix that stores structural information to generate a user node representation; (3) The user node representation is used as the input of neural variational inference, and the variational autoencoder is used as the inference framework. The inference network is used to encode the node information to generate a more coherent topic vector; (4) The decoupled representation module is used to extract the potential generative factors that influence the topic from the topic vector. The generative network is used to reconstruct the node embedding of the decoupled generative factors to obtain the topic word distribution. Sparse constraints are added to the topic word distribution to make it more consistent with the characteristics of real social media data.
2. According to the method for detecting microblog topics with decoupled representation and enhanced interpretability as claimed in claim 1, It is characterized in that Step (1) specifically includes: The user is regarded as a node in the dialogue network. At the same time, all microblogs related to the corresponding user, including source microblogs, comment microblogs, and forwarded microblogs, are aggregated into documents and regarded as the text information of the node. If there are microblog forwarding or comments between users in the dialogue network, the nodes representing the users are connected. This operation constructs a user-level dialogue network G = (V, E, T), where V is the set of nodes in the dialogue network G, and v i It means V={v i |1≤v i ≤L}; is the set of edges in the dialogue network G, e(i,j)∈E, representing the node v i and node v j There is a comment forwarding relationship between them; T = {t x1 ,t x2 ,...,t xL } is a collection of text information attached to the node, each t xi (i=1,2…L) contains M words; L is the number of users.
3. According to the method for detecting microblog topics with decoupled representation and enhanced interpretability as claimed in claim 1, It is characterized in that Step (2) specifically includes: In the user-level dialogue network, the attribute matrix X∈R L×C , R is a matrix, L is the number of users, and C is the dimension of the user node representation; replace each word with its corresponding low-dimensional, continuous and real-valued embedding, and make a weighted average of X, the adjacency matrix A∈R L×L Contains e(i,j), e(i,j)∈E, is the set of edges in the dialogue network G, V is the set of nodes in the dialogue network G; D is the degree matrix of the adjacency matrix A; and They are the degree matrix and adjacency matrix with self-loop added, see formula (1): Among them, I n is a self-connection matrix, S is a symmetric transfer matrix, simulating random walk, see formula (2): For user v i , word embedding matrix W i ∈R M×L , M represents the Mth word; W q , W k , W v are different weight matrices, the query matrix Q of the word-level attention layer wi , key matrix K wi Sum value matrix V wi The calculation is shown in formula (3): Q wi =W q ·W i K wi =W k ·W i V wi =W v ·W i (3) The word-level attention mechanism reduces the noise impact of colloquial vocabulary from a fine-grained level, uses softmax(·) for normalization, highlights the weights of important elements, and calculates the weight representation X of each word. wi , see formula (4): C is the dimension of user node representation. The user-level attention mechanism is introduced to learn the user's contribution to the topic distribution. The influence of different users on the topic is converted into the corresponding hierarchical text representation X calculated using the attention mechanism. r : Q s =W q ·X s K s =W k ·X s V s =W v ·X s (5) In formula (5) and (6), input X s ={X w1 ,X w2 ,...,X wL }, first embed the user into X s Perform nonlinear transformation to obtain the user-level key matrix K s , followed by the user attention vector Q s Calculate the similarity and further normalize it through the softmax function to get user v i The importance of the topic and the user's own semantics V s Multiply to get the hierarchical text representation X r ; Introducing a two-layer graph convolutional network to aggregate neighbor node representations containing relationship information and hierarchical text representation X r , learning node representation: In formula (7)(8)(9), W 1 and W 2 are the parameters of the first and second layers of the graph convolutional network respectively; f(·) takes ReLU(·) as the nonlinear activation function; update the neighbor node representation Graph convolutional networks aggregate the information sent by each node from its neighbors; H 1 H represents the user node embedding representation obtained by the first layer of graph convolutional network GCN; 2 The hierarchical text representation and neighbor node representation are integrated; the loss function of the graph convolutional network GCN is shown in (10), which minimizes the following objective function so that neighboring nodes contain more common information: Among them, v i represents the fusion representation of the i-th node, v j represents the fusion representation of the jth node, v j The selection range is v i The conditional probability p(v j |v i ) is formalized as shown in formula (11): Among them, h j It is from H 2 Intercepted from, user v i Neighbor v j ∈N i , cos(·,·) represents the cosine similarity.
4. According to claim 1, a decoupled representation microblog topic detection method with enhanced interpretability, It is characterized in that Step (3) specifically includes: The neural variational inference framework is used to infer the user-topic distribution θ = (p(t 1 |e),p(t 2 |e),…,p(t K |e)) and topic-word distribution φ w =(p(w|t 1 ),p(w|t 2 ),…,p(w|t K )), where e represents the user, t i represents the i-th topic, K represents the number of topics, and w represents words; p(t i |e)(i=1,2,..,K) represents the probability that user e belongs to the i-th topic, p(w|t i )(i=1,2,..,K) represents the probability that word w belongs to the i-th topic; User-topic distribution: Given a user embedding h i , first map it into the nonlinear latent space e: e=ReLU(W e h i +b e ) (12) Among them, W e and b e is the parameter of the encoder, ReLU is the nonlinear activation function; assuming that the user embeds h i The prior distribution and posterior distribution of are both Gaussian distributions, then the mean μ and variance σ of the posterior Gaussian distribution are 2 Obtained through linear transformation, see formula (13)(14): μ=W μ ·e+b μ (13) log(σ 2 )DW σ ·e+b σ (14) Among them, W μ 、b μ , W σ 、b σ is the parameter of the encoder; the latent semantic vector z∈R is further obtained by using the reparameterization technique K , the formalization is shown in formula (15): z=μ+∈×σ (15) Among them, ∈ is sampled from the Gaussian distribution N(0,I), and the latent semantic vector z is normalized and calculated using the sparsemax(·) function w , to achieve its sparse effect; Topic-word distribution: Topic-word distribution in the text φ w It can be regarded as the parameter of the decoder, see formula (16) for details: d=sparsemax(φ w ×(θ) T ) (16) Afterwards, h is reconstructed by the decoder i , and obtain the reconstructed user embedding h′ i , see formula (17): h′ i =ReLU(W d d+b d ) (17) Where W d 、b d These are all decoder parameters; For topic generation, the objective function of this part is shown in formula (18): In formula (18), L V It is expressed as the loss function formula for learning user-topic distribution and topic-word distribution, and uses KL divergence to measure the prior distribution q(z) and the true posterior distribution p(z|h ′ i ), where q(z) satisfies the prior Gaussian distribution N(0,I); By combining formulas (10) and (18), the overall objective function is defined to mine potential topics on Weibo, see formula (19), where λ is the trade-off L G and L V Hyperparameters: L=(1-λ)L G +λL V (19)。 5. According to claim 1, a decoupled representation microblog topic detection method with enhanced explainability, It is characterized in that Step (4) specifically includes: To extract the generative factor, we first make each dimension in the topic vector z independent of each other and add a control parameter β to the KL term, as shown in (20): The increase of β reduces the KL term, making p(z|h ′ i ) is closer to q(z), encouraging the encoder to extract more relevant information and achieve dimension separation. After separation, each dimension in the topic vector corresponds to an independent potential generation factor. According to the information bottleneck theory, this objective function maximizes the information bottleneck z and the output h′ of the downstream task. i The mutual information between them, and filter out h′ i may exist in the input h i The irrelevant information in is as shown in formula (21): max[I(z;h′ i )-β I(h i ;z)] (21) Where I(·; ·) represents mutual information and β is the Lagrange multiplier.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the program, the steps of the decoupled representation microblog topic detection method with enhanced interpretability as described in any one of claims 1 to 5 are implemented.
7. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the decoupled representation microblog topic detection method with enhanced interpretability described in any one of claims 1 to 5 are implemented.
