A text key point mining method based on abstract semantic representation and graph attention network

Abstract

The application discloses a text key point mining method based on abstract semantic representation and a graph attention network. The method combines text semantic information and text structure information, introduces label classification loss and spherical projection of an embedding space while narrowing the distance between the same categories and widening the distance between different categories in the embedding space, thereby effectively gathering the text while avoiding feature collapse. In addition, by introducing a random walk strategy of a Markov chain, the model further enhances the discriminability of the clustering space. The algorithm combines the advantages of a graph neural network and a Markov chain, preserves the combination of text semantic information and text structure information, and realizes accurate extraction of text key points.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and knowledge engineering technology, specifically involving a method for mining argument key points by combining abstract semantic parsing, graph deep learning and Markov clustering. Background Technology

[0002] In the context of the rapid development of internet information technology, social networks, news commentaries, and online debate platforms have become core arenas for the public to express their opinions and emotions. The data generated by these platforms is characterized by its massive volume, fragmentation, and high degree of unstructured nature. How to accurately and automatically extract core viewpoints from the complex stream of comments—that is, Key Point Analysis (KPA)—has become a critical task that urgently needs to be addressed in the fields of government public opinion monitoring, corporate market decision-making, and social science research.

[0003] Traditional text mining techniques primarily rely on bag-of-words models or statistical feature extraction methods. While these methods are efficient for simple texts, they often exhibit significant semantic recognition lag when dealing with logically complex and context-rich argument texts. With the emergence of pre-trained language models (such as BERT and RoBERTa), researchers have begun to explore extracting sequence features from fully connected layers using the Transformer architecture. Although these models capture statistical correlations between words through pre-training on large-scale corpora, their essence remains sequence feature extraction based on contextual probabilities. This approach faces profound challenges when handling argument mining tasks.

[0004] First, existing technologies generally suffer from a disconnect between shallow semantic features and deep logical structures. Argumentative texts are not simply a collection of words; their core claims are often manifested through complex predicate-centered structures, negation tendencies, and modification dependencies. Because sequence models have a weak perception of the internal logical topology of text, they often map logically disparate arguments to extremely similar vector spaces, leading to semantic misjudgments in subsequent data mining processes.

[0005] Secondly, insufficient discriminativeness in the clustering space is another major bottleneck limiting the improvement of keypoint mining accuracy. Under specific topics, user-generated arguments often exhibit extremely high local linguistic similarity, with arguments belonging to different keypoints frequently sharing a large number of keywords. In this case, the original representation vectors generated by the pre-trained model are prone to collapse in the feature space, meaning all arguments are tightly clustered together, lacking clear inter-class distances. This lack of discriminative spatial distribution makes traditional clustering algorithms face significant uncertainty when defining decision boundaries, easily leading to classification overlap between different viewpoints.

[0006] Meanwhile, the lack of adaptability in clustering mechanisms is also a challenge for the practical application of current technologies. Traditional clustering methods typically require a pre-defined number of key points. However, in real-world internet public opinion scenarios, the potential number of key points is often dynamically changing and unpredictable as topics evolve and user discussions deepen. A rigid pre-defined number can lead to the merging of minor but important viewpoints, or even the over-segmentation of core viewpoints. Furthermore, existing metric learning methods often lack in-depth utilization of multi-label supervision signals and cannot optimize feature distribution during the learning process by comparing similar samples with dissimilar samples, resulting in limited ability of representation vectors to distinguish fine-grained semantics.

[0007] To address the aforementioned bottlenecks, this invention proposes a text keypoint mining scheme based on Abstract Semantic Representation (AMR) and Graph Attention Network (GAT). The scheme first utilizes AMR to transform unstructured text into a semantic graph structure, establishing an alignment mapping between lexical units and logical nodes. Based on this, a graph attention network captures deep logical interactions between nodes, and deeply fuses structured features with sequence features to generate argument representation vectors with logical awareness. To solve the problem of cluster space collapse, this invention further introduces a spatial metric learning mechanism based on contrastive constraints. A contrastive loss function is constructed using multi-label supervision signals, forcing the model to establish highly discriminative boundaries in the representation space. Finally, iterative operators of the Markov Clustering (MCL) algorithm are used to automatically group arguments, utilizing the expansion and dilation properties of matrices to capture affinity fluctuations between arguments, thereby achieving automatic text keypoint mining and summary output without pre-setting the number of classifications. This end-to-end technical solution effectively compensates for the shortcomings of existing technologies in logical awareness recognition, feature space distribution, and cluster adaptability, providing scientific and technical support for the deep governance of massive amounts of unstructured text. Summary of the Invention

[0008] This invention proposes a text keypoint mining method based on Abstract Semantic Representation (AMR) and Graph Attention Network (GAT). Compared to previous methods that rely solely on the sequence features of pre-trained language models, this method simultaneously utilizes AMR to capture the deep logical structure of text and leverages GAT to learn the dependency weights between nodes. This enables the model to accurately distinguish semantically similar but logically different arguments when extracting keypoints. Furthermore, this invention introduces a supervised contrastive learning strategy to enhance the discriminative power of the representation space. For application scenarios with an uncertain number of keypoints, this invention employs an adaptive Markov clustering algorithm combined with an attention pooling mechanism, solving the problems of traditional clustering algorithms requiring a pre-defined number of centers and incomplete feature capture for long arguments.

[0009] This invention proposes a text keypoint mining method based on abstract semantic representation and graph attention network, comprising the following steps:

[0010] 1) Data parsing and alignment. The original argument text is transformed into a directed semantic graph using an AMR parser. Based on the alignment information generated by the parsing, the word-level embeddings extracted by BERT are mapped to the corresponding semantic graph nodes, completing the initial fusion of heterogeneous features.

[0011] 2) Structured Feature Extraction. The semantic graph with initial features is input into a multi-layer graph attention network, and the node representations are dynamically updated based on logical relationships using a multi-head attention mechanism. The topological features of the entire graph are compressed into a fixed-length vector reflecting the core theme of the argument through an attention pooling layer, so as to more comprehensively evaluate the model's representation ability in complex logical scenarios.

[0012] 3) Spatial optimization based on contrastive learning. A supervised contrastive loss is constructed based on known keypoint labels, and the similarity between positive and negative sample pairs within a batch is calculated to stretch the feature space distribution.

[0013] 4) Dynamic Clustering and Summary Generation. Based on the optimized vectors, the affinity matrix is ​​calculated, and keypoint clusters are automatically discovered using Markov Clustering (MCL). After clustering, a generative model is called to perform text summarization for each cluster, outputting the final keypoints.

[0014] Furthermore, in step 1), a pre-trained bert-base-uncased model is used to extract context-related word embeddings, and the Penman library is used to perform structured decoding of the AMR graph data, ensuring the richness of the initial features.

[0015] Furthermore, in step 2), a joint encoding mechanism is constructed to simultaneously capture local logic and global semantics during feature extraction. First, regarding local logic, the GAT network processes core semantic relationships such as ARG0 and ARG1, adjusting the contribution of different logical paths through weight coefficients. The final feature vector is the concatenation of the [CLS] global vector and the attention pooling graph vector.

[0016] Secondly, regarding the attention mechanism, a node importance evaluation benchmark is constructed based on multi-head attention scores, enabling the model to automatically ignore irrelevant stop word nodes or descriptive nodes. By introducing the ReLU activation function and LayerNorm normalization, the feature distribution is kept consistent, meaning that arguments with more prominent logical cores have higher cohesion in the space and correspondingly higher similarity weights.

[0017] Furthermore, in step 4), to avoid the clustering process becoming too fragmented or the cluster centers shifting, we explicitly retain the supervised contrastive loss constraint in the optimization objective to limit the distribution range of features on the hypersphere, thereby ensuring that the model can maintain a stable and discriminative trajectory during the iteration process.

[0018] Meanwhile, to address the issue of feature collapse during semantic clustering of long argument sequences, this paper employs the expansion and dilation strategies from Markov clustering algorithms. This allows each argument's neighbors to directly participate in the redistribution of clustering probabilities, significantly enhancing the keypoint contribution of dense regions. This adaptive optimization mechanism effectively alleviates the problem of determining the number of clusters in traditional K-Means algorithms for dynamic keypoint mining tasks, enabling the model to more fully discover long-distance semantic dependencies and complex topic structures.

[0019] Through the above improvements, the proposed method significantly improves the clustering efficiency and overall performance of the model in dynamic dataset scenarios while ensuring the accuracy of data mining.

[0020] The method of this invention can effectively perform argument summarization and key point analysis of large-scale texts, and has the following advantages compared with existing technologies:

[0021] 1. This invention designs a feature extraction mechanism that integrates logical structure. When evaluating arguments, it introduces both AMR graph topological constraints and BERT deep semantic constraints. Based on this, it achieves a synergistic balance between logical details and global semantics, thereby effectively improving the model's accuracy in capturing key points in complex debate scenarios.

[0022] 2. To address the issues of uneven distribution of key points and difficulty in predicting the number of clusters, this invention introduces supervised contrastive learning constraints during the optimization process and combines them with the adaptive discovery method of Markov clustering to finely control the cluster space. This enables the model to more fully capture the inherent attribution relationships between arguments and stably extract complex key point structures. Attached Figure Description

[0023] Figure 1 This diagram illustrates the overall process, depicting the entire chain from raw text input to final keypoint output, covering four core stages: heterogeneous feature extraction, graph attention representation, contrastive learning optimization, and Markov clustering generation. Step A, the heterogeneous feature extraction and graph structure construction process, demonstrates how text terms are transformed into semantic feature vectors by a pre-trained model. Simultaneously, the parser generates an AMR semantic graph, and alignment relationships map word vectors to graph nodes, forming a feature-rich semantic graph. The graph attention encoder aggregates neighborhood logical information and updates node representations through multi-head attention weights, while supervised contrastive learning utilizes attraction and repulsion to cluster similar arguments within the space. Detailed Implementation

[0024] To make the above-mentioned features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings. The specific process is as follows: Figure 1 As shown, its main steps include:

[0025] Step 1: Extract the semantic features of the text to be processed, and use abstract semantic representation parsing technology to transform the text into a semantic graph structure, and establish the mapping and alignment relationship between text words and semantic graph nodes;

[0026] Step 101: Extract the context embedding vector sequence of the text using a pre-trained language model. Where n is the total number of tokens in the input text after word segmentation, d is the dimension of the hidden layer of the training language model, and the th... The feature vector of each word is represented as follows: ,Depend on express;

[0027] Step 102: Use an abstract semantic representation parser to transform the text into a directed acyclic graph. , where V is the set of semantic concept nodes and E is the set of logical relationship edges between concepts;

[0028] Step 103: Use the Leamr alignment tool to obtain the alignment probability matrix between nodes and words. ,in Indicates the first Nodes With the each word element The probability of alignment between them satisfies Based on the alignment probability, the word embeddings are mapped to the corresponding semantic concept nodes through mean pooling to obtain the initial feature vectors of the nodes. ;in Each Depend on Calculated.

[0029] Step 2: Input the semantic graph structure and semantic features into the preset graph attention enhancement encoder, and generate the argument representation vector by fusing the deep semantics of the text and the graph topology features;

[0030] Step 201: Utilize a multi-layer graph attention network to analyze node features. To update, calculate the attention coefficient between node i and its neighbor node j. :

[0031]

[0032] in, It is an exponential function. For activation function, Let W be a learnable attention weight vector, and let W be a learnable weight matrix, used to perform a linear transformation on the node features, mapping the original features to a high-dimensional space. The initial node features obtained in step A3;

[0033] Step 202: Use an attention pooling layer to perform full-image aggregation on the updated node features and extract structured representation vectors. :

[0034]

[0035] in, Let i be the contribution weight of node i during aggregation. This represents the lexical feature vector representation of step A1;

[0036] Step 203: Extract the first and second feature vectors of the text. With structured representation vectors The data is then spliced ​​together and fused through a fusion layer to obtain the final argument feature representation. :

[0037]

[0038] in The output of the pre-trained model Bert vector, For structured representation of vectors, For learnable weights, Representation layer normalization, For activation function, This is the final generated argument feature representation vector.

[0039] Step 3: Construct a contrastive loss function using multi-label supervision signals, and perform spatial metric learning on the argument representation vectors so that arguments belonging to the same key points are close to each other in the representation space;

[0040] Step 301: Construct the supervised contrastive loss function By calculating the cosine similarity between samples with the same keypoint labels within a batch, the distance between positive sample pairs is shortened while the distance between negative sample pairs is widened.

[0041]

[0042] in, For all sample arguments in the training batch, To be consistent with the sample A set of indexes with the same label. For the sample The representation vector, For temperature hyperparameters, To exclude samples The remaining samples outside;

[0043] Step 302: Combine multi-label classification loss

[0044]

[0045] in For the number of texts, For the total number of tags, For real labels, For the model to predict the first The first argument belongs to the first The probability of each label is determined through joint optimization. Iteratively update the encoder parameters until the loss reaches a preset threshold.

[0046] Step 4: Construct an argument affinity matrix based on the learned representation vectors, use Markov clustering algorithm to automatically group arguments, and perform text summarization on each group to output the mined text key points.

[0047] Step 401: Calculate the equivalence similarity of all argument representation vectors in the dataset to be tested. :

[0048]

[0049] in For the text in step B3 The final argument is characterized by the following features: For the text in step B3 The final argument features are represented by each argument. The set is obtained by calculating its similarity to other arguments. By retaining the most similar ones Construct a sparse adjacency matrix from the neighbors. ;

[0050]

[0051] Step 402, for the matrix Markov clustering is performed to achieve matrix convergence by alternating expansion and dilation operators:

[0052]

[0053] in The coefficient of thermal expansion is 1 / 3. For the original matrix, The summation index indicates that each column of the matrix is ​​normalized;

[0054] Step 403: Extract the column vectors of the converged matrix as clusters, and call the generative summarization model T5 for the text in each cluster to generate text phrases representing the core semantics of the cluster as key point results.

[0055] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A text keypoint mining method based on abstract semantic representation and graph attention network, characterized in that, include: A. Heterogeneous Feature Extraction and Graph Structure Construction: Extract the semantic features of the text to be processed, and use abstract semantic representation parsing technology to transform the text into a semantic graph structure, and establish the mapping and alignment relationship between text words and semantic graph nodes; B. Representation learning that integrates structural information: Input the semantic graph structure and semantic features into a pre-defined graph attention-enhanced encoder, and generate argument representation vectors by integrating the deep semantics of the text with the graph topology features; C. Clustering space construction based on contrast constraints: Contrast loss function is constructed using multi-label supervision signals, and spatial metric learning is performed on the argument representation vector to make arguments belonging to the same key points closer to each other in the representation space; D. Keypoint extraction based on Markov clustering: Construct an argument affinity matrix based on the learned representation vectors, use the Markov clustering algorithm to automatically group arguments, and perform text summarization on each group to output the mined text keypoints.

2. The text key point mining method based on abstract semantic representation and graph attention network according to claim 1, characterized in that, Step A includes: A1. Extract the context embedding vector sequence of the text using a pre-trained language model. Where n is the total number of tokens in the input text after word segmentation, d is the dimension of the hidden layer of the training language model, and the th... The feature vector of each word is represented as follows: ,Depend on express; A2. Use an abstract semantic representation parser to convert text into a directed acyclic graph. , where V is the set of semantic concept nodes and E is the set of logical relationship edges between concepts; A3. Use the Leamr alignment tool to obtain the alignment probability matrix between nodes and words. ,in Indicates the first Nodes With the each word element The probability of alignment between them satisfies Based on the alignment probability, the word embeddings are mapped to the corresponding semantic concept nodes through mean pooling to obtain the initial feature vectors of the nodes. ;in Each Depend on Calculated.

3. The text key point mining method based on abstract semantic representation and graph attention network according to claim 1, characterized in that, Step B includes: B1. Utilizing multi-layer graph attention networks to analyze node features To update, calculate the attention coefficient between node i and its neighbor node j. : in, It is an exponential function. For activation function, Let W be a learnable attention weight vector, and let W be a learnable weight matrix, used to perform a linear transformation on the node features, mapping the original features to a high-dimensional space. The initial node features obtained in step A3; B2. Use an attention pooling layer to perform full-image aggregation on the updated node features and extract structured representation vectors. : in, Let i be the contribution weight of node i during aggregation. This represents the lexical feature vector representation of step A1; B3. Extract the first and second feature vectors of the text. With structured representation vectors The data is then spliced ​​together and fused through a fusion layer to obtain the final argument feature representation. : in The output of the pre-trained model Bert vector, For structured representation of vectors, For learnable weights, Representation layer normalization, For activation function, This is the final generated argument feature representation vector.

4. The text key point mining method based on abstract semantic representation and graph attention network according to claim 1, characterized in that, Step C includes: C1. Constructing the supervised comparison loss function By calculating the cosine similarity between samples with the same keypoint labels within a batch, the distance between positive sample pairs is shortened while the distance between negative sample pairs is widened. in, For all sample arguments in the training batch, To be consistent with the sample A set of indexes with the same label. For the sample The representation vector, For temperature hyperparameters, To exclude samples The remaining samples outside; C2. Combining multi-label classification loss in For the number of texts, For the total number of tags, For real labels, For the model to predict the first The first argument belongs to the first The probability of each label is determined through joint optimization. Iteratively update the encoder parameters until the loss reaches a preset threshold.

5. The text key point mining method based on abstract semantic representation and graph attention network according to claim 1, characterized in that, Step D includes: D1. Calculate the equivalence similarity of all argument representation vectors in the dataset to be tested. : in For the text in step B3 The final argument is characterized by the following features: For the text in step B3 The final argument features are represented by each argument. The set is obtained by calculating its similarity to other arguments. By retaining the most similar ones Construct a sparse adjacency matrix from the neighbors. ; D2, For the matrix Markov clustering is performed to achieve matrix convergence by alternating expansion and dilation operators: in The coefficient of thermal expansion is 1 / 3. For the original matrix, The summation index indicates that each column of the matrix is ​​normalized; D3. Extract the column vectors of the converged matrix as clusters. For the text in each cluster, call the generative summarization model T5 to generate text phrases representing the core semantics of the cluster as key point results.

Images