News text classification method, system, storage medium and device

Abstract

The present invention provides a news text classification method, system, storage medium, and device. A semantic dictionary is introduced to enrich the constructed text graph. For each enriched text graph, a graph convolutional network is used to extract text structural features, and a BERT model is used to extract text contextual semantic features. A multi-head attention mechanism is used to interact the extracted text structural features and contextual semantic features, combining the two features of different granularities. The interacted features are aggregated using different aggregation methods to achieve news text classification. The present invention can utilize graph convolution to propagate and extract text structural features, and BERT to extract semantic features. Simultaneously, a multi-head attention mechanism is used to achieve feature interaction between the two features of different granularities, effectively improving the accuracy of news text classification.

Description

Technical Field

[0001] The present invention belongs to the technical field of news text classification, and relates to a news text classification method, system, storage medium and equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Traditional text classification methods primarily combine training and prediction with machine learning algorithms. The basic idea is to use feature engineering to represent text, then train a classifier to predict the text's category. Feature engineering methods, such as bag-of-words models and n-grams, are commonly used in word-based representations, representing sentences as a collection of words or sequences of byte segments of length n. These features, combined with machine learning algorithms such as support vector machines and naive Bayes, have achieved promising results. However, traditional feature extraction methods have numerous drawbacks. They fail to consider contextual information during text classification, and their processing and generalization capabilities for high-dimensional data are relatively poor.

[0004] To address the limitations of manual feature engineering, many researchers have applied deep learning to text classification. Its advantage lies in its ability to automatically learn feature representations for data using its inherent network architecture, eliminating the need for manual feature engineering. The emergence of word embeddings has provided a new solution for solving natural language tasks. Researchers have demonstrated that pre-trained word embeddings, such as Word2Vec and Glove, can capture meaningful semantic features of text. In terms of models, TextCNN uses a simple convolutional neural network, combining multiple convolutional layers with different kernel sizes for feature mapping. Recently, pre-trained language models have garnered widespread attention. For example, BERT uses a masked language model (MLM) to pre-train bidirectional Transformers to generate deep, bidirectional language representations. After pre-training, simply adding an additional output layer for fine-tuning significantly improves performance on downstream tasks. Attention mechanisms have also been applied to various models. The self-attention mechanism in BERT forms a multi-layered self-attention network, enhancing semantic feature representation and improving performance on NLP tasks. However, BERT fails to fully leverage the structural information in the text.

[0005] Graph neural networks (GNNs) can capture the characteristics and structural features of nodes in a graph and learn more effective representations of nodes or the entire graph. Many GNN variants have been proposed and applied to text classification tasks. Among them, Kipf et al. proposed a GNN based on spectral graph theory. This model considers both the node's own properties and those of its neighbors to obtain a node's feature vector, ultimately enabling tasks such as graph node classification and regression. It achieved promising results on node classification tasks. The TextGCN model proposed by Yao et al. was the first to apply GNNs to text classification. TextGCN constructs a large, heterogeneous graph network with words and documents as nodes for the entire corpus and learns the global information structure within the graph. VGCN-BERT also builds a graph over the entire dataset and uses features extracted by a GNN to enhance BERT's performance. However, for a new document, prediction requires updating the entire graph structure, making it difficult to consider local features between words within a single document, weakening the individual functionality of each document. TextING builds a graph structure for each individual document. Some approaches also combine pre-trained language models with GNNs to extract features.

[0006] As a type of data to be classified, news text has some particularities and is more difficult to classify than other texts. Specifically, the following aspects are involved:

[0007] Diversity and complexity: News texts cover a wide range of topics, including politics, economy, technology, entertainment, etc. This leads to the diversity and complexity of text content, making the classification task more challenging.

[0008] Semantic and contextual understanding: News texts often involve complex semantics and contextual relationships. Understanding the implicit meaning, sentiment, and specific references in the text is crucial for correctly classifying news texts.

[0009] Noise and interference: There may be a lot of noise and interference in news text, such as spelling errors, abbreviations, punctuation marks, etc. These noise and interference may have a negative impact on the performance of the classification algorithm, requiring appropriate data cleaning and preprocessing.

[0010] Topic overlap and ambiguity: In news texts, there may be overlap and correlation between different topics. Some news may cover multiple topics and may be ambiguous, making it difficult to accurately classify them into a specific topic.

[0011] According to the inventors’ understanding, most of the news text classification models currently proposed use methods such as LSTM and BERT to process serialized data, which can handle the contextual information of the text well and improve the accuracy of text encoding. However, the structural information of the text is not considered. Using GCN for modeling can better utilize the structural information of text data. And usually, the text graph constructed based on word co-occurrence only considers the co-occurrence relationship between nodes, without considering richer semantic information. In addition, some methods combine the semantic features and structural features of text to extract, but they do not consider individual text features or the interaction between features, which limits their representation capabilities. Summary of the Invention

[0012] In order to solve the above problems, the present invention proposes a news text classification method, system, storage medium and device. The present invention can use graph convolution to propagate and extract text structure features, use BERT to extract semantic features, and at the same time, use a multi-head attention mechanism to realize feature interaction between two features of different granularity, which can effectively improve the accuracy of news text classification.

[0013] According to some embodiments, the present invention adopts the following technical solutions:

[0014] A news text classification method comprises the following steps:

[0015] Build a separate text graph for each document stored in the news text dataset based on word co-occurrence;

[0016] Introducing semantic dictionaries to enrich the constructed text graph;

[0017] For each enriched text graph, we use graph convolutional networks to extract text structural features, and use the BERT model to extract contextual semantic features of the text.

[0018] A multi-head attention mechanism is used to interact the extracted text structure features and contextual semantic features, so as to combine the features of two different granularities;

[0019] Different aggregation methods are used to aggregate the features after interaction to achieve the classification of news text.

[0020] As an optional implementation, the specific process of constructing a separate text graph for each document based on word co-occurrence includes using G = (V, E) to represent the graph of the document, where V represents the set of nodes in the graph, including all words in the text, and E represents the set of edges between the nodes.

[0021] As a further step, the normalized point-by-point mutual information method is used to calculate the edge weight between two words. The greater the frequency of co-occurrence of two words within the set range, the greater the weight. When the semantic correlation between words in the corpus exceeds the set value, the edge weight is positive, otherwise it is negative. Only when the edge weight is positive, an edge is created between the word pairs.

[0022] As an optional implementation, a semantic dictionary is introduced to enrich the constructed text graph. The specific process includes organizing the lexical semantic network according to semantic relationships by the semantic dictionary. Vocabulary is represented by synonym sets, each set indicates a lexical concept, and semantic relationships including hyponym, part-whole, synonym and antonym are expressed by links.

[0023] If the edge weight is negative, the semantic dictionary is used to expand the synonyms and calculate the semantic similarity between the words in the two synonym sets.

[0024] As an optional implementation, the specific process of extracting text structural features using a graph convolutional network includes performing a convolution operation on a text graph using the graph convolutional network to extract feature representations of nodes while considering structural information between nodes;

[0025] In graph convolutional networks, propagation at each layer is performed by updating the weighted sum of a node's neighbors and the node itself;

[0026] And nonlinear functions are used as activation functions in the graph convolutional network training process.

[0027] Furthermore, the graph convolutional network includes multiple graphs stacked on top of each other.

[0028] As an optional implementation, the specific process of extracting contextual semantic features of text using the BERT model includes that the BERT model includes an encoder part, and the encoder part is composed of a stack of multiple layers of encoders, each encoder including a self-attention layer and a feedforward network layer.

[0029] As an optional implementation, the specific process of using a multi-head attention mechanism to interact with the extracted text structure features and contextual semantic features includes calculating the query matrix, key and value matrix of the text structure features and contextual semantic features, and passing the keys and values of the text structure features and contextual semantic features as input to each other's multi-head attention modules.

[0030] As an optional implementation method, the specific process of using different aggregation methods to aggregate the features after interaction includes using Max-pooling, splicing and element-by-element addition methods to aggregate the extracted features.

[0031] A news text classification system, comprising:

[0032] a text graph construction module, configured to construct a separate text graph for each document storing the news text dataset based on word co-occurrence;

[0033] a text graph enrichment module, configured to introduce a semantic dictionary to enrich the constructed text graph;

[0034] The feature extraction module is configured to use the graph convolutional network to extract text structural features and the BERT model to extract contextual semantic features of the text for each enriched text graph;

[0035] The feature interaction module is configured to use a multi-head attention mechanism to interact with the extracted text structure features and contextual semantic features, so as to combine the features of two different granularities;

[0036] The aggregation module is configured to perform feature aggregation on the features after interaction using different aggregation methods to achieve classification of news texts.

[0037] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the steps in the method.

[0038] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded by the processor and executing the steps in the described method.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention first constructs a graph structure for each document based on word co-occurrence. To address the problem of insufficient word connections and inability to capture contextual dependencies, a semantic dictionary (WordNet) is introduced to enrich the construction of the text graph. The present invention also uses a graph convolutional network to extract text structural features. Simultaneously, the BERT model is used to extract the semantic features of the text. Subsequently, a feature interaction layer is added, using a multi-head attention module to address the interaction between two features of different granularity, and three aggregation methods are designed for feature aggregation, thereby maximizing the representational power of the extracted features.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 Schematic diagram of the overall model structure used in this embodiment;

[0044] Figure 2 is a schematic diagram of the feature interaction of this embodiment;

[0045] Figure 3 3 is a schematic diagram comparing the effects of the method of this embodiment and other methods. DETAILED DESCRIPTION

[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0049] Example 1

[0050] The present invention provides a news text classification method. First, a graph structure is constructed for each document based on word co-occurrence. However, the word connections are not rich enough and the context dependency cannot be well captured. Therefore, a semantic dictionary (WordNet) is introduced to enrich the construction of the text graph. A graph convolutional network is used to extract text structural features. At the same time, the BERT model is used to extract the semantic features of the text. Subsequently, a feature interaction layer is added, and a multi-head attention module is used to solve the interaction problem between two features of different granularity. Three aggregation methods are designed for feature aggregation, thereby maximizing the representation ability of the extracted features. Finally, a fully connected layer is used to predict the category to obtain the final category of the text.

[0051] like Figure 1 As shown, combined with the model structure, the steps of the method of this embodiment are described:

[0052] Step 1: Text graph construction

[0053] The data set used in this embodiment is a news data set, which consists of multiple news documents. The present invention constructs a separate text graph for each news document text in the data set.

[0054] To use graph neural networks to process text data, a text graph must be constructed. A graph describing the semantic structure of text is constructed based on word co-occurrence relationships and a semantic dictionary. Word co-occurrence relationships can be used to measure the correlation between two words, thereby describing their semantic similarity and association. The semantic dictionary is also introduced to co-construct the text graph, allowing it to capture more relationships between words and enrich the text graph information.

[0055] In this embodiment, a separate graph is created for each document. Formally, this is represented by G = (V, E). V represents the set of nodes in the graph, including all words in the document. E represents the set of edges between the nodes. In this embodiment, for each piece of data in the dataset, standard methods are used to preprocess the text to remove stop words defined in NLTK and low-frequency words that appear less than a set number of times (e.g., 5 times) in the entire corpus.

[0056] This paper uses the Normalized Pointwise Mutual Information (NPMI) semantic dictionary (WordNet) to construct a text graph. For two words i and j, the NPMI method is first used to calculate the edge weight between the two words. Formally, the edge weight between word node i and word node j is defined as:

[0057]

[0058] in #w(i,j) is the number of all sliding windows that contain both words i and j, #w(i) is the number of all sliding windows that contain word i, and #w is the total number of sliding windows. To capture long-term dependencies, this article sets the window setting to the entire sentence. The NPMI value range is [-1, 1]. Simply put, the greater the frequency of co-occurrence between two words within a certain range, the greater the weight. NPMI values are positive when there is a high semantic correlation between words in the corpus. Conversely, NPMI values are negative when there is low or no semantic correlation between words. Therefore, an edge is created between word pairs only when the NPMI value is positive.

[0059] However, the calculation of NPMI depends on the corpus. When some words in a corpus have low occurrence probabilities, the NPMI result may be small, indicating low or no similarity between these words. Furthermore, there are many connections between words. Many words represent the same concept and are often interchangeable, the most direct example being synonyms. In this case, constructing a text graph based solely on word co-occurrence ignores this information about word relationships.

[0060] To better utilize word synonymy to enrich text graph information, this paper introduces a semantic lexicon. WordNet is chosen for this purpose. WordNet organizes lexical semantic networks based on semantic relationships. Words are represented by synonym sets, each of which represents a lexical concept. Links are used to express semantic relationships such as hyponymy, part-wholeness, and synonymy and antonymity. This creates a relatively complete semantic network while providing a well-organized conceptual hierarchy. Semantic relationships between synonym sets are implemented through links. Therefore, this paper first uses WordNet to expand synonyms. Then, the WordNet-based Wup method is used to calculate the semantic similarity between words in two synonym sets. The Wup method considers not only the path between two concepts but also their common parent nodes and their relative depth in the classification tree. It returns a score indicating the degree of similarity between the two word meanings. Here, the average of all similarity scores is calculated; a larger value indicates higher semantic similarity. This minimizes issues caused by low-frequency words in the corpus and adds more information about the relationships between words.

[0061] Therefore, for two words i and j, if the calculated NPMI value is negative, the Wup method based on WordNet is used, and the semantic similarity between the words in the two synonym sets is calculated as:

[0062]

[0063]

[0064] Where, let c3 be S x 、P y N1 represents the deepest common child node of S x The distance to c3, N2 represents P y The distance from c3 to c3, N3 represents the distance from c3 to the root node of the concept hierarchy tree. Let c1 be the synonym word set of word i, that is, c1 = {S1, S2, ..., S x}, x∈[1,L1], L1 is the number of words in c1. Let c2 be the synonym set of word j, that is, c2={P1,P2,...,P y}, y∈[1,L2], where L2 is the number of words in set c2. n represents the number of word pairs with a similarity of 0. The value of Sim(i,j) ranges from [0 to 1]. Here, we set the value of Sim(i,j) to be greater than 0.5, creating an edge between two words.

[0065] This embodiment constructs a separate text graph for each news document, taking into account individual text features to capture more fine-grained features. It also introduces a semantic dictionary (WordNet) to resolve ambiguity based solely on word co-occurrence, expand the word set, and enrich the connections between words. This provides some assistance in addressing topic overlap and ambiguity in news text classification.

[0066] Step 2: Feature extraction based on GCN

[0067] For each text graph, this paper uses a graph convolutional neural network (GCN) for feature propagation and extraction. GCN performs convolution operations on the text graph to extract feature representations of nodes while also considering the structural information between nodes. This structural information helps GCN better capture the semantic information of text data, improving its representation capabilities and ultimately increasing the accuracy of text classification.

[0068] In GCN, propagation at each layer is performed by updating the weighted sum of a node’s neighbors and the node itself. For a single-layer GCN, the new representation is calculated as follows:

[0069]

[0070] Where X∈R |v|×m It is an input matrix with |V| nodes and m-dimensional features. is a normalized symmetric adjacency matrix, D is the degree matrix and D ii =∑ j A ij ,The purpose of A normalization is to solve the problem of unstable feature values and gradient disappearance / explosion. is the adjacency matrix plus the self-loop matrix, I n is the identity matrix. W0 is a weight matrix and σ is the activation function.

[0071] When multiple GCN layers are stacked, more neighborhood information is integrated. Specifically, for multiple layers of GCN, the new calculation is expressed as follows:

[0072]

[0073] Where l represents the number of layers, H (0) =X is the initial feature matrix. Here, we use Glove word vectors for initialization and use two layers of GCN to train it. The dimension of the second layer of GCN is the number of classes in the dataset. After two layers of message passing, we get the final word feature representation.

[0074] Here, we choose the nonlinear function LeakyReLU as the activation function in the GCN training process. This not only reduces the complexity of the model, but also reduces the risk of overfitting. The LeakyReLU function is as follows:

[0075]

[0076] Step 3: BERT-based feature extraction

[0077] In addition to using GCN to extract text structural features, this paper also uses BERT to extract contextual semantic features. BERT can understand the meaning and function of words in a sentence, thereby achieving better contextual understanding. BERT is also able to infer the intended meaning of polysemous words in the current context, thereby better representing text.

[0078] BERT is based on the Transformer, but it only uses the encoder part of the Transformer. Each encoder consists of two sub-layers: a self-attention layer and a feed-forward network layer. For self-attention, the entire calculation process can be expressed as:

[0079]

[0080] Among them, Q(Query), K(Key) and V(Value) are the matrices of query, key and value respectively, and these three matrices all come from the same input. k is the dimension of the matrix.

[0081] The BERT architecture consists of multiple layers of stacked Transformer encoders. Each encoder layer consists of a multi-head attention layer and a feedforward network. Each attention group is used to map the input to a different sub-representation space, allowing the model to focus on different locations in different sub-representation spaces. For multi-head attention, the entire computation process can be expressed as:

[0082] MultiHead(Q,K,V)=CONCAT(head4,head2,...,head h ) (8)

[0083]

[0084] Extract features through BERT to obtain the final word feature representation

[0085] This embodiment uses BERT to extract contextual semantic features. BERT has powerful semantic representation capabilities and can capture richer semantic information. GCN is used to extract structural information of text data, such as the adjacency relationship and relative position between nodes. The structural information between nodes can help GCN better capture the semantic information of text data. By interacting the text information extracted by GCN and BERT, the graph structure processing capabilities of GCN and the semantic understanding capabilities of BERT can be used to complement each other. The multi-head attention module can help the model better capture the correlation and importance between the two. In the feature aggregation stage, the use of maximum pooling can extract the most significant features, element-by-element addition can retain the complementary information between different features, and splicing can connect the dimensions of different features together. This can generate richer and more comprehensive feature representations, which helps to improve the accuracy of text classification.

[0086] Step 4: Feature Interaction

[0087] In order to take advantage of the different granularity feature expressions, improve feature representation capabilities, and enhance classification accuracy, the text structure features and semantic features extracted by GCN and BERT are combined through a multi-head attention module to achieve interaction between the structural features and semantic features.

[0088] Like the calculation in the standard self-attention mechanism, the query matrix (Q), key (K) and value (V) matrices are calculated by the text representation extracted by GCN and BERT. The keys (K) and values (V) of the text features extracted by GCN and BERT are passed as input to each other's multi-head attention modules, as shown in Figure 2 shown.

[0089] Specifically, first, and Calculate the corresponding query matrix (Q), key matrix (K) and value matrix (V) respectively.

[0090]

[0091]

[0092]

[0093]

[0094]

[0095]

[0096] Different from the self-attention mechanism, taking Q L , K bert and V bertAs the input of formula (7), we get H L , take Q bert , K L and V L As the input of formula (7), we get H bert Where W Q 、W K 、W V is the parameter matrix.

[0097] Then, we can obtain the attention representation of GCN conditioned on BERT output and the attention representation of BERT conditioned on GCN output. Let the extracted text structure features and semantic features interact and make full use of the advantages expressed by features of different granularity.

[0098] Step 5: Feature Aggregation

[0099] Aggregating features of different granularities can improve classification accuracy and performance, simplify the model structure, and improve the robustness and stability of the model. This paper uses three methods to aggregate the extracted features, including Max-pooling, concatenation, and element-by-element addition. For the word w in the sequence j , the features extracted by GCN and BERT are respectively recorded as and The specific method is as follows:

[0100] Max-pooling: Let each word play a role in the text, and keywords should play a more clear role. Select the maximum eigenvalue of the two features in each dimension to form the final representation:

[0101]

[0102] Concatenation: Directly concatenate the two features so that the output of each module remains complete. The final representation is:

[0103]

[0104] Element-wise addition: This is achieved by directly adding the data of the corresponding dimensions. This method does not change the dimension of the feature vector. The final representation is:

[0105]

[0106] The ⊕ operation means element-by-element addition.

[0107] The final representation of the entire document j is denoted as H j .

[0108] Figure 3 Ablation experiments were conducted on three news datasets: 20NG, AGNews, and R8. BEGCN-nowordnet is a model that uses only normalized point-by-point mutual information to construct a text graph and learn a graph network. BEGCN-noAttention is a model without a feature interaction module. Specifically, it uses only features extracted by BERT and GCN, and concatenates the structural information obtained by GCN with the semantic information obtained by BERT. There is no interaction between the two features. BEGCN is the complete news text classification model proposed in this example.

[0109] It can be seen that this embodiment uses three news datasets, 20NG, AGNews, and R8, for experiments. For each news dataset, BertTokenizer is used to segment documents, and 10% of the training data is used for verification. Compare the gap between the model in this article and the current most advanced model. The results show that the model in this article is very competitive compared to other models. After ablation experiments, it can be seen that the model that introduces a semantic dictionary when constructing a text graph and uses a multi-head attention module for feature interaction has a significant improvement compared to the baseline model, which directly shows that the method proposed in this model can significantly improve the performance of text classification.

[0110] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0114] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0115] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A news text classification method, characterized by: The following steps are involved: Build a separate text graph for each document stored in the news text dataset based on word co-occurrence; Introducing semantic dictionaries to enrich the constructed text graph; For each enriched text graph, the graph convolutional network is used to extract text structure features, and the BERT The model extracts contextual semantic features of the text; A multi-head attention mechanism is used to interact the extracted text structure features and contextual semantic features, so as to combine the features of two different granularities; Use different aggregation methods to aggregate the features after interaction to achieve the classification of news text; The normalized point-by-point mutual information method is used to calculate the edge weight between two words. The greater the frequency of co-occurrence of two words within a set range, the greater the weight. When the semantic relevance between words in the corpus exceeds the set value, the edge weight is positive, otherwise it is negative. Only when the edge weight is positive, an edge is created between the word pairs. The specific process of introducing a semantic dictionary to enrich the constructed text graph includes organizing the semantic dictionary into a vocabulary semantic network according to semantic relationships, representing vocabulary through synonym sets, each set indicating a vocabulary concept, and expressing semantic relationships including hyponym / hypernym, part-whole, synonym / antonym through links; If the edge weight is negative, use WordNet Expand synonyms, using WordNet of Wup Method to calculate the semantic similarity between words in two synsets.

2. A news text classification method as claimed in claim 1, characterized in that: The specific process of constructing a separate text graph for each document based on word co-occurrences involves using G = (V, E) To represent the graph of the document, V represents the set of nodes in the graph, including all the words in the text, E Represents the set of edges between nodes.

3. A news text classification method as claimed in claim 1, characterized in that: The specific process of using graph convolutional networks to extract text structural features includes using graph convolutional networks to perform convolution operations on text graphs, extracting feature representations of nodes, and considering the structural information between nodes; In graph convolutional networks, propagation at each layer is performed by updating the weighted sum of a node's neighbors and the node itself; And nonlinear functions are used as activation functions in the graph convolutional network training process.

4. A news text classification method as claimed in claim 1, characterized in that: use BERT The specific process of extracting the contextual semantic features of the text by the model includes the following: BERT The model includes an encoder part, which is composed of multiple layers of encoders stacked together, each of which includes a self-attention layer and a feedforward network layer.

5. A news text classification method as claimed in claim 1, characterized in that: The specific process of using the multi-head attention mechanism to interact with the extracted text structure features and contextual semantic features includes calculating the query matrix, key and value matrix of the text structure features and contextual semantic features, and passing the keys and values of the text structure features and contextual semantic features as input to each other's multi-head attention modules.

6. A news text classification method as claimed in claim 1, characterized in that: The specific process of using different aggregation methods to aggregate the features after interaction includes using Max - pooling , concatenation and element-wise addition methods are used to aggregate the extracted features.

7. A news text classification system, characterized by comprising: a text graph construction module, configured to construct a separate text graph for each document storing the news text dataset based on word co-occurrence; a text graph enrichment module, configured to introduce a semantic dictionary to enrich the constructed text graph; The feature extraction module is configured to extract text structure features using graph convolutional networks for each enriched text graph, and use BERT The model extracts contextual semantic features of the text; The feature interaction module is configured to use a multi-head attention mechanism to interact with the extracted text structure features and contextual semantic features, so as to combine the features of two different granularities; An aggregation module is configured to aggregate the features after interaction using different aggregation methods to achieve classification of news texts; The normalized point-by-point mutual information method is used to calculate the edge weight between two words. The greater the frequency of co-occurrence of two words within a set range, the greater the weight. When the semantic relevance between words in the corpus exceeds the set value, the edge weight is positive, otherwise it is negative. Only when the edge weight is positive, an edge is created between the word pairs. The specific process of introducing a semantic dictionary to enrich the constructed text graph includes organizing the semantic dictionary into a vocabulary semantic network according to semantic relationships, representing vocabulary through synonym sets, each set indicating a vocabulary concept, and expressing semantic relationships including hyponym / hypernym, part-whole, synonym / antonym through links; If the edge weight is negative, use WordNet Expand synonyms, using WordNet of Wup Method to calculate the semantic similarity between words in two synsets.

8. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executing the steps of the method according to any one of claims 1 to 6.

9. A terminal device, characterized in that: The method comprises a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the steps in the method according to any one of claims 1 to 6.

Images