A short text stream classification method, medium, device and product

Abstract

The application discloses a short text stream classification method, medium, equipment and product, relates to the technical field of natural language processing, and comprises the following steps: performing word segmentation and part-of-speech tagging on a short text sample to obtain a text word sequence and a corresponding part-of-speech label sequence; performing concept expansion on the text word sequence to obtain a concept word sequence; constructing a short text stream classification model, including an attention-enhanced graph convolution network, a multi-resolution integrated feature extraction network and a classification network; the multi-resolution integrated feature extraction network comprises M parallel branches, each branch is a structure in which a Transformer encoder, the attention-enhanced graph convolution network and an attention pooling layer are connected in series; the feature dimension of each branch is gradually reduced layer by layer; and the classification network is a full-connection classifier connected behind the M parallel branches, the outputs of the M classifiers are averaged to obtain a final classification result. The application avoids noise interference, captures deep semantics, and realizes high-precision classification.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method, medium, device and product for classifying short text streams. Background Technology

[0002] With the advent of the mobile internet and big data era, social media platforms such as Twitter and Weibo continuously generate massive streams of short text data. These data streams, characterized by their real-time nature and rapid updates, serve as crucial sensors for monitoring social events and public sentiment. In modern comprehensive social governance systems, disputes and complaints reported by the public in real-time through government platforms, as well as grid worker patrol records, also constitute typical short text data streams. Efficiently classifying these data streams has become an important prerequisite for achieving intelligent grassroots governance. Unlike traditional long documents, effectively classifying these short text streams faces significant challenges.

[0003] First, semantic sparsity is an inherent characteristic of short texts. Due to their extremely limited length and lack of sufficient contextual information, traditional word frequency-based methods struggle to capture deep semantics. Second, data streams are often accompanied by concept drift, meaning data distribution dynamically evolves over time, such as the changing seasonal and policy-related conflicts in public security scenarios. Models that cannot quickly adapt to these changes will suffer from forgetting. Furthermore, real-world streaming data is often filled with peripheral topic noise—irrelevant topics not belonging to the dominant category persist, easily interfering with classifier judgments. Most importantly, acquiring labeled data from high-speed data streams is extremely costly, especially in grassroots government and public security operations where workload reduction is urgently needed. Relying on manual labeling is impractical, often resulting in a severe shortage of labels. This makes supervised learning methods that rely on large amounts of labeled data unsuitable.

[0004] While some classification methods exist for short texts or data streams, they still have significant limitations in the complex scenarios described above. 1. Pre-trained language models and large language models: such as BERT, T5, or GPT series models, although they capture rich contextual semantics through massive pre-training, have a huge number of parameters, and the cost of inference and fine-tuning is high, making it difficult to meet the real-time requirements of data stream processing. In addition, in the case of scarce labels, these models are prone to overfitting or relying on surface signals, and lack flexible update mechanisms when facing concept drift.

[0005] 2. Traditional Graph Neural Networks (GCN): Although graph networks can alleviate sparsity by utilizing global co-occurrence information, standard GCNs use a uniform aggregation mechanism, treating all neighbor nodes equally. This fails to distinguish between valid information and noise, causing noise to propagate throughout the graph. Furthermore, most graph models are static and cannot adapt to dynamically changing data flow structures.

[0006] 3. Existing stream classification methods, such as incremental learning frameworks like CDAGNN, while attempting to address the drift problem, often treat drift detection and feature learning as two separate processes. This makes it difficult for the model to distinguish between temporary edge noise fluctuations and real concept drift, easily leading to problems such as being overly sensitive to noise or slow to react to real drift.

[0007] Therefore, there is an urgent need for a short text stream classification method that can alleviate semantic sparsity by fusing multi-view information, suppress noise by using attention mechanisms, and unify feature learning and drift detection under the condition of scarce labels, so as to achieve robust classification of noisy and dynamic short text streams. Summary of the Invention

[0008] The purpose of this invention is to address the problems of traditional methods struggling to capture deep semantics and exhibiting weak noise resistance in short text classification with scarce labels and sparse semantics. This invention proposes a short text stream classification method, comprising the following steps: S1. Obtain short text samples, perform word segmentation and part-of-speech tagging on the short text samples to obtain text word sequences and corresponding part-of-speech tag sequences; perform concept expansion on the text word sequences to obtain concept word sequences; S2. Construct a short text stream classification model, including an attention-enhanced graph convolutional network, a multi-resolution ensemble feature extraction network, and a classification network; The graph structure in the attention-enhanced graph convolutional network is constructed using the union of words in the text word sequence and the concept word sequence as the node set; the attention-enhanced graph convolutional network outputs feature X; The multi-resolution integrated feature extraction network includes M parallel branches. Each branch is a structure consisting of a Transformer encoder, an attention-enhanced graph convolutional network, and an attention pooling layer. The feature X is used as the input to the Transformer encoder of the first branch. The input to the Transformer encoders of other branches is the output of the previous branch's Transformer encoder through a linear projection layer. The feature dimension of each Transformer encoder decreases layer by layer. The input to the attention-enhanced graph convolutional network of each branch is the output of the corresponding branch's Transformer encoder and the adjacency matrix of the graph structure; the input to the attention pooling layer is the output of the corresponding branch's attention-enhanced graph convolutional network. The classification network is a fully connected classifier connected to M parallel branches; S3, during classification model training, with The sum of the prediction errors of the M classifiers on labeled samples is used as the loss function to obtain the trained classification model. The final classification result is obtained by averaging the outputs of the M classifiers.

[0009] Furthermore, the adjacency matrix of the graph structure is:

[0010] Where A represents the adjacency matrix. The adjacency matrix of the text subgraph. For cross-view adjacency matrix, The adjacency matrix of the concept subgraph; in, and The subgraphs and concept subgraphs are constructed based on words in the text word sequence and concept word sequence, respectively, and are calculated in the same way. The calculation formula is:

[0011]

[0012] in, This is a part-of-speech co-occurrence intensity matrix. Indicates the element-product symbol. This represents the normalization operation function. Based on the adjacency weight matrix, The element in the i-th row and j-th column of the basic adjacency weight matrix. Represents the i-th word node in the text word sequence. and the j-th word node Co-occurrence probability within the sliding window Indicates the size of the sliding window. and Representing word nodes respectively and The marginal probability; Calculate the similarity between text subgraph nodes and concept subgraph nodes, retain the top-K connections with the highest similarity as semantic edges, and construct a cross-view adjacency matrix by combining the edges between lexical nodes in the text word sequence and their corresponding lexical nodes in the concept word sequence.

[0013] Furthermore, the attention-enhanced graph convolutional network includes parallel graph convolutional branches and self-attention branches. The output of the graph convolutional branches is as follows:

[0014] in, This represents the output features of the graph convolution branch. The normalized adjacency matrix represents the graph structure. This represents the initial feature vector of the graph node set mapping. The learnable weight matrix represents the graph convolutional branch; The output of the self-attention branch is as follows:

[0015]

[0016] in, This represents the output features of the self-attention branch. For query matrix, For the key matrix, For value matrices, Scaling factor This is the learnable weight matrix for the self-attention branch; Features With features After concatenation, a gated weight vector is generated through nonlinear transformation. and ; Using gated weight vectors and Features and characteristics We perform a weighted summation to obtain the output feature H.

[0017] Furthermore, the loss function is:

[0018] in, Let M be the loss function for the t-th batch. Indicates the first Each branch pairs labeled samples Predicted as true label The probability of.

[0019] Furthermore, the output of the attention pooling layer is expressed by the formula:

[0020]

[0021] in, Indicates the first The graph-level feature vectors output by the attention pooling layers of each branch, where N represents the total number of nodes. This represents the importance weight of the i-th node. Indicates the first Output features of attention-enhanced graph convolutional networks in each branch The Middle The feature representation of each node. For the weight vector, and This represents the learnable weights and bias parameters of the attention pooling layer.

[0022] Furthermore, the output of the fully connected classifier is represented as:

[0023] in, This represents the class probability distribution of the k-th branch. and For the first The weights and bias parameters of the fully connected classifiers in each branch.

[0024] The present invention also proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described short text stream classification method.

[0025] The present invention also proposes an electronic device, including a processor and a memory, wherein the processor is interconnected with the memory, the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to execute the above-described short text stream classification method.

[0026] The present invention also proposes a computer program product, including a computer program / instruction that, when executed by a processor, implements the steps of the short text stream classification method described above.

[0027] The beneficial effects of the technical solution provided by this invention are: (1) The graph structure in the attention-enhanced graph convolutional network of the present invention is constructed using the union of words in the text word sequence and the concept word sequence as the node set; the multi-resolution integrated feature extraction network includes M parallel branches, each of which is a structure of Transformer encoder, attention-enhanced graph convolutional network and attention pooling layer connected in series; the classification network is a fully connected classifier connected after the M parallel branches; the extended generated concepts are used to complete the view, resulting in richer and more discriminative feature representations, which can accurately understand the semantics of short texts in scenarios with scarce labels and sparse semantics; the resolution integrated feature extraction network constructs a semantic pyramid with progressively decreasing feature dimensions, gradually filtering edge noise, avoiding noise interference, capturing deep semantics, and achieving high-precision classification.

[0028] (2) The adjacency matrix of the present invention includes: text subgraph adjacency matrix, cross-view adjacency matrix and concept subgraph adjacency matrix, realizing multi-source information fusion and alleviating semantic sparsity of short text; the text subgraph adjacency matrix of the present invention fuses the co-occurrence intensity matrix through element product operation, the accidental co-occurrence edge is suppressed, the noise entering the graph structure is reduced, and the noise interference of feature aggregation is reduced. Attached Figure Description

[0029] Figure 1 This is a flowchart of a short text stream classification method according to an example of the present invention; Figure 2 This is a block diagram of an electronic device according to an exemplary embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0031] A flowchart of a short text stream classification method according to an example of the present invention is shown below. Figure 1 Specifically, it includes: S1. Obtain short text samples, perform word segmentation and part-of-speech tagging on the short text samples to obtain text word sequences and corresponding part-of-speech tag sequences; perform concept expansion on the text word sequences to obtain concept word sequences.

[0032] First, obtain the current time step. Batch of arriving short text stream data For data batches Each short text in Perform the following operations: (1) Word segmentation and cleaning: Using natural language processing tools, in this embodiment, NLTK or SpaCy tool libraries are preferred for short text segmentation and cleaning. Word segmentation is performed, and stop words and low-frequency words are removed to obtain the original text word sequence. ; (2) Part-of-speech tagging: For the original text word sequence Each word in the text is tagged with its part of speech to obtain the corresponding part-of-speech tag sequence. ; (3) Conceptual expansion: In order to enrich the semantics of short texts, external knowledge bases are used. In this embodiment, WordNet or a predefined thesaurus is preferred to be used to expand the original text word sequence. For each content word, perform a synonym lookup, select the synonym with the highest confidence level as the concept word, and generate a concept word sequence. .

[0033] S2. Construct a short text stream classification model, including an attention-enhanced graph convolutional network (GBR), a multi-resolution ensemble feature extraction network (MIG), and a classification network. The graph structure in the GBR is constructed using the union of words in the text word sequence and the concept word sequence as the node set; the GBR outputs a feature X. The MIG consists of M parallel branches, each branch being a structure of a Transformer encoder, an GBR, and an attention pooling layer connected in series. The output feature X serves as the input to the Transformer encoder of the first branch, while the inputs to the Transformer encoders of other branches are the features from the output of the previous branch's Transformer encoder passed through a linear projection layer; the feature dimensions of each Transformer encoder decrease progressively. The input to the GBR of each branch is the output of the corresponding branch's Transformer encoder and the adjacency matrix of the graph structure; the input to the attention pooling layer is the output of the corresponding branch's GBR. The classification network is a fully connected classifier connected to the M parallel branches.

[0034] (1) Attention-enhanced graph convolutional network ① Node set definition Original text word sequence The words in the text are used as text nodes to represent the sequence of concept words. The words in the graph are used as concept nodes, and the union of these two sets constitutes the node set of the current graph instance. .

[0035] ② Construct a text view subgraph To capture local and global dependencies between words and to filter noise using syntactic information, a node set is constructed. Adjacency matrix between Chinese text nodes : Calculating PMI weights: A fixed-size sliding window is set. In this embodiment, the sliding window size is... Set the value to 5, and slide it across the entire corpus. Count any two text word nodes. and The point mutual information (PMI) of the co-occurrence frequencies within the sliding window is calculated as follows:

[0036] in, Indicates the first The and the first The joint probability of 1 text node. and They represent the first The and the first The edge probabilities of each text node; if Then at node and Edges are established between the two adjacency pairs, with weights equal to the PMI value; otherwise, the weights are 0, thus obtaining the basic adjacency matrix. .

[0037]

[0038] in, The element in the i-th row and j-th column of the basic adjacency weight matrix. Represents the i-th word node in the text word sequence. and the j-th word node Co-occurrence probability within the sliding window Indicates the size of the sliding window. and Representing word nodes respectively and The marginal probability.

[0039] Introducing part-of-speech syntactic gating: Statistically calculating the global co-occurrence frequency of different part-of-speech pairs in the corpus, and constructing a part-of-speech co-occurrence matrix. The part-of-speech co-occurrence matrix As a gated filter, it is associated with the basic adjacency matrix. Perform element-wise product operation, as shown in the following formula:

[0040] in, This is a part-of-speech co-occurrence intensity matrix. Based on the adjacency weight matrix, This represents the normalization operation function. Represents the element product symbol.

[0041] This mechanism can enhance edges that conform to grammatical rules and suppress noisy edges that do not conform to grammar.

[0042] ③ Construct concept view subgraphs Using a sliding window to statistically analyze concept word sequences The PMI values ​​between each concept term are used to construct an adjacency matrix between concept terms. This allows us to capture the underlying semantic connections between abstract concepts.

[0043] ④ Construct cross-view interaction blocks To bridge the text space and the concept space, a connection matrix is ​​constructed between text nodes and concept nodes. It contains two types of edges: Alignment edge: Directly connects the original text word with its corresponding generated concept word, assigning it a fixed high weight. In this embodiment, Set to 1.0; Semantic edge: for any text node and concept nodes Calculate the cosine similarity between their corresponding pre-trained word vectors. .

[0044]

[0045] in, Represents the i-th text word node Embedded vector, Represents the j-th concept word node The embedding vector.

[0046] The top-K connections with the highest cosine similarity are retained as semantic edges. Combined with the alignment edges between text word nodes and their corresponding concept word nodes, a cross-view adjacency matrix is ​​constructed. .

[0047] ⑤ Adjacency Matrix Adjacency matrix of text subgraph Based on concept word sequence The generated concept subgraph adjacency matrix and cross-view adjacency matrix Concatenate to generate a unified adjacency matrix The calculation is as follows:

[0048] To prevent numerical instability during graph convolution, a unified adjacency matrix is ​​used. The normalized adjacency matrix is ​​obtained by performing symmetric normalization. The calculation is shown in the following formula:

[0049] in, Adjacency matrix The degree matrix, It is an identity matrix.

[0050] ⑥ Feature initialization Using pre-trained GloVe word vectors, in this embodiment, a 300-dimensional version of the Common Crawl is selected as the lookup table to store the node set. Each node in the matrix is ​​mapped to an initial feature vector, thus obtaining the initial node feature matrix. ,in For the number of nodes, feature dimension For out-of-vocabulary (OOV) words, random initialization is used. This completes the blending graph for the current short text. The graph was constructed by integrating lexical co-occurrence, syntactic constraints, and semantic extension information, providing a rich data foundation for subsequent graph convolution operations.

[0051] ⑦ Feature extraction and fusion using Attention-Enhanced Graph Convolutional Network (Att-GCN). The attention-enhanced graph convolutional network comprises two parallel branches: a graph convolution branch and a self-attention branch, and dynamically integrates their outputs through an adaptive cross-stitch fusion mechanism. The specific implementation process includes the following steps: Step 1: Extract local structural features using graph convolutional branches. This branch is primarily used to capture local topological information in short texts based on lexical co-occurrence and syntactic dependencies. Its input is a normalized adjacency matrix. and the initial node feature matrix ,in For the number of nodes, For the feature dimension, first utilize a learnable weight matrix. Perform a linear transformation on the features, and then use the normalized adjacency matrix. The feature information of aggregated neighboring nodes is calculated as shown in the following formula:

[0052] in, The ReLU activation function is used to introduce nonlinearity. This is the local structural feature matrix output by the graph convolution branch.

[0053] In this preferred embodiment, let the input feature dimension be... (i.e., the dimension of the pre-trained GloVe word vectors), let the output dimension of the graph convolution branch be... This is to achieve dimensionality reduction and condensation of features.

[0054] Self-attention branches capture global semantic dependencies: This branch is primarily used to overcome the limitations of graph structures and capture long-distance semantic dependencies between non-adjacent words in short texts. Unlike traditional self-attention mechanisms, this embodiment introduces graph structure information when generating keys and values ​​to enhance the robustness of semantic representation. Query Matrix Key matrix Sum matrix The calculation is shown in the following formula:

[0055] in, , and The learnable weight matrix for the self-attention branch; the key matrix. Sum matrix The calculation utilizes a normalized adjacency matrix. Pre-smooth the context.

[0056] Calculate the query matrix AND key matrix The dot product is then scaled and Softmax normalized to obtain the global attention map; subsequently, the global attention map is used to adjust the value matrix. We perform a weighted summation to obtain the global semantic feature matrix output by the self-attention branch. The calculation is shown in the following formula:

[0057] in, This is a scaling factor used to prevent the gradient from vanishing due to an excessively large inner product.

[0058] Step 2, Adaptive Cross-Stitch Fusion To address the problem that traditional simple additive fusion methods cannot distinguish between noise and valid information, this embodiment introduces a learnable gating mechanism that dynamically balances the contributions of the two branches along the feature channel dimension.

[0059] Local structural feature matrix With global semantic feature matrix By concatenating along the channel dimension, combined features are obtained. Using a multilayer perceptron and a sigmoid activation function, gated weight vectors are generated respectively. and (dimension is) For each feature channel, the calculation is shown in the following formula:

[0060]

[0061] and For the weight parameters of the gating network, and This is a bias term.

[0062] Using the generated gate weight vector and The original branch features are multiplied element-wise and summed to obtain the final enhanced node features. (The output features of the attention-enhanced graph convolutional network) are calculated as follows:

[0063] in, This indicates element-wise multiplication.

[0064] When the local topology contains a lot of edge noise, the gated network will automatically reduce... The value of suppresses noise propagation in the graph convolution branch while improving . The value of depends more on global semantic features, thus significantly enhancing the robustness of the model in noisy environments.

[0065] (2) Multi-resolution integrated feature extraction To address the issues of edge-topic noise interference and extreme tag scarcity in short text streams, a multi-resolution cascaded architecture with multiple parallel branches is constructed. The multi-resolution integrated feature extraction network includes M parallel branches. In this preferred embodiment, Let's set it to 3. Each branch consists of a Transformer encoder, an attention-enhanced graph convolutional network, and an attention pooling layer connected in series. The output feature X of the attention-enhanced graph convolutional network is used as the input to the Transformer encoder of the first branch. The input to the Transformer encoders of other branches is the feature of the previous branch's Transformer encoder output passed through a linear projection layer. The feature dimensions of each Transformer encoder decrease layer by layer, forming an inverted pyramid structure. The feature dimension of the first branch (high resolution) is set to... (Corresponding to BERT-Base dimension), used to preserve fine-grained semantic details (such as keywords); the feature dimension of the second branch (medium resolution) is set to... Used to extract mid-level semantic patterns; the feature dimension of the third branch (low resolution) is set to... It is used to extract highly abstract semantic skeletons to filter out non-dominant edge noise.

[0066] The Transformer encoder in the k-th branch uses a Transformer encoder module with a multi-head self-attention mechanism to capture the input sequence. Long-range contextual dependencies are used to obtain intermediate features. In this embodiment, the Transformer encoder module uses two Transformer Blocks to capture the sequential evolution features of text.

[0067] The projected features of the output of the previous branch Transformer encoder are as follows:

[0068] in, represents the weight matrix of the linear projection layer. This design ensures that the low-resolution branch inherits the semantic information learned by the high-resolution branch, while forcing the model to discard redundant details and retain only the most discriminative core semantics.

[0069] intermediate features As a node feature, combined with the constructed hybrid graph adjacency matrix The input is placed into the attention-enhancing graph convolutional network, where the structure of the attention-enhancing graph convolutional network is consistent with that constructed in step (1) above. Node-level features are then obtained. .

[0070]

[0071] in, This represents an attention-enhanced graph convolutional network with the k-th branch. It utilizes the graph structure to propagate the supervisory signal and leverages neighbor node information to smooth features, further suppressing isolated noisy nodes.

[0072] Utilize attention pooling module to extract node-level features Convert the data into a graph-level (document-level) vector representation, and automatically calculate the importance weight of each node using an attention mechanism. The calculation is shown in the following formula:

[0073]

[0074] in, Indicates the first The graph-level feature vectors output by the attention pooling layers of each branch, where N represents the total number of nodes. This represents the importance weight of the i-th node. Indicates the first Output features of attention-enhanced graph convolutional networks in each branch The Middle The feature representation of each node. For the weight vector, and This represents the learnable weights and bias parameters of the attention pooling layer.

[0075] Will Graph-level feature vectors of each branch Each input is fed into a fully connected classifier to obtain the class probability distribution of that branch. The calculation is shown in the following formula:

[0076] in, and For the first The weights and bias parameters of each branch of the fully connected classifier. The final classification result. By all The prediction results for each branch are determined by the arithmetic mean, calculated as follows:

[0077] The confidence level will be high when multiple branches with different resolutions tend to fall into the same category, thus greatly reducing the misjudgment caused by noise in a single branch.

[0078] S3, during classification model training, with The sum of the prediction errors of the M classifiers on labeled samples is used as the loss function to obtain the trained classification model. The final classification result is obtained by averaging the outputs of the M classifiers.

[0079] The loss function during training is:

[0080] in, Let M be the loss function for the t-th batch. Indicates the first Each branch pairs labeled samples Predicted as true label The probability of.

[0081] Minimize the loss function using the backpropagation algorithm Simultaneously update the model parameters of the Transformer encoder, attention-enhanced graph convolutional network, and classifier in all branches.

[0082] Because social media data distribution changes rapidly over time, static models are prone to forgetting. This step involves real-time monitoring of three complementary drift detection metrics and triggering an adaptive adjustment mechanism when a true drift is detected. The specific implementation process includes the following sub-steps: Step 1: Real-time calculation of drift detection indicators In each training batch At the end, calculate the following three metrics to evaluate the current model state: (1) Output distribution consistency Principle: Monitor whether the predicted category distribution of the current batch of samples changes drastically.

[0083] Calculate: Calculate the current batch The current distribution vector is obtained by averaging the predicted probability vectors of all samples. ; Calculate the current distribution vector Distribution vector at the previous time step cosine similarity The calculation is shown in the following formula:

[0084] Judgment: Set a similarity threshold In this preferred embodiment, the similarity threshold Let it be 0.7. If the calculated cosine similarity satisfies... If the ODC indicator is triggered, it indicates that the dominant topic may have shifted.

[0085] (2) Feature space similarity decay Principle: Monitor whether the semantic features of the current sample deviate from the feature prototypes of historical memory.

[0086] Calculation: Maintain a historical feature similarity benchmark. (Updated via moving average); Calculate the average similarity of features for the current batch of samples. ; Calculate the relative decline rate The calculation is shown in the following formula:

[0087] in, To prevent smoothing terms with a denominator of zero, in this embodiment, Set as .

[0088] Judgment: Set a descent threshold In this embodiment, the threshold for decreasing Set it to 0.15. If the relative rate of decline satisfies... If the FSSD indicator is triggered, it indicates that the semantic space of the input data has shifted significantly.

[0089] (3) Performance degradation monitoring Principle: Directly monitor whether the classification accuracy (or confidence) of the model on the current batch of labeled samples decreases.

[0090] Calculate: the accuracy rate recorded at the previous moment. Accuracy at the current moment .

[0091] Judgment: Set an accuracy decline threshold. In this embodiment, the accuracy decrease threshold Set it to 0.1. If it satisfies... If so, the PDM indicator is determined to be triggered.

[0092] Step 2, Drift Detection Logic Joint voting mechanism: In order to distinguish between temporary marginal topic noise and real concept drift, this embodiment adopts a two-out-of-three decision logic.

[0093] Triggering conditions: The system will officially determine that a real concept drift has occurred only when at least two of the above three indicators, ODC, FSSD, and PDM, are triggered simultaneously.

[0094] Fluctuations in a single metric are usually caused by noise, while true drift usually causes a chain reaction in the feature space, output distribution, and accuracy.

[0095] Step 3, Adaptive Response Mechanism Once a true concept drift is determined to have occurred, the system immediately performs the following two adjustment operations: (1) Activate dual-model fusion Principle: In the early stages of drift, graph structures may fail, while pre-trained language models (such as BERT) are more robust due to their general semantic knowledge.

[0096] Operation: In the following Within a batch, in this embodiment, the batch window is merged. Set to 10, and temporarily use the prediction results from the BERT model for assistance. Final predicted probability. The modified weighted sum is calculated as follows:

[0097] in, The predicted probability of the graphical model. The predicted probabilities of the pre-trained language model; and These are the fusion weights for the two components. In this embodiment, both are initially set to 0.5 and gradually increased over time. The proportion of.

[0098] (2) Dynamically adjust the learning rate Principle: When drift occurs, the model needs a larger step size to escape the current local optimum in order to adapt to the new data distribution.

[0099] Operation: Based on the absolute decrease in accuracy Adjusting the learning rate : Severe drift: If the decline is significant In this embodiment, the severely decreased threshold Then reset the learning rate to a high value. (For example To achieve rapid adaptation; General drift: If the decline is not severe enough, set the learning rate to a critical value. (For example ), and make gentle adjustments; No drift: Maintain the default learning rate (e.g.) Alternatively, it can be executed according to the preset attenuation strategy.

[0100] Through the above steps, this embodiment can keenly capture concept drift and quickly restore classification performance under conditions of extreme label scarcity and noise interference, ensuring the stability of the model in long-term data streams.

[0101] Experimental verification: This invention addresses the short text stream classification problem under conditions of extreme label scarcity, persistent multi-topic interference, and concept drift, proposing the MTAD-GNN model. The main objective of the experiments is to examine the model's classification capabilities in the following scenarios: 1. Short texts themselves contain less semantic information, resulting in semantic sparsity. 2. Data arrives continuously in streaming batches, and the dominant theme changes over time; 3. Only a very small number of labels can be used for training in each batch; the remaining labels are used only for offline evaluation. 4. Non-dominant categories are not pure noise, but rather persistent multi-topic interference.

[0102] To simulate the above scenario, the data was organized into continuously arriving batches, each containing 50 short texts. Approximately 70% of the samples came from the current dominant category, while the remaining 30% came from other categories to simulate continuous multi-topic interference. For supervision, only one ground truth label was randomly revealed for each batch to calculate the training loss, corresponding to approximately 2% label visibility. The model first predicted the current batch, then updated its parameters using a small number of delayed labels; batch accuracy was calculated using the complete ground truth label of the batch during evaluation. It is important to note that the 70% dominant category proportion was only used to construct and describe the data flow; the dominant category itself was not provided to the model as additional supervision information. Therefore, this experiment examines whether the model can achieve online adaptation using a very small number of labels, graph structure, and historical information without knowing the current dominant topic.

[0103] The experiment used four short text benchmark datasets: AG-News, Snippets, TagMyNews, and Twitter. Dataset statistics are shown in Table 1, including the total number of texts, the average length of the short texts, and the number of drifts. The average length of the short texts can be used to intuitively reflect the semantic sparsity of the text. The shorter the average length, the less context a single text can typically provide, and the more the classification relies on cross-text structural relationships, global co-occurrence information, and effective supervised propagation.

[0104] Table 1

[0105] The performance of the proposed method (MTAD-GNN) and several comparative models was compared on four datasets. The performance metric used was incremental accuracy (IA). The comparison results are shown in Table 2. The results in Table 2 are written in the form of "mean ± fluctuation term", where the first number is the mean IA and the second number represents the degree of fluctuation in the experimental results. The smaller the fluctuation, the more stable the model performance is generally. The comparative models include: static semi-supervised or semantic enhancement methods: MixText and CRFA; streaming or concept drift-oriented methods: CDAGNN; sequence models, replay models and weakly supervised baseline models: BiLSTM, GRU, ER-GNN, TWP, NPPorpt, SeedBTM and CL-WSTC; pre-trained language models and large language model baseline models: T5 (Standard), T5 (10% Label), DeepSeek-V3, GLM-Z1-32B and GPT-5.1, and the MTAD-GNN proposed in this invention.

[0106] Table 2

[0107] As can be seen from Table 2: (1) MTAD-GNN achieved the best results on AG-News, Snippets and TagMyNews. Their IAs were 0.886, 0.921 and 0.762, respectively, indicating that on these datasets, hybrid graph representation and propagation with a small number of labels can effectively alleviate the problems of semantic sparsity and insufficient labels in short texts.

[0108] (2) On Twitter, MTAD-GNN achieved a score of 0.947 ± 0.002. This result is higher than most baselines in the table, but lower than T5 (Standard) at 0.994 ± 0.001. It remains highly competitive on Twitter.

[0109] (3) MTAD-GNN has relatively small fluctuation terms. For example, the fluctuation terms on its four datasets are all approximately 0.002-0.004, indicating that the model performs relatively stably in different batches or repeated experiments. Stability is especially important for concept drift and continuous disturbance environments.

[0110] In one exemplary embodiment, a computer-readable storage medium is included, which stores a computer program that, when executed by a processor, implements the short text stream classification method described above.

[0111] Please see Figure 2 In one exemplary embodiment, the device further includes an electronic device including at least one processor, at least one memory, and at least one communication bus.

[0112] The memory stores a computer program, which includes computer-readable instructions. The processor calls the computer-readable instructions stored in the memory through the communication bus to execute the short text stream classification method described above.

[0113] In one exemplary embodiment, a computer program product is proposed, including a computer program / instructions that, when executed by a processor, implement the steps of the short text stream classification method described above.

[0114] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A short text stream classification method, characterized in that, Includes the following steps: S1. Obtain short text samples, perform word segmentation and part-of-speech tagging on the short text samples to obtain text word sequences and corresponding part-of-speech tag sequences; perform concept expansion on the text word sequences to obtain concept word sequences; S2. Construct a short text stream classification model, including an attention-enhanced graph convolutional network, a multi-resolution ensemble feature extraction network, and a classification network; The graph structure in the attention-enhanced graph convolutional network is constructed using the union of words in the text word sequence and the concept word sequence as the node set; the attention-enhanced graph convolutional network outputs feature X; The multi-resolution integrated feature extraction network includes M parallel branches. Each branch is a structure consisting of a Transformer encoder, an attention-enhanced graph convolutional network, and an attention pooling layer. The feature X is used as the input to the Transformer encoder of the first branch. The input to the Transformer encoders of other branches is the output of the previous branch's Transformer encoder through a linear projection layer. The feature dimension of each Transformer encoder decreases layer by layer. The input to the attention-enhanced graph convolutional network of each branch is the output of the corresponding branch's Transformer encoder and the adjacency matrix of the graph structure; the input to the attention pooling layer is the output of the corresponding branch's attention-enhanced graph convolutional network. The classification network is a fully connected classifier connected to M parallel branches; S3, during classification model training, with The sum of the prediction errors of the M classifiers on labeled samples is used as the loss function to obtain the trained classification model. The final classification result is obtained by averaging the outputs of the M classifiers.

2. The short text stream classification method according to claim 1, characterized in that, The adjacency matrix of the graph structure is: Where A represents the adjacency matrix. The adjacency matrix of the text subgraph. For cross-view adjacency matrix, The adjacency matrix of the concept subgraph; in, and The subgraphs and concept subgraphs are constructed based on words in the text word sequence and concept word sequence, respectively, and are calculated in the same way. The calculation formula is: in, This is a part-of-speech co-occurrence intensity matrix. Indicates the element-product symbol. This represents the normalization operation function. Based on the adjacency weight matrix, The element in the i-th row and j-th column of the basic adjacency weight matrix. Represents the i-th word node in the text word sequence. and the j-th word node Co-occurrence probability within the sliding window Indicates the size of the sliding window. and Representing word nodes respectively and The marginal probability; Calculate the similarity between text subgraph nodes and concept subgraph nodes, retain the top-K connections with the highest similarity as semantic edges, and construct a cross-view adjacency matrix by combining the edges between lexical nodes in the text word sequence and their corresponding lexical nodes in the concept word sequence.

3. The short text stream classification method according to claim 1, characterized in that, The attention-enhanced graph convolutional network contains parallel graph convolutional branches and self-attention branches. The output of the graph convolutional branch is as follows: in, This represents the output features of the graph convolution branch. The normalized adjacency matrix represents the graph structure. This represents the initial feature vector of the graph node set mapping. The learnable weight matrix represents the graph convolutional branch; The output of the self-attention branch is as follows: in, This represents the output features of the self-attention branch. For query matrix, For the key matrix, For value matrices, Scaling factor This is the learnable weight matrix for the self-attention branch; Features With features After concatenation, a gated weight vector is generated through nonlinear transformation. and ; Using gated weight vectors and Features and characteristics We perform a weighted summation to obtain the output feature H.

4. The short text stream classification method according to claim 1, characterized in that, The loss function is: in, Let M be the loss function for the t-th batch. Indicates the first Each branch pairs labeled samples Predicted as true label The probability of.

5. The short text stream classification method according to claim 1, characterized in that, The output of the attention pooling layer can be expressed by the following formula: in, Indicates the first The graph-level feature vectors output by the attention pooling layers of each branch, where N represents the total number of nodes. This represents the importance weight of the i-th node. Indicates the first Output features of attention-enhanced graph convolutional networks in each branch The Middle The feature representation of each node. For the weight vector, and This represents the learnable weights and bias parameters of the attention pooling layer.

6. The short text stream classification method according to claim 1, characterized in that, The output of a fully connected classifier is represented as: in, This represents the class probability distribution of the k-th branch. and For the first The weights and bias parameters of the fully connected classifiers in each branch.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

8. An electronic device, characterized in that, The device includes a processor and a memory, the processor being interconnected with the memory, wherein the memory is used to store a computer program, the computer program including computer-readable instructions, and the processor is configured to invoke the computer-readable instructions to perform the method as described in any one of claims 1 to 6.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method described in any one of claims 1 to 6.

Images