A short text clustering method based on multi-view alignment and optimal transmission
By employing a multi-view alignment and optimal transmission method, the problems of semantic sparsity and view inconsistency in short text clustering are solved, achieving more stable clustering results, especially significantly improving clustering accuracy and robustness in imbalanced class scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUN YAT SEN UNIV
- Filing Date
- 2026-01-19
- Publication Date
- 2026-06-05
AI Technical Summary
Existing short text clustering methods have shortcomings in terms of semantic sparsity, view inconsistency, and accumulation of pseudo-label noise. They are difficult to obtain stable and reliable clustering results under multi-view consistency modeling and global allocation constraints, and their performance degrades, especially in scenarios with uneven class distribution.
We employ a multi-view alignment and optimal transmission approach, which involves data augmentation using short text samples to generate multiple views and perform comparative learning. We combine graph convolutional networks to construct sentence-level and cluster-level structural views, optimize the loss function to stabilize clustering results, reduce assignment bias, and suppress noise propagation.
It improves the clustering accuracy and robustness of short text clustering, especially under conditions of semantic sparsity and class imbalance, showing higher stability and consistency, thus enhancing the reliability and engineering usability of the system.
Smart Images
Figure CN122153059A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence comparative learning, specifically to a short text clustering method based on multi-view alignment and optimal transmission. Background Technology
[0002] In recent years, with the rapid development of natural language processing, self-supervised representation learning, and large-scale pre-trained language models, automatic clustering and topic organization technologies for massive texts have received widespread attention in applications such as sentiment analysis, search intent aggregation, user feedback merging, and knowledge base construction. Compared with long texts, short texts typically contain only a small number of words or fragmented information, with sparse semantic cues and insufficient context, resulting in more ambiguous semantic boundaries and more difficult-to-characterize relationships between samples. Therefore, how to learn discriminative semantic representations under unlabeled conditions and stably form a consistent clustering structure has become one of the key issues for the continued development of the short text clustering field.
[0003] At the key technology level, existing short text clustering methods have generally evolved from "word frequency statistical features + traditional clustering" to "deep representation learning + clustering optimization". Early methods mostly used sparse vector representations based on word frequency, such as BOW and TF-IDF, and combined them with algorithms such as k-means and spectral clustering to complete clustering; however, such representations ignore word order and contextual semantics, and are high in dimensionality and sparsity, making it difficult to model semantic similarity. Subsequently, Word2Vec, GloVe, and sentence vector encoders based on pre-trained language models gradually became mainstream, enabling short texts to be mapped to a continuous semantic space, providing a stronger representational foundation for clustering. Furthermore, the contrastive clustering framework that has emerged in recent years introduces a contrastive learning objective on top of the pre-trained representation. By constructing positive sample pairs between the original text and the enhanced text, it brings semantically consistent representations closer and pushes away irrelevant samples, thereby improving the separability between clusters; at the same time, it combines iterative clustering optimization, continuously correcting the representation space through soft assignment and cluster center updates, so that the clustering structure gradually converges.
[0004] While the aforementioned technologies have significantly advanced the development of short text clustering, key shortcomings remain in practical applications that are most relevant to this invention and have not yet been fully addressed. First, existing contrastive clustering methods typically rely on an explicit augmentation mechanism of "original view + single augmented view" to construct contrastive learning signals. Due to the highly sparse semantics of short texts, the semantic granularity provided by a single augmented view is limited, and the augmentation process may introduce semantic drift, making it difficult for the model to consistently obtain richer and more reliable semantic cues. Second, existing methods generally rely on iterative pseudo-labels or soft assignment for self-training during the clustering optimization phase. However, in the absence of global consistency constraints, pseudo-label noise easily accumulates and propagates with each training epoch, leading to cluster center drift, assignment oscillations, and decreased semantic consistency. Especially in short text data with significantly unbalanced category distribution, cluster assignment may also tilt towards the majority clusters, causing smaller clusters to be merged and triggering structural biases. On the one hand, many methods rely solely on comparative learning between the original view and a single augmented view. The semantic granularity provided by the augmented text is limited and prone to semantic drift, making it difficult for the model to obtain richer structured semantic cues. On the other hand, the clustering process often relies on iterative pseudo-label self-training, but lacks global consistency constraints. Pseudo-label noise easily accumulates and propagates during training, leading to cluster center drift, unstable allocation, and cross-view semantic inconsistency. Performance degradation is particularly pronounced in scenarios with extremely short semantics, high noise, or severe class imbalance. These problems make existing methods unable to meet the requirements of real-world systems for the stability, interpretability, and consistency of clustering results, highlighting the need for short text clustering in multi-view systems. Figure 1 Technical shortcomings in consistency modeling and global constraint allocation. Summary of the Invention
[0005] To overcome the shortcomings of existing short text clustering methods in terms of semantic sparsity, view inconsistency, and pseudo-label noise accumulation, this invention proposes a short text clustering method based on multi-view alignment and optimal transmission.
[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: including the following steps:
[0007] Step 1: Obtain short text samples, perform data augmentation on the short text samples to obtain augmented short text samples, and input the short text samples and augmented short text samples into a pre-trained language model encoder to obtain the original text embedding and the augmented text embedding, respectively. Step 2: Generate the original semantic view based on the original text embedding, generate the explicit augmented view based on the augmented text embedding, and obtain the negative sample view from the intermediate layer of the pre-trained language model encoder based on the original text embedding; Step 3: Use the k-means clustering algorithm to obtain the first initial cluster center and the second initial cluster center for the original text embedding and the enhanced text embedding, respectively. Obtain a sentence-level structure view based on the original text embedding and a cluster-level structure view based on the first initial cluster center. Step 4: Perform comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structure view, and the cluster-level structure view to obtain the comparative loss function; Step 5: Obtain the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center; Step 6: Obtain the overall loss function based on the contrast loss function and the optimization loss function, and train the pre-trained language model encoder based on the overall loss function to obtain the trained encoder; Step 7: Input real-time short text samples into the trained encoder, apply the k-means clustering algorithm to the original text embeddings to obtain clustering results, and complete the clustering of short texts.
[0008] Furthermore, the step of data augmentation of short text samples to obtain augmented short text samples includes: for each input short text sample, using a BERT-based context augmenter to perform 20% random word replacement to generate a corresponding augmented short text sample.
[0009] Further, the step of obtaining a sentence-level structural view based on the original text embedding and a cluster-level structural view based on the first initial cluster center includes: Each original text embedding and each first initial cluster center are treated as a node. A sentence-level structure graph is obtained based on the nodes of the original text embeddings, and a cluster-level structure graph is obtained based on the nodes of the first initial cluster centers. The sentence-level structure graph and the cluster-level structure graph are respectively input into a graph convolutional network to obtain the encoded sentence-level structure graph and the encoded cluster-level structure graph, respectively. Then, the encoded sentence-level structure graph is mapped to obtain a sentence-level structure view through a sentence-level propagation matrix, and the encoded cluster-level structure graph is mapped to obtain a cluster-level structure view through a cluster-level propagation matrix.
[0010] Furthermore, the sentence-level structure diagram It includes the original node set, the original node feature matrix, and the sentence-level adjacency matrix, as shown in the formula:
[0011] In the formula, the original set of nodes is , This represents the B-th original node. B Indicates the batch size; the original node feature matrix is... , Indicates the embedding of the original text. Represents sentence embedding dimension, sentence-level adjacency matrix It is calculated using cosine similarity, and the formula is:
[0012] In the formula, T is the transpose. Represented as the first i A sample of original text embeddings, Represented as the first j A sample of original text embeddings; The cluster-level structure diagram It includes the cluster-level node set, the cluster-level node feature matrix, and the cluster-level adjacency matrix, as shown in the formula:
[0013] In the formula, the cluster-level node set , Indicates the first K Cluster-level nodes, K Represents the number of cluster centers, cluster-level node characteristic matrix , Represents the cluster center dimension, cluster-level adjacency matrix Similarly, it is calculated using the cosine similarity between cluster centers, and the formula is:
[0014] In the formula, Indicates the first k Cluster center vectors, Indicates the first l Cluster center vectors.
[0015] Furthermore, the sentence-level propagation matrix It is obtained from the normalized sentence-level adjacency matrix, and the formula is:
[0016] In the formula, This indicates that the similarity matrix has been normalized. The cluster-level propagation matrix It uses Student's t-distribution to calculate the soft membership matrix under the original semantic view. The formula is:
[0017] In the formula, Define the hyperparameters for the degrees of freedom. .
[0018] Furthermore, the comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structural view, and the cluster-level structural view yields a comparative loss function, including: Combine explicit enhanced views, sentence-level structured views, and cluster-level structured views into a multi-view set; Positive items and a first negative item are obtained based on the original semantic view and the multi-view set. A second negative item is obtained based on the negative sample view and the multi-view set. The first negative item and the second negative item are added together to obtain the total negative item. The contrast loss function is obtained based on the positive terms and the total negative terms.
[0019] Further, the step of obtaining positive items and a first negative item based on the original semantic view and the multi-view set, obtaining a second negative item based on the negative sample view and the multi-view set, and adding the first negative item and the second negative item to obtain the total negative item includes: Take the i-th sample in the original semantic view as the anchor sample, and compare it with the multi-view set to obtain the positive sample pair. Then, the similarity scores of all positive samples are aggregated to obtain the positive terms. The formula is: Multiview Collection
[0020] In the formula, Representation of an explicitly enhanced view. and These represent the sample-level representations of the sentence-level structure view and the cluster-level structure view, respectively.
[0021] In the formula, Temperature coefficient; The samples in the original semantic view, excluding the anchor samples, are also compared with the multi-view set to obtain the first negative sample pair. Then, the similarity of all first negative samples is aggregated to obtain the negative terms to be weighted, and a clustering relation matrix is constructed based on the soft membership matrix of the original semantic view. S The first negative term is obtained by weighting the negative terms to be weighted using the same cluster relation matrix as the weights. The formula is: Cluster relation matrix S ,
[0022] In the formula, h≠i, h、t {1, ..., B};
[0023] In the formula, For the hyperparameter of the reweighted exponent; For the negative sample view By comparing the samples with a multi-view set, the second negative sample pair is obtained, and the similarity of all second negative sample pairs is aggregated to obtain the second negative term. The formula is:
[0024] In the formula, This represents the B-th negative sample;
[0025] The sum of the first and second negative terms equals the total negative term. The formula is:
[0026] Furthermore, the comparative loss function is obtained based on the positive terms and the total negative terms. The formula is:
[0027] Further, the step of obtaining the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center includes: The soft membership matrix of the original semantic view is obtained based on the original text embedding and the first initial cluster center. This matrix is then squared-sharpened and normalized to obtain the target distribution. The soft membership matrix of the original semantic view and the target distribution are aligned according to the cross-entropy constraint to obtain the first optimized loss function. The formula is: Target distribution P,
[0028]
[0029]
[0030] A cost matrix is constructed based on the soft membership matrix of the original semantic view, and a hard pseudo-label matrix is obtained from the cost matrix. The soft membership matrix of the explicit enhanced view is obtained based on the enhanced text embedding and the second initial cluster center. The second optimization loss function is obtained by aligning the soft membership matrix and hard pseudo-label matrix of the explicit augmented view based on the cross-entropy constraint. The formula is:
[0031] The rules for generating the hard pseudo-label matrix are as follows:
[0032] In the formula, Represents the original semantic view.i Optimal transmission constraint pseudo-labels for each sample. Represents the transmission matrix;
[0033] The optimization loss function is obtained by adding the first optimization loss function and the second optimization loss function, as shown in the formula: .
[0034] Furthermore, obtaining the overall loss function based on the contrastive loss function and the optimization loss function includes weighting the contrastive loss function and the optimization loss function according to their different training contributions, and then summing the weighted contrastive loss function and the optimization loss function to obtain the overall loss function. The formula is:
[0035] In the formula, and These represent the weights of the contrastive loss function and the optimization loss function in the overall training loss function, respectively.
[0036] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: By fusing explicit augmented views, sentence-level structural views, and cluster-level structural views, information from the original semantic view is stably transferred to other views, thereby learning multi-granular and structurally consistent semantic representations and improving the separability of cluster boundaries and clustering stability. Furthermore, this invention aligns the explicit augmented views with an optimized loss function obtained from the original text embedding, the first initial cluster center, the augmented text embedding, and the second initial cluster center, reducing assignment bias and suppressing noise propagation. Experimental results demonstrate that the method of this invention achieves higher clustering accuracy and stronger robustness on various short text benchmark datasets, especially under conditions of semantic sparsity and class imbalance, overcoming the limitations of existing technologies in cross-view... Figure 1 Despite limitations in consistency and global allocation stability, this innovative method not only improves the reliability and engineering usability of short text clustering systems, but also provides new technical paths and support for text classification and information aggregation applications in multiple scenarios. Attached Figure Description
[0037] Figure 1 This is a flowchart of a short text clustering method based on multi-view alignment and optimal transmission.
[0038] Figure 2 This is a diagram illustrating the overall training framework for a short text clustering method based on multi-view alignment and optimal transmission. Detailed Implementation
[0039] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0040] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.
[0041] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0042] In the first embodiment of the present invention, a short text clustering method based on multi-view alignment and optimal transmission is provided, combined with Figure 1 and Figure 2 Explanation: Step 1: Obtain short text samples, perform data augmentation on the short text samples to obtain augmented short text samples, and input the short text samples and augmented short text samples into a pre-trained language model encoder to obtain the original text embedding and the augmented text embedding, respectively. Furthermore, the step of data augmentation of short text samples to obtain augmented short text samples includes: for each input short text sample, using a BERT-based context augmenter to perform 20% random word replacement to generate a corresponding augmented short text sample.
[0043] Step 2: Generate the original semantic view based on the original text embedding, generate the explicit augmented view based on the augmented text embedding, and obtain the negative sample view from the intermediate layer of the pre-trained language model encoder based on the original text embedding; Step 3: Use the k-means clustering algorithm to obtain the first initial cluster center and the second initial cluster center for the original text embedding and the enhanced text embedding, respectively. Obtain a sentence-level structure view based on the original text embedding and a cluster-level structure view based on the first initial cluster center. Further, the step of obtaining a sentence-level structural view based on the original text embedding and a cluster-level structural view based on the first initial cluster center includes: Each original text embedding and each first initial cluster center are treated as a node. A sentence-level structure graph is obtained based on the nodes of the original text embeddings, and a cluster-level structure graph is obtained based on the nodes of the first initial cluster centers. The sentence-level structure graph and the cluster-level structure graph are respectively input into a graph convolutional network to obtain the encoded sentence-level structure graph and the encoded cluster-level structure graph, respectively. Then, the encoded sentence-level structure graph is mapped to obtain a sentence-level structure view through a sentence-level propagation matrix, and the encoded cluster-level structure graph is mapped to obtain a cluster-level structure view through a cluster-level propagation matrix.
[0044] Furthermore, the sentence-level structure diagram It includes the original node set, the original node feature matrix, and the sentence-level adjacency matrix, as shown in the formula:
[0045] In the formula, the original set of nodes is , This represents the B-th original node. B Indicates the batch size; the original node feature matrix is... , Indicates the embedding of the original text. Represents sentence embedding dimension, sentence-level adjacency matrix It is calculated using cosine similarity, and the formula is:
[0046] In the formula, T is the transpose. Represented as the first i A sample of original text embeddings, Represented as the first j A sample of original text embeddings; The cluster-level structure diagram It includes the cluster-level node set, the cluster-level node feature matrix, and the cluster-level adjacency matrix, as shown in the formula:
[0047] In the formula, the cluster-level node set , Indicates the first K Cluster-level nodes, K Represents the number of cluster centers, cluster-level node characteristic matrix , Represents the cluster center dimension, cluster-level adjacency matrix Similarly, it is calculated using the cosine similarity between cluster centers, and the formula is:
[0048] In the formula, Indicates the first k Cluster center vectors, Indicates the first l Cluster center vectors.
[0049] The graph convolutional network (GCN) defines the update rule for any structure graph as follows:
[0050] In the formula, Let I represent the adjacency matrix after adding the self-loop, and let I be the identity matrix. It is a degree matrix and , For trainable weight matrix, It is a non-linear activation function; The sentence-level structure graph and cluster-level structure graph can be processed by GCN to obtain the encoded sentence-level structure graph. and the encoded cluster-level structure diagram The formula is:
[0051] In the formula, ; Furthermore, the sentence-level propagation matrix It is obtained from the normalized sentence-level adjacency matrix, and the formula is:
[0052] In the formula, This indicates that the similarity matrix has been normalized. The cluster-level propagation matrix It uses Student's t-distribution to calculate the soft membership matrix under the original semantic view. The formula is:
[0053] In the formula, Define the hyperparameters for the degrees of freedom. ; Among them, the sentence-level structure view Cluster-level structure view The formula is:
[0054] In the formula, Strengthen local semantic adjacency consistency. Injecting global cluster-level structural signals, which complement the original semantic view, provides structural priors for subsequent multi-view alignment learning anchored by the original semantic view.
[0055] Step 4: Perform comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structure view, and the cluster-level structure view to obtain the comparative loss function; Furthermore, the comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structural view, and the cluster-level structural view yields a comparative loss function, including: Combine explicit enhanced views, sentence-level structured views, and cluster-level structured views into a multi-view set; Positive items and a first negative item are obtained based on the original semantic view and the multi-view set. A second negative item is obtained based on the negative sample view and the multi-view set. The first negative item and the second negative item are added together to obtain the total negative item. The contrast loss function is obtained based on the positive terms and the total negative terms.
[0056] Further, the step of obtaining positive items and a first negative item based on the original semantic view and the multi-view set, obtaining a second negative item based on the negative sample view and the multi-view set, and adding the first negative item and the second negative item to obtain the total negative item includes: Take the i-th sample in the original semantic view as the anchor sample, and compare it with the multi-view set to obtain the positive sample pair. Then, the similarity scores of all positive samples are aggregated to obtain the positive terms. The formula is: Multiview Collection
[0057] In the formula, Representation of an explicitly enhanced view. and These represent the sample-level representations of the sentence-level structure view and the cluster-level structure view, respectively.
[0058] In the formula, Temperature coefficient; The samples in the original semantic view, excluding the anchor samples, are also compared with the multi-view set to obtain the first negative sample pair. Then, the similarity of all first negative samples is aggregated to obtain the negative terms to be weighted, and a clustering relation matrix is constructed based on the soft membership matrix of the original semantic view. S The first negative term is obtained by weighting the negative terms to be weighted using the same cluster relation matrix as the weights. The formula is: Cluster relation matrix S ,
[0059] In the formula, h≠i, h、t {1, ..., B};
[0060] In the formula, For the hyperparameter of the reweighted exponent; For the negative sample view By comparing the samples with a multi-view set, the second negative sample pair is obtained, and the similarity of all second negative sample pairs is aggregated to obtain the second negative term. The formula is:
[0061] In the formula, This represents the B-th negative sample;
[0062] The sum of the first and second negative terms equals the total negative term. The formula is:
[0063] Furthermore, the comparative loss function is obtained based on the positive terms and the total negative terms. The formula is:
[0064] Step 5: Obtain the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center; Further, the step of obtaining the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center includes: The soft membership matrix of the original semantic view is obtained based on the original text embedding and the first initial cluster center. This matrix is then squared-sharpened and normalized to obtain the target distribution. The soft membership matrix of the original semantic view and the target distribution are aligned according to the cross-entropy constraint to obtain the first optimized loss function. The formula is: Target distribution P,
[0065]
[0066]
[0067] A cost matrix is constructed based on the soft membership matrix of the original semantic view, and a hard pseudo-label matrix is obtained from the cost matrix. The soft membership matrix of the explicit enhanced view is obtained based on the enhanced text embedding and the second initial cluster center. The second optimization loss function is obtained by aligning the soft membership matrix and hard pseudo-label matrix of the explicit augmented view based on the cross-entropy constraint. The formula is:
[0068] The rules for generating the hard pseudo-label matrix are as follows:
[0069] In the formula, Represents the original semantic view. i Optimal transmission constraint pseudo-labels for each sample. Represents the transmission matrix;
[0070] The optimization loss function is obtained by adding the first optimization loss function and the second optimization loss function, as shown in the formula: .
[0071] Step 6: Obtain the overall loss function based on the contrast loss function and the optimization loss function, and train the pre-trained language model encoder based on the overall loss function to obtain the trained encoder; Furthermore, obtaining the overall loss function based on the contrastive loss function and the optimization loss function includes weighting the contrastive loss function and the optimization loss function according to their different training contributions, and then summing the weighted contrastive loss function and the optimization loss function to obtain the overall loss function. The formula is:
[0072] In the formula, and These represent the weights of the contrastive loss function and the optimization loss function in the overall training loss function, respectively.
[0073] Step 7: Input real-time short text samples into the trained encoder, apply the k-means clustering algorithm to the original text embeddings to obtain clustering results, and complete the clustering of short texts.
[0074] A second embodiment of the present invention provides a short text clustering device based on multi-view alignment and optimal transmission, the device comprising: Memory, used to store computer programs; A processor is used to implement the steps of the short text clustering method based on multi-view alignment and optimal transmission described above when executing a computer program.
[0075] For example, the processor in this embodiment of the invention can be used to implement the following method steps: Step 1: Obtain short text samples, perform data augmentation on the short text samples to obtain augmented short text samples, and input the short text samples and augmented short text samples into a pre-trained language model encoder to obtain the original text embedding and the augmented text embedding, respectively. Step 2: Generate the original semantic view based on the original text embedding, generate the explicit augmented view based on the augmented text embedding, and obtain the negative sample view from the intermediate layer of the pre-trained language model encoder based on the original text embedding; Step 3: Use the k-means clustering algorithm to obtain the first initial cluster center and the second initial cluster center for the original text embedding and the enhanced text embedding, respectively. Obtain a sentence-level structure view based on the original text embedding and a cluster-level structure view based on the first initial cluster center. Step 4: Perform comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structure view, and the cluster-level structure view to obtain the comparative loss function; Step 5: Obtain the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center; Step 6: Obtain the overall loss function based on the contrast loss function and the optimization loss function, and train the pre-trained language model encoder based on the overall loss function to obtain the trained encoder; Step 7: Input real-time short text samples into the trained encoder, apply the k-means clustering algorithm to the original text embeddings to obtain clustering results, and complete the clustering of short texts.
[0076] The third embodiment of the present invention provides a short text clustering method based on multi-view alignment and optimal transmission. The invention is verified through experiments and described in conjunction with Table 1, including: Table 1
[0077] This invention conducted systematic performance evaluation experiments. The experiments were carried out on six real-world short text clustering benchmark datasets, including AgNews, SearchSnippets, GoogleNews-TS, GoogleNews-T, GoogleNews-S, and Twitter. These datasets cover both relatively balanced and highly imbalanced class distributions, and simultaneously include semantically relatively complete news short texts and semantically extremely sparse ultra-short texts, thus comprehensively testing the robustness of this invention under typical challenging scenarios such as semantic sparsity, ambiguous boundaries, and class imbalance.
[0078] This invention compares the proposed method with several representative existing schemes, including traditional word frequency feature methods (BOW, TF-IDF), deep representation learning methods (Self-Train, STCC, Sentence-BERT (SBERT), BGE-M3), pseudo-label or semi-supervised optimization methods (RSTC, Multi-MCCR), and contrastive clustering framework methods (SCCL, ProPos, CLSESSP, FNSCC). To ensure fairness in the comparison, each baseline method is configured according to the recommended or commonly used settings given in its published literature. Clustering performance is evaluated using clustering accuracy (ACC) and normalized mutual information (NMI). ACC measures the consistency between predicted clusters and true categories through optimal one-to-one mapping, while NMI measures the degree of information consistency between predicted and true distributions.
[0079] This invention implements the method using PyTorch, employing Sentence-BERT as the encoder and a two-layer GCN as the structure-view modeling module. Context enhancement utilizes a BERT-based masked language model for word substitution enhancement, with a default substitution rate of 20%. The maximum sequence length is set to 32, and the embedding dimension is 768. The training batch size is set to 128, the total number of iterations is set to 4000, and the optimizer is Adam. The learning rate is set as follows: 1e-5 for the encoder part and 1e-3 for the projection head and learnable cluster centers. The contrastive learning temperature coefficient is fixed, and the loss weights are set to [value missing]. The Student's t-distribution parameters are set to [value missing], and the structural negative sample reweighting exponent is set to [value missing]. Optimization of the transfer is achieved through 10 iterations to obtain stable pseudo-label alignment. To reduce the impact of randomness, all experiments are run five times under the same settings, and the average results are reported.
[0080] Experimental results show that this invention achieves optimal clustering performance on all six datasets. Particularly on datasets with highly imbalanced class distributions (such as Google News-T, Google News-S, and Twitter), this invention demonstrates a significant advantage over strong baseline methods, indicating that the pseudo-label alignment based on optimal transmission can alleviate cluster frequency bias and suppress noise propagation under global constraints. Furthermore, in the scenario of extremely sparse semantically short texts, this invention enhances structural consistency through joint modeling of sentence-level and cluster-level structural views, and reduces cross-view inconsistency through a multi-view alignment mechanism anchored to the original semantic view, thereby significantly improving clustering stability and separability.
[0081] In summary, this invention, through multi-view structure modeling and alignment learning using the original semantic view as an anchor point, explicitly constrains cross-view representations while maintaining representational diversity. Figure 1 By combining optimal transmission with distribution-level pseudo-label alignment, a more stable and accurate short text clustering effect can be achieved in semantically sparse and class-imbalanced scenarios.
[0082] The terms used to describe positional relationships in the accompanying drawings are for illustrative purposes only and should not be construed as limiting this patent. Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A short text clustering method based on multi-view alignment and optimal transmission, characterized in that, Includes the following steps: Step 1: Obtain short text samples, perform data augmentation on the short text samples to obtain augmented short text samples, and input the short text samples and augmented short text samples into a pre-trained language model encoder to obtain the original text embedding and the augmented text embedding, respectively. Step 2: Generate the original semantic view based on the original text embedding, generate the explicit augmented view based on the augmented text embedding, and obtain the negative sample view from the intermediate layer of the pre-trained language model encoder based on the original text embedding; Step 3: Use the k-means clustering algorithm to obtain the first initial cluster center and the second initial cluster center for the original text embedding and the enhanced text embedding, respectively. Obtain a sentence-level structure view based on the original text embedding and a cluster-level structure view based on the first initial cluster center. Step 4: Perform comparative learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structure view, and the cluster-level structure view to obtain the comparative loss function; Step 5: Obtain the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center; Step 6: Obtain the overall loss function based on the contrast loss function and the optimization loss function, and train the pre-trained language model encoder based on the overall loss function to obtain the trained encoder; Step 7: Input real-time short text samples into the trained encoder, apply the k-means clustering algorithm to the original text embeddings to obtain clustering results, and complete the clustering of short texts.
2. The short text clustering method based on multi-view alignment and optimal transmission according to claim 1, characterized in that, The process of augmenting short text samples to obtain augmented short text samples includes: for each input short text sample, using a BERT-based context enhancer to perform 20% random word replacements to generate corresponding augmented short text samples.
3. The short text clustering method based on multi-view alignment and optimal transmission according to claim 1, characterized in that, The process of obtaining a sentence-level structural view based on the original text embedding and a cluster-level structural view based on the first initial cluster center includes: Each original text embedding and each first initial cluster center are treated as a node. A sentence-level structure graph is obtained based on the nodes of the original text embeddings, and a cluster-level structure graph is obtained based on the nodes of the first initial cluster centers. The sentence-level structure graph and the cluster-level structure graph are respectively input into a graph convolutional network to obtain the encoded sentence-level structure graph and the encoded cluster-level structure graph, respectively. Then, the encoded sentence-level structure graph is mapped to obtain a sentence-level structure view through a sentence-level propagation matrix, and the encoded cluster-level structure graph is mapped to obtain a cluster-level structure view through a cluster-level propagation matrix.
4. The short text clustering method based on multi-view alignment and optimal transmission according to claim 3, characterized in that, The sentence-level structure diagram It includes the original node set, the original node feature matrix, and the sentence-level adjacency matrix, as shown in the formula: In the formula, the original set of nodes is , This represents the B-th original node. B Indicates the batch size; the original node feature matrix is... , Indicates the embedding of the original text. Represents sentence embedding dimension, sentence-level adjacency matrix It is calculated using cosine similarity, and the formula is: In the formula, T is the transpose. Represented as the first i A sample of original text embeddings, Represented as the first j A sample of original text embeddings; The cluster-level structure diagram It includes the cluster-level node set, the cluster-level node feature matrix, and the cluster-level adjacency matrix, as shown in the formula: In the formula, the cluster-level node set , Indicates the first K Cluster-level nodes, K Represents the number of cluster centers, cluster-level node characteristic matrix , Represents the cluster center dimension, cluster-level adjacency matrix Similarly, it is calculated using the cosine similarity between cluster centers, and the formula is: In the formula, Indicates the first k Cluster center vectors, Indicates the first l Cluster center vectors.
5. The short text clustering method based on multi-view alignment and optimal transmission according to claim 4, characterized in that, The sentence-level propagation matrix It is obtained from the normalized sentence-level adjacency matrix, and the formula is: In the formula, This indicates that the similarity matrix has been normalized. The cluster-level propagation matrix It uses Student's t-distribution to calculate the soft membership matrix under the original semantic view. The formula is: In the formula, Define the hyperparameters for the degrees of freedom. .
6. The short text clustering method based on multi-view alignment and optimal transmission according to claim 1, characterized in that, The comparison learning based on the negative sample view, the original semantic view, the explicit augmented view, the sentence-level structure view, and the cluster-level structure view yields a comparison loss function, including: Combine explicit enhanced views, sentence-level structured views, and cluster-level structured views into a multi-view set; Positive items and a first negative item are obtained based on the original semantic view and the multi-view set. A second negative item is obtained based on the negative sample view and the multi-view set. The first negative item and the second negative item are added together to obtain the total negative item. The contrast loss function is obtained based on the positive terms and the total negative terms.
7. The short text clustering method based on multi-view alignment and optimal transmission according to claim 6, characterized in that, The process of obtaining positive items and a first negative item based on the original semantic view and the multi-view set, obtaining a second negative item based on the negative sample view and the multi-view set, and adding the first negative item and the second negative item to obtain the total negative item includes: Take the i-th sample in the original semantic view as the anchor sample, and compare it with the multi-view set to obtain the positive sample pair. Then, the similarity scores of all positive samples are aggregated to obtain the positive terms. The formula is: Multiview Collection In the formula, Representation of an explicitly enhanced view. and These represent the sample-level representations of the sentence-level structure view and the cluster-level structure view, respectively. In the formula, Temperature coefficient; The samples in the original semantic view, excluding the anchor samples, are also compared with the multi-view set to obtain the first negative sample pair. Then, the similarity of all first negative samples is aggregated to obtain the negative terms to be weighted, and a clustering relation matrix is constructed based on the soft membership matrix of the original semantic view. S The first negative term is obtained by weighting the negative terms to be weighted using the same cluster relation matrix as the weights. The formula is: Cluster relation matrix S , In the formula, h≠i, h、t {1, ..., B}; In the formula, For the hyperparameter of the reweighted exponent; For the negative sample view By comparing the samples with a multi-view set, the second negative sample pair is obtained, and the similarity of all second negative sample pairs is aggregated to obtain the second negative term. The formula is: In the formula, This represents the B-th negative sample; The sum of the first and second negative terms equals the total negative term. The formula is: 。 8. The short text clustering method based on multi-view alignment and optimal transmission according to claim 6, characterized in that, The comparison loss function is obtained based on the positive terms and the total negative terms. The formula is: 。 9. The short text clustering method based on multi-view alignment and optimal transmission according to claim 1, characterized in that, The step of obtaining the optimized loss function based on the original text embedding, the first initial cluster center, the enhanced text embedding, and the second initial cluster center includes: The soft membership matrix of the original semantic view is obtained based on the original text embedding and the first initial cluster center. This matrix is then squared-sharpened and normalized to obtain the target distribution. The soft membership matrix of the original semantic view and the target distribution are aligned according to the cross-entropy constraint to obtain the first optimized loss function. The formula is: Target distribution P, A cost matrix is constructed based on the soft membership matrix of the original semantic view, and a hard pseudo-label matrix is obtained from the cost matrix. The soft membership matrix of the explicit enhanced view is obtained based on the enhanced text embedding and the second initial cluster center. The second optimization loss function is obtained by aligning the soft membership matrix and hard pseudo-label matrix of the explicit augmented view based on the cross-entropy constraint. The formula is: The rules for generating the hard pseudo-label matrix are as follows: In the formula, Represents the original semantic view. i Optimal transmission constraint pseudo-labels for each sample. Represents the transmission matrix; The optimization loss function is obtained by adding the first optimization loss function and the second optimization loss function, as shown in the formula: .
10. The short text clustering method based on multi-view alignment and optimal transmission according to claim 6 or 9, characterized in that, The process of obtaining the overall loss function based on the contrastive loss function and the optimized loss function includes weighting the contrastive loss function and the optimized loss function according to their different training contributions, and then summing the weighted contrastive loss function and the optimized loss function to obtain the overall loss function. The formula is: In the formula, and These represent the weights of the contrastive loss function and the optimization loss function in the overall training loss function, respectively.