Text clustering method and system based on contrastive learning and graph embedding optimization

Abstract

The application belongs to the field of natural language processing, and provides a text clustering method and system based on contrast learning and graph embedding optimization, and the technical scheme is as follows: contrast learning is adopted in combination with a graph embedding method, so that the global structure of data is considered, and the local graph structure of features is combined. The graph embedding fully utilizes the neighbor prior information of the original high-dimensional sample space, and applies constraints on the embedding of the low-dimensional representation space, so that the connection strength of the original sample is reflected in the hidden layer features. The correlation between the sample and the nearest neighbor is utilized to make up for the clustering center deviation caused by relying on the global structure only, so that the clustering precision is improved, and accurate clustering results are obtained from high-data-volume and high-sparse text data sets.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of natural language processing, and particularly relates to a text clustering method and system based on contrastive learning and graph embedding optimization. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] The advent of the big data era has led to a rapid increase in data volume. Text clustering clusters tweets, news or reviews according to their contextual similarity. It can be used in various applications such as content recommendation, topic modeling or sentiment analysis. For short text data such as tweets, short news, search terms, etc., traditional text clustering cannot meet the demand due to its high dimensionality, high sparsity and high noise. In order to solve the problem of growing data volume and high-dimensional data characteristics, a large number of deep learning-based text clustering algorithms have been widely studied.

[0004] The inventors have found that although the low-dimensional vectors extracted by deep neural networks can well represent short texts, the low-dimensional vectors of texts still have overlaps in the representation space, which will significantly affect the clustering effect. Contrastive learning has achieved great success in the field of deep representation learning, which further alleviates the problem of data point overlap by pulling the distance between positive pairs and pushing away negative pairs, but these methods only consider the global structure of data without utilizing the local information between samples and neighbors. SUMMARY

[0005] In order to solve at least one of the technical problems in the above background art, the present application provides a text clustering method and system based on contrastive learning and graph embedding optimization, which adopts contrastive learning and combines with graph embedding method, considering both the global structure of data and the local graph structure of features. Graph embedding makes full use of the neighbor prior information of the original high-dimensional sample space, and imposes constraints on the embedding of the low-dimensional representation space to ensure that the connection strength of the original sample is reflected in the hidden layer features. The correlation between samples and nearest neighbors is used to compensate for the clustering center deviation caused by relying only on global structure, achieving the purpose of improving clustering accuracy, so as to obtain accurate clustering results from high data volume and high sparsity text data sets.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides a text clustering method based on contrastive learning and graph embedding optimization, comprising:

[0008] obtaining text data;

[0009] obtaining a clustering result based on the text data and the trained text clustering model; wherein a construction process of the text clustering model comprises:

[0010] performing feature extraction based on the original text data and the trained feature extractor to obtain original text features; and constructing an affinity matrix by calculating cosine similarity between the original text features;

[0011] jointly training the contrast head and the clustering head, and making the graph embedding constrain the feature space to construct a local structure of data, specifically comprising:

[0012] for the clustering head, calculating membership by combining the clustering center, the original text features and the student t-distribution, constructing an auxiliary distribution by the membership, making the student t-distribution fit the auxiliary distribution, and obtaining clustering probability of the sample based on the auxiliary distribution;

[0013] for the contrast head, obtaining an enhanced sample pair based on data enhancement of the text data; mapping features obtained by the enhanced sample pair of the same text data as positive pairs and others as negative pairs; and adopting a contrast loss to constrain,

[0014] adjusting topological relations between points of the positive pairs and the negative pairs mapped to a subspace to make the positive pairs close and the negative pairs far away;

[0015] combining the affinity matrix, imposing a constraint on embedding of the low-dimensional representation space by affinity between samples and neighbors through graph embedding, so that connection strength of the original sample is reflected in the hidden layer features.

[0016] A second aspect of the present application provides a text clustering system based on contrast learning and graph embedding optimization, comprising:

[0017] a data acquisition module configured to acquire text data;

[0018] a text clustering module configured to obtain a clustering result based on the text data and a trained text clustering model; wherein a construction process of the text clustering model comprises:

[0019] performing feature extraction based on the original text data and the trained feature extractor to obtain original text features; and constructing an affinity matrix by calculating cosine similarity between the original text features;

[0020] jointly training the contrast head and the clustering head, and making the graph embedding constrain the feature space to construct a local structure of data, specifically comprising:

[0021] for the clustering head, calculating membership by combining the clustering center, the original text features and the student t-distribution, constructing an auxiliary distribution by the membership, making the student t-distribution fit the auxiliary distribution, and obtaining clustering probability of the sample based on the auxiliary distribution;

[0022] For the contrast head, the data enhancement based on the text data obtains the enhanced sample pair; the features mapped after the enhanced sample pair of the same text data are obtained as positive pairs, and others are negative pairs; the contrast loss is adopted to constrain,

[0023] The topological relationship between the points of the positive pairs and the negative pairs mapped to the subspace is adjusted, so that the positive pairs are close and the negative pairs are far away;

[0024] In combination with the affinity matrix, the clustering allocation of the cluster edge is optimized by graph embedding and by using the affinity between the sample and the neighbor to impose constraints on the low-dimensional representation space.

[0025] The third aspect of the present application provides a computer readable storage medium.

[0026] A computer readable storage medium has a computer program stored thereon, and the program is executed by a processor to implement the steps in the text clustering method based on contrast learning and graph embedding optimization according to the first aspect.

[0027] The fourth aspect of the present application provides a computer device.

[0028] A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor executes the program to implement the steps in the text clustering method based on contrast learning and graph embedding optimization according to the first aspect.

[0029] Compared with the prior art, the present application has the following advantages:

[0030] Therefore, the inventors use contrast learning and combine it with the graph embedding method, which considers both the global structure of the data and the local graph structure of the features. Graph embedding fully utilizes the prior information of the nearest neighbors in the original high-dimensional sample space, imposes constraints on the embedding of the low-dimensional representation space, and ensures that the connection strength of the original sample is reflected in the hidden layer features. The correlation between the sample and the nearest neighbor is used to compensate for the clustering center deviation caused by relying only on the global structure, so as to improve the clustering accuracy and obtain accurate clustering results from the high-data-volume and high-sparse text data set.

[0031] 1、The application overcomes the limitations of traditional text clustering in handling high sparse, high latitude, high data volume short text data through a contrast learning framework; adopts contrast learning combined with graph embedding method, which considers both the global structure of data and the local graph structure of features; graph embedding fully utilizes the affinity between samples and neighbors to impose constraints on the embedding of low-dimensional representation space to ensure that the connection strength of the original sample is reflected in the hidden layer features. The correlation between samples and nearest neighbors is used to compensate for the clustering center deviation caused by relying only on the global structure, so as to improve the clustering accuracy and obtain accurate clustering results from high data volume and high sparse text data set.

[0032] 2、In order to ensure the effectiveness of contrast learning, the application adopts two different text enhancement methods. Data enhancement plays an important role in contrast learning, which can help increase the diversity of training data, thereby improving the performance of the contrast learning model. By introducing data diversity, the contrast learning model can better learn robust feature representation and improve the generalization ability of the model.

[0033] 3、The application further increases the diversity of data through data enhancement, maximizes the effect of contrast learning in feature extraction, and ensures the correctness of feature semantics; through KL divergence, the fuzzy membership degree tends to be sharp, and the result with higher confidence is obtained; the application verifies its effectiveness on multiple data sets, and good results are achieved in accuracy and convergence speed.

[0034] The advantages of the additional aspects of the application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0035] The drawings accompanying the specification of this application form a part thereof, serve to provide further understanding of the application, and together with the description of the exemplary embodiments of the application and their description serve to explain the application, and do not constitute an improper limitation of the application.

[0036] Figure 1 is the overall flow chart of text clustering based on contrast learning and graph embedding optimization provided by the embodiment of the application;

[0037] Figure 2 is the affinity matrix extraction method framework based on contrast learning provided by the embodiment of the application;

[0038] Figure 3 is the text clustering framework based on contrast learning and graph embedding optimization provided by the embodiment of the application. DETAILED DESCRIPTION

[0039] The application will be further described below in conjunction with the drawings and embodiments.

[0040] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0042] Example 1

[0043] like Figure 1 As shown, this embodiment provides a text clustering method based on contrastive learning and graph embedding optimization, including the following steps:

[0044] Step 1: Obtain text data;

[0045] Step 2: Extract features from the original text data and the trained feature extractor to obtain the original text features; calculate the cosine similarity between the original text features to construct the affinity matrix;

[0046] To effectively mine local information in data and construct a local graph structure, this invention utilizes a contrastive learning framework to enrich data diversity through text enhancement, thereby maximizing the benefits of contrastive learning in feature extraction and ultimately obtaining a more accurate affinity matrix.

[0047] Specifically, it includes:

[0048] Enhanced sample pairs are obtained by using data augmentation, and feature extraction is performed using a contrastive learning framework. The superiority of contrastive learning in feature extraction is used to transform the high-dimensional and sparse original text into a semantically low-dimensional vector.

[0049] Using an MLP as the comparison head, the enhanced sample pairs are fed into the comparison head, and the comparison loss function is used as the optimization objective on the obtained results.

[0050] By training a contrastive model, the low-dimensional vector output by the model is obtained. Similarity calculations are performed, and an affinity matrix is ​​built for subsequent clustering.

[0051] Step 3: Jointly train the contrast head and clustering head and use graph embedding to constrain the feature space to construct the local structure of the data to improve clustering accuracy, including:

[0052] Randomly initialize cluster centers.

[0053] The original text is text enhanced to obtain an enhanced sample pair, and the original text and the enhanced sample are sent into a feature extractor to obtain the features of the original sample and the enhanced sample.

[0054] For enhanced sample features, they are sent into a contrast head, and a contrast loss is used to adjust the topological relationship between the points mapped to the subspace, which can make the positive sample pair closer and the negative sample pair farther away.

[0055] For original sample features, they are sent into a clustering head. Unlike the contrast head, the clustering head calculates the membership by student t distribution. By sharpening the membership, an auxiliary distribution is constructed, and the student t distribution is fitted to the auxiliary distribution to obtain a clustering result with high confidence.

[0056] For local information of data, according to the previously extracted neighbor information, a local graph structure of data is constructed. On the basis of global modeling, graph embedding constraints are used to supplement the local information of the subspace representation;

[0057] As shown in Figure 2 , the step 2 specifically includes:

[0058] Step 201: text enhancement stage.

[0059] In this example, two text enhancement methods can be used. For a text , two different text enhancement methods are used to obtain an enhanced sample pair .

[0060] Specifically, the two enhancement methods are:

[0061] First: replace with synonyms at a certain proportion. The synonyms are found by using two language models such as BERT and RoBERTa to replace the most suitable synonyms for enhancement.

[0062] Second: randomly exchange the order of words in the sentence at a certain proportion.

[0063] Step 202: contrast learning stage.

[0064] In this example, BERT is used as a feature extractor, and the enhanced sample pair is sent into the feature extractor to obtain . A multi-layer perceptron is used as a contrast head , and is obtained by .

[0065] Contrastive learning sets an optimization goal to train a better network parameter, and the implementation of this goal is to pull the similarity between positive samples closer and expand the difference between negative samples.

[0066] To better achieve this goal, the following loss function is used, as shown in equation (1):

[0067]

[0068] In the formula, is the contrastive loss calculated based on enhanced text features . is the enhanced text output after the contrastive head, is the enhanced text output after the contrastive head, is the enhanced text output after the contrastive head, is the enhanced text output after the contrastive head is the total number of texts.

[0069] is the cosine similarity, used to measure the similarity and proximity between sample pairs. is the temperature parameter, used to control the degree of differentiation of the model for negative pairs. The greater the temperature parameter setting, the more equally the contrastive loss treats all negative pairs. The greater the temperature parameter setting, the more equally the contrastive loss treats all negative pairs. If the temperature parameter is set too small, the model will focus more on particularly difficult negative pairs, making it difficult for the model to converge or generalize.

[0070] The loss of the contrastive head is calculated on each enhanced sample, and all contrastive losses are averaged, i.e.

[0071]

[0072] represents the contrastive loss calculated based on enhanced text features .

[0073] Step 203: Affinity calculation stage.

[0074] By saving the network parameters trained in the contrastive learning stage, the original text is sent to the feature extractor to obtain low-dimensional features .

[0075] An affinity matrix S is constructed by calculating the cosine similarity between features, i.e.

[0076]

[0077] For the original text The output after the feature extractor, For the original sample The output after the feature extractor.

[0078] In order to ensure the effectiveness of the graph embedding, it is important to select the correct affinity matrix extraction method. The accuracy of the affinity matrix greatly affects the improvement of the clustering effect of the graph embedding. If the affinity matrix cannot correctly reflect the similarity between data, it may even have a negative optimization phenomenon on the clustering result.

[0079] The technical advantage of the above scheme is that the method uses text enhancement to enrich the reliability of the sample and uses the BERT language model to obtain low-dimensional and reliable text sentence vectors in combination with the contrast learning framework. Contrast learning makes full use of data diversity, so that the network fully learns the text representation.

[0080] In order to ensure the integrity of the semantics, the original text is used to calculate the affinity matrix after the feature extractor SBERT The features obtained Instead of the subsequent contrast head The features obtained by further dimension reduction .

[0081] By this method, a reliable affinity matrix is constructed. For each data, the similarity information with other data can be obtained through the affinity matrix, which can be used for data analysis and subsequent clustering.

[0082] As Figure 3 shown, the step 3 specifically includes:

[0083] Step 301: initialization of cluster center stage.

[0084] The basic principle of initializing the cluster center is that the mutual distance between the cluster centers should be as far as possible. The initialization process is divided into the following three steps:

[0085] Step 3011: randomly select a sample as the first cluster center;

[0086] Step 3012: calculate the shortest distance between each sample and the existing cluster center. The greater the distance, the greater the probability of being selected as a cluster center; use the roulette method to select the next cluster center;

[0087] Step 3013: repeat step 3012 until K cluster centers are selected .

[0088] Step 302: Text enhancement stage.

[0089] For each text Two enhancements were performed to obtain the enhanced sample. The two text enhancement methods employ the following strategies:

[0090] First enhancement

[0091] 1. Replacement: Replace text randomly with a 20% probability, using a BERT pre-trained model to generate replacement content.

[0092] 2. Swap: Randomly swap the order of words with a 20% probability.

[0093] 3. Execute steps 1 and 2 in sequence to obtain the desired result. .

[0094] Second enhancement:

[0095] 1. Replacement: Replace text randomly with a 20% probability, using a RoBERTa pre-trained model to generate replacement content.

[0096] 2. Swap: Randomly swap the order of words with a 20% probability.

[0097] 3. Execute steps 1 and 2 in sequence to obtain the desired result. .

[0098] Step 303: Joint training phase of comparison head and clustering head.

[0099] First, the original text Enhanced sample pair Together they are fed into SBERT as the feature extractor The low-dimensional vector of the original text is obtained from this.

[0100] For the clustering head, the original text is processed by the feature extractor. Output

[0101] Membership degrees are calculated using the Student t-distribution. The formula is as follows:

[0102]

[0103] in The cluster centers are randomly initialized. The degrees of freedom represent the student's t-distribution. The original text features are represented by K, which represents the number of cluster centers selected. For iteration variables, Indicates the first Cluster centers of each cluster.

[0104] is the probability that the sample belongs to the cluster, which is a soft assignment, by constructing a side distribution to obtain a sharper cluster assignment, i.e.,

[0105]

[0106] side distribution First, square and normalize the soft distribution .

[0107] The side distribution can obtain a sharper clustering result. The student t distribution is used to fit the side distribution by constraining the KL divergence, and the optimization objective is:

[0108]

[0109] For the contrastive head, the output of the feature extractor is passed through a two-layer MLP to further map to 128 dimensions to obtain . The goal of contrastive learning is to learn text representations, considering the features obtained from two augmentations of the same sample as positive pairs and others as negative pairs.

[0110] In this embodiment, the cosine similarity is used to measure the closeness between sample pairs. The contrastive loss constraint is adopted, and the rule is that positive sample pairs are as close as possible, and negative sample pairs are as far apart as possible. The optimization objective is as follows:

[0111]

[0112]

[0113] Using graph embedding optimization uses local information of samples to constrain the closeness of samples and their neighbors in the feature space. The sample and its neighbors are sent to the feature extractor to obtain representations and .

[0114] A two-layer nonlinear MLP is used to map the representations to a 128-dimensional subspace, i.e. .

[0115]

[0116] where m denotes the number of neighbors of each sample. is the affinity matrix, which represents the closeness between the i-th sample and the j-th sample.​​i a similarity between the first feature and the second feature. m

[0117] This is used to constrain the proximity of the sample and its neighbors in the representation space, and to construct a local graph relationship for the sample. That is, if the neighbors are close to the original data, the features of the hidden layer are also required to be close to each other.

[0118] Therefore, the overall optimization target is:

[0119]

[0120] The advantage of the above scheme is that the feature extractor , the contrast head , the clustering center , learns the data representation globally, and uses the superiority of data augmentation to maximize contrast learning in feature extraction, and uses graph embedding to constrain the samples and neighbors in the representation space locally. If the sample and the neighbor are similar, they should be close in the representation space. By constructing the local structure of the data, the problem of unclear cluster boundary division in traditional text clustering is solved, and the accuracy of the clustering result is greatly improved.

[0121] Embodiment Two

[0122] The embodiment provides a text clustering system based on contrast learning and graph embedding optimization, comprising:

[0123] A data acquisition module is configured to acquire text data;

[0124] A text clustering module is configured to obtain a clustering result based on the text data and a trained text clustering model; wherein the construction process of the text clustering model comprises:

[0125] Feature extraction is performed based on the original text data and the trained feature extractor to obtain original text features; and a cosine similarity between the original text features is calculated to construct an affinity matrix;

[0126] The contrast head and the clustering head are jointly trained, and the graph embedding is used to constrain the feature space, and the local space of the data is constructed, specifically comprising:

[0127] For the clustering head, the membership is calculated in combination with the clustering center, the original text features and the student t-distribution, the auxiliary distribution is constructed through the membership, the student t-distribution is fitted to the auxiliary distribution, and the clustering probability of the sample is obtained based on the auxiliary distribution;

[0128] For the contrast head, enhanced sample pairs are obtained through data augmentation based on the text data; the features obtained by the enhanced sample pairs of the same text data are mapped as positive pairs, and others are negative pairs; a contrast loss constraint is used,​

[0129] Adjust the topological relationship between the points that positive pairs and negative pairs map to the subspace, so that the positive pairs are close and the negative pairs are far away;

[0130] In combination with the affinity matrix, by graph embedding, the affinity between samples and neighbors is used to impose constraints on the low-dimensional representation space to optimize the clustering assignment of cluster edges.

[0131] Embodiment three

[0132] The embodiment provides a computer readable storage medium, which has stored thereon a computer program, the program being executed by a processor to implement steps in the text clustering method based on contrastive learning and graph embedding optimization as described above.

[0133] Embodiment four

[0134] The embodiment provides a computer device, which comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements steps in the text clustering method based on contrastive learning and graph embedding optimization as described above when executing the program.

[0135] Those skilled in the art will understand that embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer usable program code.

[0136] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The means for implementing each flow or multiple flows and / or blocks Figure 1 The means for implementing each flow or multiple flows and / or blocks

[0137] These computer program instructions can also be stored in a computer readable memory capable of directing a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0138] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flow Figure 1 one or more processes and / or blocks Figure 1 the function specified in the one or more blocks.

[0139] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by computer programs instructing related hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, they can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like.

[0140] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those of ordinary skill in the art can make various modifications and changes to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A text clustering method based on contrastive learning and graph embedding optimization, characterized in that, include: Get text data; Clustering results are obtained based on text data and a trained text clustering model; wherein, the construction process of the text clustering model includes: Original text features are obtained by extracting features from the original text data and the trained feature extractor; cosine similarity between the original text features is calculated to construct an affinity matrix. Jointly train the comparison head and the clustering head, and embed the graph into the constrained feature space to construct the local space of the data, specifically including: For the cluster head, the membership degree is calculated by combining the cluster center, the original text features and the student t-distribution. An auxiliary distribution is constructed by the membership degree, and the student t-distribution is fitted to the auxiliary distribution. The clustering probability of the sample is obtained based on the auxiliary distribution. For the contrast head, data augmentation is performed on the text data to obtain augmented sample pairs; the feature maps obtained from augmented sample pairs of the same text data are used as positive pairs, and others are used as negative pairs; a contrastive loss constraint is applied. Adjust the topological relationship between points mapped to the subspace by positive and negative pairs, so that positive pairs are close together and negative pairs are far apart; By combining the affinity matrix and using graph embedding, constraints are imposed on the embedding of the low-dimensional representation space using the affinity between samples and their neighbors, so that the connection strength of the original samples is reflected in the hidden layer features.

2. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 1, characterized in that, The training process of the feature extractor includes: Data augmentation based on text data to obtain augmented sample pairs; Based on the enhanced sample pairs, BERT is used as the feature extractor. The enhanced sample pairs are fed into the feature extractor to obtain the corresponding enhanced sample features. A multilayer perceptron is used as the comparison head. The loss function is set according to the optimization objective, which is to bring the similarity between positive samples closer and increase the difference between negative samples. When the loss function is minimized, the corresponding network parameters are the optimized parameters, and the feature extractor is built based on these network parameters.

3. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 2, characterized in that, When augmenting data, a method of randomly replacing words with synonyms is used; or, Randomly swap the order of words in a sentence.

4. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 1, characterized in that, The initialization process of cluster centers specifically includes: Randomly select a sample as the first cluster center; Calculate the shortest distance between each sample and the existing cluster centers. The larger the distance, the greater the probability of being selected as a cluster center. Use the roulette wheel method to select the next cluster center. Continue until K cluster centers are selected.

5. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 1, characterized in that, The membership degree distribution is as follows: in The cluster centers are randomly initialized. The degrees of freedom represent the student's t-distribution. The original text features are represented by K, which represents the number of cluster centers selected. For iteration variables, Indicates the first Cluster centers of each cluster.

6. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 1, characterized in that, Cosine similarity is used to adjust the topological relationship between points mapped to the subspace by positive and negative pairs.

7. The text clustering method based on contrastive learning and graph embedding optimization as described in claim 1, characterized in that, The affinity matrix, through graph embedding, imposes constraints on the embedding of the low-dimensional representation space using the affinity between samples and their neighbors, so that the connection strength of the original samples is reflected in the hidden layer features. Specifically, this includes: Obtain the representation of the sample and its neighbors; A two-layer nonlinear MLP is used to map the representations of samples and their neighbors to a subspace to obtain the corresponding subspace mapping; By combining the affinity matrix and the subspace mapping between the sample and its neighbors, constraints on the proximity of the sample and its neighbors in the representation space can be calculated.

8. A text clustering system based on contrastive learning and graph embedding optimization, characterized in that, include: The data acquisition module is used to acquire text data. A text clustering module is used to obtain clustering results based on text data and a trained text clustering model; wherein, the construction process of the text clustering model includes: Original text features are obtained by extracting features from the original text data and the trained feature extractor; cosine similarity between the original text features is calculated to construct an affinity matrix. Jointly train the comparison head and the clustering head, and embed the graph into the constrained feature space to construct the local space of the data, specifically including: For the cluster head, the membership degree is calculated by combining the cluster center, the original text features and the student t-distribution. An auxiliary distribution is constructed by the membership degree, and the student t-distribution is fitted to the auxiliary distribution. The clustering probability of the sample is obtained based on the auxiliary distribution. For the contrast head, data augmentation is performed on the text data to obtain augmented sample pairs; the feature maps obtained from augmented sample pairs of the same text data are used as positive pairs, and others are used as negative pairs; a contrastive loss constraint is applied. Adjust the topological relationship between points mapped to the subspace by positive and negative pairs, so that positive pairs are close together and negative pairs are far apart; By combining the affinity matrix and using graph embedding, constraints are imposed on the embedding of the low-dimensional representation space using the affinity between samples and their neighbors, so that the connection strength of the original samples is reflected in the hidden layer features.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the text clustering method based on contrastive learning and graph embedding optimization as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the text clustering method based on contrastive learning and graph embedding optimization as described in any one of claims 1-7.

Images