Multi-view text clustering method based on convolutional neural network

Abstract

The invention provides a multi-view text clustering method based on a convolutional neural network. The method comprises the following steps: preprocessing original multi-view text data; calculating a TF-IDF value of the preprocessed original multi-view text data, and then combining the TF-IDF value with a Word2Vec word vector to generate a text word embedding weighted vector; generating an LDA text topic vector for the preprocessed original multi-view text data; respectively carrying out spectral clustering on vector groups formed by the text word embedding weighting vector and the LDA text topic vector to obtain clustering results; the same classification data in the clustering result serve as marked data to be input into a convolutional neural network for training, different classification data in the clustering result serve as unmarked samples, the trained convolutional neural network is used for conducting feature extraction on the unmarked samples, spectral clustering is conducted on obtained feature vectors, and a final clustering result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and in particular to a multi-view text clustering method based on convolutional neural network. Background Art

[0002] With the rapid development of the Internet, the amount of text information has grown rapidly, and the massive amount of unstructured text information has increased day by day, showing an explosive growth. The related text clustering algorithms that have been proposed are difficult to meet people's needs in text analysis, business applications, recommendation systems, biomedicine and other fields. How to mine useful information and knowledge from large amounts of text data is a major problem that people urgently need to solve.

[0003] Text clustering is an important natural language processing task that aims to automatically divide a large amount of text data into several groups based on the similarity of the texts. The texts in each group have similarities in content or topic. It has a wide range of applications in information retrieval, data mining, text classification, recommendation systems, etc. However, the text clustering task faces many challenges and difficulties.

[0004] Traditional unsupervised clustering methods do not require labeled data, so they are suitable for large-scale datasets that lack labels. This method is highly adaptable to new data, but the interpretability and accuracy of clustering results may be limited due to the lack of prior knowledge. Since there is no clear category information in unlabeled data, the model cannot use labels to correct or verify clustering results. The quality of clustering depends entirely on the algorithm and feature selection, and is easily affected by noise and improper feature selection, resulting in unstable or inaccurate results. In contrast, semi-supervised text clustering combines a small amount of labeled data with a large amount of unlabeled data. The clustering process is guided by the prior information provided by the labeled data, which can significantly improve the accuracy and stability of the clustering results. Semi-supervised text clustering still has some problems and challenges: Although semi-supervised methods reduce the need for labeled samples, in some text datasets, labeled samples may still be scarce or difficult to obtain. Especially in areas that require expert knowledge or expertise, the acquisition of labeled samples may become a bottleneck; semi-supervised text clustering methods are highly dependent on the quality and representativeness of labeled samples. If the labeled samples are not representative or there are labeling errors, inaccurate clustering results may result. The selection and labeling process of labeled samples may also introduce bias, thus affecting the final clustering effect.

[0005] In the prior art, for example, patent CN106776740A discloses a social network text clustering method based on a convolutional neural network. The social network text clustering method based on a convolutional neural network uses the powerful abstract representation capability of the convolutional neural network for local features to learn the features of microblog texts, and uses the clustering algorithm in machine learning to process the features to achieve clustering of microblog texts. Considering the high cost of manually labeling data set categories, this method uses the local preservation restriction algorithm in natural language processing to process the original features, and maps the text information into a binary numerical vector as an abstract representation of the microblog text. For example, patent CN113704429A discloses an intent recognition method based on semi-supervised learning, which includes the following steps: Step 1: extracting the user's intent text from the conversation intent corpus, performing entity annotation and entity enhancement processing on the intent text, and obtaining an intent entity set; Step 2: performing label clustering on the intent entities in the intent entity set to obtain an intent category; Step 3: obtaining a labeled training set using the intent labels of the intent category, and training a pre-constructed neural network using the labeled training set to obtain an original intent recognition model; Step 4: extracting an unlabeled training set from the conversation intent corpus, and iteratively training the original intent recognition model using the unlabeled training set and the labeled training set to obtain a semi-supervised training model; Step 5: cross-training the semi-supervised training model using a cross-validation method to obtain a standard intent recognition model, and using the standard intent recognition model to output the intent recognition result of the text to be recognized.

[0006] Disadvantages of existing technology:

[0007] 1. In traditional semi-supervised text clustering methods, labeled samples are crucial. However, obtaining sufficient and representative labeled samples is usually time-consuming and expensive, especially in areas that require expert knowledge. The scarcity and high cost of labeled samples limit the widespread application of these methods. At the same time, manually labeled samples still have problems such as sample selection bias and class imbalance, which have a negative impact on the quality of clustering results.

[0008] 2. Most existing methods use traditional feature representations, such as TF-IDF or bag-of-words models, which cannot fully capture the semantic information of the text, resulting in poor clustering results. In addition, high-dimensional and sparse feature representations also increase the complexity and computational overhead of the clustering process.

[0009] 3 Traditional semi-supervised clustering methods may require complex computational steps, such as similarity calculations, large-scale matrix operations, etc. The computational costs of these operations are high, which affects the efficiency and practical application of the algorithm. Summary of the invention

[0010] One of the purposes of the present invention is to provide a multi-view text clustering method based on convolutional neural network to solve the problem that the existing text clustering method cannot reduce the feature dimension and fully capture the semantic information of the text; to solve the problems of manually labeled samples, sample selection bias and category imbalance in semi-supervised text clustering methods, and reduce overhead and human resources.

[0011] An embodiment of the present invention provides a multi-view text clustering method based on a convolutional neural network, comprising:

[0012] S01: preprocessing the original multi-view text data;

[0013] S02: Calculate the TF-IDF value of the preprocessed original multi-view text data and then combine it with the Word2Vec word vector to generate a text word embedding weighted vector;

[0014] S03: Generate LDA text topic vectors for the preprocessed original multi-view text data;

[0015] S04: Perform spectral clustering on the vector groups formed by the text word embedding weighted vector and the LDA text topic vector to obtain clustering results;

[0016] S05: The same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and the different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features of the unlabeled samples and the obtained feature vectors are spectrally clustered to obtain the final clustering results.

[0017] Preferably, the preprocessing of the original multi-view text data includes:

[0018] S11: Use regular expressions to traverse the article content of the original multi-view text data, retain only Chinese characters in the article content, filter out all non-Chinese characters and numeric characters, use the Jieba word segmentation tool to segment the article content, and divide the article content into single words as basic units;

[0019] S12: Use the stop word list of Harbin Institute of Technology, Baidu stop word list or a custom stop word list to delete stop words from the article content;

[0020] S13: Use the Word2Vec method in the CBOW model to generate word vector representations for each word in the article content, and calculate the cosine similarity between word vectors. Where w i is the i-th word, w j is the jth word, v i For word w i The word vector, v j For word wj The word vectors of the word are taken as related words whose similarity exceeds the custom threshold, and the set of related words with the closest semantics is found for each word;

[0021] S14: Select the word with the highest frequency in each associated word set as a representative word, use regular expressions to search and replace all words in the associated word set as representative words in the article content, and obtain the preprocessed text data.

[0022] Preferably, the step of calculating the TF-IDF value of the preprocessed original multi-view text data and then combining it with the Word2Vec word vector to generate a text word embedding weighted vector includes:

[0023] S21: For each text in the preprocessed original multi-view text data, calculate the word frequency of each word in the text Where n d represents the total number of words in the text d in the original multi-view text data, Count(w, d) represents the number of times word w appears in the text d in the original multi-view text data; calculate the inverse text frequency Where N is the number of texts in the original multi-view text data, DF(w) is the number of texts containing word w in the original multi-view text data; for word w of each text in the original multi-view text data, calculate TF-IDF(w, d) = TF(w, d) · IDF(w);

[0024] S22: Use the trained Word2Vec model to determine each word in the text in the original multi-view text data to obtain its word vector representation;

[0025] S23: For each word in the original multi-view text data, perform word embedding weighting v′ w =TF-IDF(w, d)·v w , where v w is the Word2Vec vector of word w, TF-IDF(w, d) is the TF-IDF value of word w in text d;

[0026] S24: Perform weighted averaging on all weighted word vectors in the text of the original multi-view text data to obtain the text word embedding weighted feature vector Where v d is the vector representation of the text d in the original multi-view text data, ∑ wed TF-IDF(w, d) is the sum of the TF-IDF values ​​of all words in the text in the original multi-view text data.

[0027] Preferably, generating LDA text topic vectors from the preprocessed original multi-view text data includes:

[0028] S31: Model the LDA text topic model and use the joint probability formula to generate the mth article. The process is expressed as Where m represents the generation of the mth article, α is the hyperparameter of the topic distribution, and β is the hyperparameter of the vocabulary distribution. is the word distribution probability under a certain topic, W m is the set of all words in the mth article, W m,n is the nth word in the mth article, Z m The topics assigned to all words in the mth article, Z m,n The topic assigned to the nth word in the mth article, θ m is the topic distribution of the mth article, is the probability of the nth word in topic Z in the mth article, N m is the total number of words in the mth article;

[0029] S32: Input the preprocessed original multi-view text data into the LDA text topic model to obtain the LDA text topic feature vector.

[0030] Preferably, performing spectral clustering on the vector groups formed by the text word embedding weighted vectors and the LDA text topic vectors to obtain clustering results includes:

[0031] S41: constructing a similarity matrix using cosine similarity or Gaussian kernel function for the feature vectors obtained in S24 and S32, and constructing a degree matrix according to the similarity matrix;

[0032] S42: Obtain a Laplacian matrix L=DW based on the degree matrix and the similarity matrix, where D is the degree matrix and W is the Laplacian matrix;

[0033] S43: performing eigenvalue decomposition on the Laplace matrix to obtain eigenvectors corresponding to the first k smallest eigenvalues ​​thereof, where k is a preset number of clusters;

[0034] S44: stack the feature vectors obtained in S43 into a new feature vector matrix, with each sample corresponding to a row. The matrix serves as a new representation of the sample in the low-dimensional space.

[0035] S45: Apply the K-means clustering algorithm to the new feature vector matrix obtained in S44, divide the samples into k classes, and obtain clustering results.

[0036] Preferably, the same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and the different classification data in the clustering results are used as unlabeled samples, and the trained convolutional neural network is used to extract features of the unlabeled samples and perform spectral clustering on the obtained feature vectors to obtain the final clustering results, including:

[0037] S51: The word vector sequence of each text in the word embedding weighted view is an n·m1 matrix, where n is the number of words in the text in the word embedding weighted view, and m1 is the word embedding weighted feature vector of each word in the text in the word embedding weighted view; the LDA feature of each text in the LDA text topic view can be regarded as a 1·m2 vector, where m2 is the LDA text topic feature vector of each text in the text topic view;

[0038] S52: concatenate the word embedding weighted feature vector of each word with the LDA text topic feature vector to obtain a new word vector with a dimension of m1+m2. For each text, the concatenated feature vectors form an n·m matrix, where the feature dimension of each word is m=m1+m2.

[0039] S53: The same classification data in the two groups of clustering results obtained in S45 are used as labeled samples, and the different classification data are used as unlabeled samples; the labeled samples are input into the convolutional neural network for training, and a convolution operation is performed using a convolution kernel with a window size of 1·m to obtain a 1-dimensional vector, and then a pooling window with a size of 1·n is used for maximum pooling. After pooling, a fully connected layer with dropout is connected to perform convolutional neural network training;

[0040] S54: Use f convolution kernels with different weight parameters to perform convolution neural network feature extraction on g unlabeled samples, use convolution kernel with window size of 1·m to perform convolution operation on the unlabeled samples to obtain a 1-dimensional vector, and then use a pooling window with size of 1·n to perform maximum pooling to extract the most important features to obtain a scalar. The pooled results of all unlabeled samples can be composed into a feature vector of f·g dimensions;

[0041] S55: Perform S04 operation on the obtained feature vector to perform spectral clustering.

[0042] Preferably, the step of acquiring the original multi-view text data comprises:

[0043] Displaying a multi-view text data selection box to the user;

[0044] Obtain a portrait sequence of the user; the portrait sequence includes new portraits generated at preset time intervals within the most recent preset time in the history of the user;

[0045] Determine a sequence item cluster that meets the sequence item cluster condition from the portrait sequence;

[0046] The sequence items in the portrait sequence are traversed in sequence. When the i-th sequence item in the portrait sequence is traversed, if the i-th sequence item falls into the j-th sequence item cluster in the portrait sequence, the first i sequence items and the first j sequence item clusters in the portrait sequence are characterized to obtain a first description vector.

[0047] Determine a trigger value corresponding to the first description vector from a trigger value library;

[0048] When the trigger value is greater than or equal to the trigger threshold, the data source screening condition corresponding to the first description vector is determined from the data source screening condition library; the trigger threshold is the sum of the product of the first weight and i plus the product of the second weight and j;

[0049] Based on the data source screening conditions, filter out data sources from the data source library;

[0050] Add additional data sources to the multi-view text data selection box;

[0051] Describing the selection operation sequence generated by the user based on the multi-view text data selection box to obtain a second feature vector;

[0052] Try to determine the data screening condition corresponding to the second eigenvector from the data screening condition library;

[0053] When the attempt is successful, based on the data screening condition, the target data is screened out from the data to be selected in the current multi-view text data selection box;

[0054] When the attempt fails, the sequence item clusters are traversed sequentially starting from the sequence item after the last sequence item in the j-th sequence item cluster in the portrait sequence;

[0055] After traversing the sequence item cluster, the original multi-view text data is determined based on the target data obtained each time a sequence item is traversed.

[0056] Preferably, the sequence term cluster condition includes multiple combinations:

[0057] The sequence item cluster contains at least a preset number of consecutive sequence items;

[0058] There is a generative association relationship between any two sequence items in the sequence item cluster;

[0059] All sequence items in a sequence item cluster have the same sequence item type.

[0060] This application has achieved the following beneficial effects:

[0061] 1. Select consistent samples from clustering results of different views as labeled samples. The present invention reduces the need for external labeled samples, thereby reducing labeling costs and time investment;

[0062] 2. A method combining LDA text topic view and text word embedding weighted view is proposed to make full use of the information complementarity of multiple views and enhance the clustering effect. By integrating the features of multiple views, the present invention can better capture the multiple semantic information of the text and improve the stability of the clustering results;

[0063] 3. The word embedding weighted feature vector and the LDA text topic feature vector are concatenated into a unified feature vector, which improves the quality of feature representation;

[0064] 4. Use convolutional neural network to extract text features, effectively capture the semantic information of the text, reduce feature dimensions, and reduce the impact of label noise, thereby improving the accuracy and robustness of clustering results.

[0065] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0068] Figure 1 It is a flow chart of text data processing of the multi-view text clustering method based on convolutional neural network in the present invention;

[0069] Figure 2 This is a flow chart of a spectral clustering algorithm based on a convolutional neural network in the multi-view text clustering method based on a convolutional neural network in the present invention. DETAILED DESCRIPTION

[0070] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0071] The embodiment of the present invention provides a multi-view text clustering method based on a convolutional neural network, comprising:

[0072] S01: preprocessing the original multi-view text data;

[0073] S02: Calculate the TF-IDF value of the preprocessed original multi-view text data and then combine it with the Word2Vec word vector to generate a text word embedding weighted vector;

[0074] S03: Generate LDA text topic vectors for the preprocessed original multi-view text data;

[0075] S04: Perform spectral clustering on the vector groups formed by the text word embedding weighted vector and the LDA text topic vector to obtain clustering results;

[0076] S05: The same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and the different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features of the unlabeled samples and the obtained feature vectors are spectrally clustered to obtain the final clustering results.

[0077] like Figure 1 to Figure 2 As shown, the specific implementation includes:

[0078] S01: Text data preprocessing, filtering out illegal strings, word segmentation, removing stop words, and replacing synonyms;

[0079] S02: Calculate the TF-IDF value of the preprocessed text data and then combine it with the Word2Vec word vector to generate a text word embedding weighted vector;

[0080] S03: Generate LDA text topic vectors for the preprocessed text data;

[0081] S04: Perform spectral clustering on the vector groups formed by text word embedding weights and LDA text topics respectively;

[0082] S05: The same classification data in the clustering results are used as labeled data to input the convolutional neural network for training, and the different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features and the obtained feature vectors are spectrally clustered;

[0083] This application selects consistent samples from clustering results of different views as labeled samples, reducing the need for external labeled samples, thereby reducing labeling costs and time investment; combining LDA text topic view and text word embedding weighted view, fully utilizing the information complementarity of multiple views, and enhancing the clustering effect; by integrating the features of multiple views, it can better capture the multiple semantic information of the text and improve the stability of the clustering results; splicing the word embedding weighted feature vector and the LDA text topic feature vector into a unified feature vector, improving the quality of feature representation; using convolutional neural network for text feature extraction, effectively capturing the semantic information of the text, reducing feature dimensions, and reducing the impact of label noise, thereby improving the accuracy and robustness of clustering results. It is suitable for various practical application scenarios.

[0084] In one embodiment, the preprocessing of the original multi-view text data includes:

[0085] S11: Use regular expressions to traverse the article content of the original multi-view text data, retain only Chinese characters in the article content, filter out all non-Chinese characters and numeric characters, use the Jieba word segmentation tool to segment the article content, and divide the article content into single words as basic units;

[0086] S12: Use the stop word list of Harbin Institute of Technology, Baidu stop word list or a custom stop word list to delete stop words from the article content;

[0087] S13: Use the Word2Vec method in the CBOW model to generate word vector representations for each word in the article content, and calculate the cosine similarity between word vectors. Where w i is the i-th word, w j is the jth word, v i For word w i The word vector, v j For word w j The word vectors of the word are taken as related words whose similarity exceeds the custom threshold, and the set of related words with the closest semantics is found for each word;

[0088] S14: Select the word with the highest frequency in each associated word set as a representative word, use regular expressions to search and replace all words in the associated word set as representative words in the article content, and obtain the preprocessed text data.

[0089] The method of calculating the TF-IDF value of the preprocessed original multi-view text data and then combining it with the Word2Vec word vector to generate a text word embedding weighted vector includes:

[0090] S21: For each text in the preprocessed original multi-view text data, calculate the word frequency of each word in the text Where n d represents the total number of words in the text d in the original multi-view text data, Count(w, d) represents the number of times word w appears in the text d in the original multi-view text data; calculate the inverse text frequency Where N is the number of texts in the original multi-view text data, DF(w) is the number of texts containing word w in the original multi-view text data; for word w of each text in the original multi-view text data, calculate TF-IDF(w, d) = TF(w, d) · IDF(w);

[0091] S22: Use the trained Word2Vec model to determine each word in the text in the original multi-view text data to obtain its word vector representation;

[0092] S23: For each word in the original multi-view text data, perform word embedding weighting v′ w =TF-IDF(w, d)·v w , where v w is the Word2Vec vector of word w, TF-IDF(w, d) is the TF-IDF value of word w in text d;

[0093] S24: Perform weighted averaging on all weighted word vectors in the text of the original multi-view text data to obtain the text word embedding weighted feature vector Where v d is the vector representation of the text d in the original multi-view text data, ∑ wed TF-IDF(w, d) is the sum of the TF-IDF values ​​of all words in the text in the original multi-view text data.

[0094] The generating of LDA text topic vectors from the preprocessed original multi-view text data includes:

[0095] S31: Model the LDA text topic model and use the joint probability formula to generate the mth article. The process is expressed as Where m represents the generation of the mth article, α is the hyperparameter of the topic distribution, and β is the hyperparameter of the vocabulary distribution. is the word distribution probability under a certain topic, W m is the set of all words in the mth article, W m,n is the nth word in the mth article, Z m The topics assigned to all words in the mth article, Z m,n The topic assigned to the nth word in the mth article, θ m is the topic distribution of the mth article, is the probability of the nth word in topic Z in the mth article, N m is the total number of words in the mth article;

[0096] S32: Input the preprocessed original multi-view text data into the LDA text topic model to obtain the LDA text topic feature vector.

[0097] The step of performing spectral clustering on the vector groups formed by the text word embedding weighted vectors and the LDA text topic vectors to obtain clustering results includes:

[0098] S41: constructing a similarity matrix using cosine similarity or Gaussian kernel function for the feature vectors obtained in S24 and S32, and constructing a degree matrix according to the similarity matrix;

[0099] S42: Obtain a Laplacian matrix L=DW based on the degree matrix and the similarity matrix, where D is the degree matrix and W is the Laplacian matrix;

[0100] S43: performing eigenvalue decomposition on the Laplace matrix to obtain eigenvectors corresponding to the first k smallest eigenvalues ​​thereof, where k is a preset number of clusters;

[0101] S44: stack the feature vectors obtained in S43 into a new feature vector matrix, with each sample corresponding to a row. The matrix serves as a new representation of the sample in the low-dimensional space.

[0102] S45: Apply the K-means clustering algorithm to the new feature vector matrix obtained in S44, divide the samples into k classes, and obtain clustering results.

[0103] The same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features of the unlabeled samples and the obtained feature vectors are spectrally clustered to obtain the final clustering results, including:

[0104] S51: The word vector sequence of each text in the word embedding weighted view is an n·m1 matrix, where n is the number of words in the text in the word embedding weighted view, and mx is the word embedding weighted feature vector of each word in the text in the word embedding weighted view; the LDA feature of each text in the LDA text topic view can be regarded as a 1·m2 vector, where m2 is the LDA text topic feature vector of each text in the text topic view;

[0105] S52: concatenate the word embedding weighted feature vector of each word with the LDA text topic feature vector to obtain a new word vector with a dimension of m1+m2. For each text, the concatenated feature vectors form an n·m matrix, where the feature dimension of each word is m=m1+m2.

[0106] S53: The same classification data in the two groups of clustering results obtained in S45 are used as labeled samples, and the different classification data are used as unlabeled samples; the labeled samples are input into the convolutional neural network for training, and a convolution operation is performed using a convolution kernel with a window size of 1·m to obtain a 1-dimensional vector, and then a pooling window with a size of 1·n is used for maximum pooling. After pooling, a fully connected layer with dropout is connected to perform convolutional neural network training;

[0107] S54: Use f convolution kernels with different weight parameters to perform convolution neural network feature extraction on g unlabeled samples, use convolution kernel with window size of 1·m to perform convolution operation on the unlabeled samples to obtain a 1-dimensional vector, and then use a pooling window with size of 1·n to perform maximum pooling to extract the most important features to obtain a scalar. The pooled results of all unlabeled samples can be composed into a feature vector of f·g dimensions;

[0108] S55: Perform S04 operation on the obtained feature vector to perform spectral clustering.

[0109] Specific implementation includes:

[0110] The preprocessing operation of the text data in step S01 includes:

[0111] S11: Use regular expressions to traverse the original text data, only retain Chinese characters, filter out all non-Chinese characters and numeric characters, and use Jieba word segmentation tool to segment the article, dividing the article into single words as basic units;

[0112] S12: Use the stop word list of Harbin Institute of Technology, Baidu stop word list or a custom stop word list to delete stop words from the text;

[0113] S13: Use the Word2Vec method in the CBOW model to generate word vector representations for each word, and calculate the cosine similarity between word vectors Where w i is the i-th word, w j is the jth word, v i For word w i The word vector, v j For word w j Word vectors, words whose similarity exceeds the custom threshold will be regarded as related words, and the set of related words with the closest semantics to each word is found;

[0114] S14: Select the word with the highest frequency in each associated word set as a representative word, use a regular expression to search and replace all words in the associated word set as the representative word in the text, and obtain the preprocessed text data;

[0115] The step S02 comprises the following steps:

[0116] S21: For each text, calculate the frequency of each word Where n d Represents the total number of words in text d, and Count(w, d) represents the number of times word w appears in text d. Calculate the inverse text frequency Where N is the number of documents, and DF(w) is the number of documents containing word w. For each word w in a document, calculate TF-IDF(w,d) = TF(w,d)·IDF(w);

[0117] S22: Use the trained Word2Vec model to obtain the word vector representation for each word in the text data;

[0118] S23: For each word, perform word embedding weighting vw′=TF-IDF(w,d)·v w , where v w is the Word2Vec vector of word w, TF-IDF(w, d) is the TF-IDF value of word w in text d;

[0119] S24: Perform weighted averaging on all weighted word vectors in the text to obtain the text word embedding weighted feature vector Where v d is the vector representation of text d, ∑ wed TF-IDF(w, d) is the sum of the TF-IDF values ​​of all words in the text;

[0120] The step S03 comprises the following steps:

[0121] S31: Model the LDA text topic model and use the joint probability formula to generate the mth article. The process is expressed as Where m represents the generation of the mth article, α is the hyperparameter of the topic distribution, and β is the hyperparameter of the vocabulary distribution. is the word distribution probability under a certain topic, W m is the set of all words in the mth article, W m,n is the nth word in the mth article, Z m The topics assigned to all words in the mth article, Z m,n The topic assigned to the nth word in the mth article, θ m is the topic distribution of the mth article, is the probability of the nth word in topic Z in the mth article, N m is the total number of words in the mth article;

[0122] S32: input the preprocessed text data into the LDA text topic model to obtain the LDA text topic feature vector;

[0123] The step S04 comprises the following steps:

[0124] S41: constructing a similarity matrix using cosine similarity or Gaussian kernel function for the feature vectors obtained in S24 and S32, and constructing a degree matrix according to the similarity matrix;

[0125] S42: Obtain a Laplacian matrix L=DW from the degree matrix and the similarity matrix, where D is the degree matrix and W is the Laplacian matrix;

[0126] S43: performing eigenvalue decomposition on the Laplace matrix to obtain eigenvectors corresponding to the first k smallest eigenvalues ​​thereof, where k is a preset number of clusters;

[0127] S44: Stack the feature vectors obtained in S43 into a new matrix, with each sample corresponding to a row. This matrix can be regarded as a new representation of the sample in a low-dimensional space;

[0128] S45: Apply the K-means clustering algorithm to the new feature vector matrix obtained in S44 to divide the samples into k classes;

[0129] The step S05 comprises the following steps:

[0130] S51: In the word embedding weighted view, the word vector sequence of each text is an n·m1 matrix, where n is the number of words in the text and m1 is the word embedding weighted feature vector of each word; in the LDA text topic view, the LDA feature of each text can be regarded as a 1·m2 vector, where m2 is the LDA text topic feature vector of each text;

[0131] S52: concatenate the word embedding weighted feature vector of each word with the LDA text topic feature vector to obtain a new word vector with a dimension of m1+m2. For each text, the concatenated feature vectors form an n·m matrix, where the feature dimension of each word is m=m1+m2.

[0132] S53: The same classification data in the two groups of clustering results obtained in S45 are used as labeled samples, and the different classification data are used as unlabeled samples. The labeled samples are input into the convolutional neural network for training, and a convolution operation is performed using a convolution kernel with a window size of 1·m to obtain a 1-dimensional vector, and then a pooling window with a size of 1·n is used for maximum pooling. After pooling, a fully connected layer with dropout is connected to train the convolutional neural network;

[0133] S54: Use f convolution kernels with different weight parameters to perform convolution neural network feature extraction on g unlabeled samples, use convolution kernel with window size of 1·m to perform convolution operation on the unlabeled samples to obtain a 1-dimensional vector, and then use a pooling window with size of 1·n to perform maximum pooling to extract the most important features to obtain a scalar. The pooled results of all unlabeled samples can be composed into a feature vector of f·g dimensions;

[0134] S55: Perform S04 operation on the obtained feature vector to perform spectral clustering.

[0135] The present application effectively solves the above problems by combining two views (LDA text topic view and word embedding weighted view) after preprocessing the text data. The present invention first deletes illegal characters, word segmentation and stop words from the text, and adopts a method based on Word2Vec to replace synonyms. Then, text features are generated from two views: the first view generates a text topic feature vector based on the LDA algorithm, and the second view combines Word2Vec word embedding and TF-IDF weighting to generate a weighted word vector. The feature vector groups are spectrally clustered respectively, the purpose is to obtain the data classified into one category in the text data set from the results generated by the clustering of the two views, and these data are used as labeled samples instead of manually labeled samples, and other data are used as unlabeled samples, thereby reducing manpower; the word embedding weighted feature vector and the LDA text topic feature vector are spliced ​​into a unified feature vector to improve the quality of feature representation. After that, for labeled samples, the present invention uses a convolutional neural network for classification training; for unlabeled samples, the trained convolutional neural network is used for feature extraction. In this way, the present invention simplifies the feature extraction process, improves the quality of feature representation, and reduces the demand for labeled samples. Finally, the present invention inputs the text data after feature extraction into the spectral clustering algorithm for clustering to obtain an optimized clustering result.

[0136] In one embodiment, the step of acquiring the original multi-view text data includes:

[0137] Displaying a multi-view text data selection box to the user; the multi-view text data selection box is a selection box for the user to select the multi-view text data to be clustered, in which the multi-view text data to be selected can be displayed, and a selection operation function is also set;

[0138] Obtain a portrait sequence of the user; the portrait sequence includes new portraits generated at preset time intervals within the most recent preset time in the history of the user; the preset time may be 30 days; and the time interval may be 12 hours;

[0139] Determine a sequence item cluster that meets the sequence item cluster condition from the portrait sequence;

[0140] The sequence items in the portrait sequence are traversed in sequence. When the i-th sequence item in the portrait sequence is traversed, if the i-th sequence item falls into the j-th sequence item cluster in the portrait sequence, the first i sequence items and the first j sequence item clusters in the portrait sequence are characterized to obtain a first description vector; i = 1, 2, 3, ..., M; M is the total number of sequence items in the portrait sequence; j = 1, 2, 3, ..., N; N is the total number of sequence item clusters in the portrait sequence; when the i-th sequence item in the portrait sequence is traversed, if the i-th sequence item falls into the j-th sequence item cluster in the portrait sequence, it means that the data source in the multi-view text data selection box may need to be supplemented at this time, and the data source is the source of the multi-view text data to be selected displayed in the multi-view text data selection box;

[0141] Determine the trigger value corresponding to the first description vector from the trigger value library; there are trigger values ​​corresponding to different first description vectors in the trigger value library, and the trigger value represents the degree to which the first description vector reflects the need to supplement the data source in the multi-view text data selection box; specifically, for example: the first i sequence items and the first j sequence item clusters in the first description vector are characterized in that the new preferences are interested in the number of college graduates, interested in employment big data, interested in employment rate, and interested in employment salary, respectively, and the sequence clusters are interested in employment big data, interested in employment rate, and interested in employment salary, which means that the user is extremely interested in employment, and the trigger value corresponding to the first description vector described by the feature is 8;

[0142] When the trigger value is greater than or equal to the trigger threshold, the data source screening condition corresponding to the first description vector is determined from the data source screening condition library; the trigger threshold is the sum of the product of the first weight and i plus the product of the second weight and j; the first weight and the second weight are preset weights; when the trigger value is greater than or equal to the trigger threshold, it represents that it is determined that the data source in the multi-view text data selection box needs to be supplemented; the data source screening condition library has data source screening conditions corresponding to different first description vectors, and the data source screening condition is a condition for screening out the data source that needs to be used as the data source to supplement the data source in the multi-view text data selection box; specifically, for example: the data source screening condition is a condition for screening out the data source that can provide employment data;

[0143] Based on the data source screening conditions, data sources are screened out from the data source library. There are a large number of data sources in the data source library. Based on the data source screening conditions, data sources are screened out from the data source library.

[0144] The data source is supplemented and connected to the multi-view text data selection box; when the data source is supplemented and connected to the multi-view text data selection box, the multi-view text data to be selected corresponding to the data source will be newly displayed in the multi-view text data selection box;

[0145] A feature description is performed on a selection operation sequence generated by the user based on the multi-view text data selection box to obtain a second feature vector; when the user views the multi-view text data selection box at this time, a selection operation sequence is generated, and the selection operation sequence includes selection operations generated successively by the user based on the multi-view text data selection box;

[0146] Try to determine the data filtering condition corresponding to the second feature vector from the data filtering condition library; the data filtering condition is the filtering condition reflected by the second feature description vector, which needs to filter out the multi-view text data to be selected from the multi-view text data corresponding to the data source newly displayed in the multi-view text data selection box, which the user actually wants to use as the multi-view text data for clustering. Specifically, for example: the feature description is a selection operation sequence of the second feature vector, which is to check the job type data, employment salary data, etc. in turn, which means that the multi-view text data that the user actually wants to use as the clustering is employment-related data, and the data filtering condition is to filter out the employment-related data;

[0147] When the attempt is successful, based on the data screening condition, the target data is screened out from the data to be selected in the current multi-view text data selection box;

[0148] When the attempt fails, the sequence item clusters are traversed sequentially starting from the sequence item after the last sequence item in the j-th sequence item cluster in the portrait sequence; When the attempt fails, it means that the user does not see the data to be selected that actually needs to be selected, and the sequence item clusters are traversed sequentially starting from the sequence item after the last sequence item in the j-th sequence item cluster in the portrait sequence;

[0149] After traversing the sequence item cluster, the original multi-view text data is determined based on the target data obtained each time a sequence item is traversed.

[0150] The sequence term cluster conditions include multiple combinations:

[0151] The sequence item cluster contains at least a preset number of consecutive sequence items; the preset number may be 3;

[0152] There is a generation association relationship between any two sequence items in the sequence item cluster; the generation association relationship may be generated by the same generation scene, etc.;

[0153] The sequence item type of all sequence items in the sequence item cluster is the same. The sequence item type is a portrait type, etc. When the three sequence item cluster conditions are met, the sequence items in the selected sequence cluster are used to reflect the user's need to select multi-view text data. If the attempt to select the target data from the data to be selected in the current multi-view text data selection box based on the data screening condition fails, the sequence item clusters can be directly traversed from the next sequence item after the last sequence item in the jth sequence item cluster in the portrait sequence.

[0154] When acquiring original multi-view text data, the embodiment of the present invention introduces a multi-view text data selection box to assist users in selecting multi-view text data, and secondly quickly determines user needs based on the first feature vector and the second feature vector. Thirdly, sequence clusters are introduced to further improve the efficiency, accuracy and comprehensiveness of determining user needs.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A multi-view text clustering method based on convolutional neural network, characterized in that: include: S01: preprocessing the original multi-view text data; S02: Calculate the TF-IDF value of the preprocessed original multi-view text data and then combine it with the Word2Vec word vector to generate a text word embedding weighted vector; S03: Generate LDA text topic vectors for the preprocessed original multi-view text data; S04: Perform spectral clustering on the vector groups formed by the text word embedding weighted vector and the LDA text topic vector to obtain clustering results; S05: The same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and the different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features of the unlabeled samples and the obtained feature vectors are spectrally clustered to obtain the final clustering results.

2. The multi-view text clustering method based on convolutional neural network as claimed in claim 1, characterized in that: The preprocessing of the original multi-view text data comprises: S11: Use regular expressions to traverse the article content of the original multi-view text data, retain only Chinese characters in the article content, filter out all non-Chinese characters and numeric characters, use the Jieba word segmentation tool to segment the article content, and divide the article content into single words as basic units; S12: Use the stop word list of Harbin Institute of Technology, Baidu stop word list or a custom stop word list to delete stop words from the article content; S13: Use the Word2Vec method in the CBOW model to generate word vector representations for each word in the article content, and calculate the cosine similarity between word vectors. Where w i is the i-th word, w j is the jth word, v i For word w i The word vector, v j For word w j The word vectors of the word are taken as related words whose similarity exceeds the custom threshold, and the set of related words with the closest semantics is found for each word; S14: Select the word with the highest frequency in each associated word set as a representative word, use regular expressions to search and replace all words in the associated word set as representative words in the article content, and obtain the preprocessed text data.

3. The multi-view text clustering method based on convolutional neural network as claimed in claim 2, characterized in that: The method of calculating the TF-IDF value of the preprocessed original multi-view text data and then combining it with the Word2Vec word vector to generate a text word embedding weighted vector includes: S21: For each text in the preprocessed original multi-view text data, calculate the word frequency of each word in the text Where n d represents the total number of words in the text d in the original multi-view text data, Count(w, d) represents the number of times word w appears in the text d in the original multi-view text data; calculate the inverse text frequency Where N is the number of texts in the original multi-view text data, DF(w) is the number of texts containing word w in the original multi-view text data; for word w of each text in the original multi-view text data, calculate TF-IDF(w, d) = TF(w, d) · IDF(W); S22: Use the trained Word2Vec model to determine each word in the text in the original multi-view text data to obtain its word vector representation; S23: For each word in the original multi-view text data, perform word embedding weighting v′ w =TF-IDF(w, d)·v w , where v w is the Word2Vec vector of word w, TF-IDF(w, d) is the TF-IDF value of word w in text d; S24: Perform weighted averaging on all weighted word vectors in the text of the original multi-view text data to obtain the text word embedding weighted feature vector Where v d is the vector representation of the text d in the original multi-view text data, ∑ wed TF-IDF(w, d) is the sum of the TF-IDF values ​​of all words in the text in the original multi-view text data.

4. The multi-view text clustering method based on convolutional neural network as claimed in claim 3, characterized in that: The generating of LDA text topic vectors from the preprocessed original multi-view text data includes: S31: Model the LDA text topic model and use the joint probability formula to generate the mth article. The process is expressed as Where m represents the generation of the mth article, α is the hyperparameter of the topic distribution, and β is the hyperparameter of the vocabulary distribution. is the word distribution probability under a certain topic, W m is the set of all words in the mth article, W m,n is the nth word in the mth article, Z m The topics assigned to all words in the mth article, Z m,n The topic assigned to the nth word in the mth article, θ m is the topic distribution of the mth article, is the probability of the nth word in topic Z in the mth article, N m is the total number of words in the mth article; S32: Input the preprocessed original multi-view text data into the LDA text topic model to obtain the LDA text topic feature vector.

5. The multi-view text clustering method based on convolutional neural network as claimed in claim 4, characterized in that: The step of performing spectral clustering on the vector groups formed by the text word embedding weighted vectors and the LDA text topic vectors to obtain clustering results includes: S41: constructing a similarity matrix using cosine similarity or Gaussian kernel function for the feature vectors obtained in S24 and S32, and constructing a degree matrix according to the similarity matrix; S42: Obtain a Laplacian matrix L=DW based on the degree matrix and the similarity matrix, where D is the degree matrix and W is the Laplacian matrix; S43: performing eigenvalue decomposition on the Laplace matrix to obtain eigenvectors corresponding to the first k smallest eigenvalues ​​thereof, where k is a preset number of clusters; S44: stack the feature vectors obtained in S43 into a new feature vector matrix, with each sample corresponding to a row. The matrix serves as a new representation of the sample in the low-dimensional space. S45: Apply the K-means clustering algorithm to the new feature vector matrix obtained in S44, divide the samples into k classes, and obtain clustering results.

6. The multi-view text clustering method based on convolutional neural network as claimed in claim 5, characterized in that: The same classification data in the clustering results are input as labeled data into the convolutional neural network for training, and different classification data in the clustering results are used as unlabeled samples. The trained convolutional neural network is used to extract features of the unlabeled samples and the obtained feature vectors are spectrally clustered to obtain the final clustering results, including: S51: The word vector sequence of each text in the word embedding weighted view is an n·m1 matrix, where n is the number of words in the text in the word embedding weighted view, and m1 is the word embedding weighted feature vector of each word in the text in the word embedding weighted view; the LDA feature of each text in the LDA text topic view can be regarded as a 1·m2 vector, where m2 is the LDA text topic feature vector of each text in the text topic view; S52: concatenate the word embedding weighted feature vector of each word with the LDA text topic feature vector to obtain a new word vector with a dimension of m1+m2. For each text, the concatenated feature vectors form an n·m matrix, where the feature dimension of each word is m=m1+m2. S53: The same classification data in the two groups of clustering results obtained in S45 are used as labeled samples, and the different classification data are used as unlabeled samples; the labeled samples are input into the convolutional neural network for training, and a convolution operation is performed using a convolution kernel with a window size of 1·m to obtain a 1-dimensional vector, and then a pooling window with a size of 1·n is used for maximum pooling. After pooling, a fully connected layer with dropout is connected to perform convolutional neural network training; S54: Use f convolution kernels with different weight parameters to perform convolution neural network feature extraction on g unlabeled samples, use convolution kernel with window size of 1·m to perform convolution operation on the unlabeled samples to obtain a 1-dimensional vector, and then use a pooling window with size of 1·n to perform maximum pooling to extract the most important features to obtain a scalar. The pooled results of all unlabeled samples can be composed into a feature vector of f·g dimensions; S55: Perform S04 operation on the obtained feature vector to perform spectral clustering.

7. The multi-view text clustering method based on convolutional neural network as claimed in claim 1, characterized in that: The step of acquiring the original multi-view text data comprises: Displaying a multi-view text data selection box to the user; Obtain a portrait sequence of the user; the portrait sequence includes new portraits generated at preset time intervals within the most recent preset time in the history of the user; Determine a sequence item cluster that meets the sequence item cluster condition from the portrait sequence; The sequence items in the portrait sequence are traversed in sequence. When the i-th sequence item in the portrait sequence is traversed, if the i-th sequence item falls into the j-th sequence item cluster in the portrait sequence, the first i sequence items and the first j sequence item clusters in the portrait sequence are characterized to obtain a first description vector. Determine a trigger value corresponding to the first description vector from a trigger value library; When the trigger value is greater than or equal to the trigger threshold, the data source screening condition corresponding to the first description vector is determined from the data source screening condition library; the trigger threshold is the sum of the product of the first weight and i plus the product of the second weight and j; Based on the data source screening conditions, filter out data sources from the data source library; Add additional data sources to the multi-view text data selection box; Describing the selection operation sequence generated by the user based on the multi-view text data selection box to obtain a second feature vector; Try to determine the data screening condition corresponding to the second eigenvector from the data screening condition library; When the attempt is successful, based on the data screening condition, the target data is screened out from the data to be selected in the current multi-view text data selection box; When the attempt fails, the sequence item clusters are traversed sequentially starting from the sequence item after the last sequence item in the j-th sequence item cluster in the portrait sequence; After traversing the sequence item cluster, the original multi-view text data is determined based on the target data obtained each time a sequence item is traversed.

8. The multi-view text clustering method based on convolutional neural network as claimed in claim 7, characterized in that: The sequence term cluster conditions include multiple combinations: The sequence item cluster contains at least a preset number of consecutive sequence items; There is a generative association relationship between any two sequence items in the sequence item cluster; All sequence items in a sequence item cluster have the same sequence item type.

Images