Text clustering method, device, electronic device and computer program product

Abstract

The present application relates to the technical field of text processing, and provides a text clustering method, device, electronic device and computer program product, the method comprising: calculating the category label keyword vector and text keyword vector of each text to be clustered based on a pre-trained word vector set and the category label keywords and text keywords of each text to be clustered; calculating the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph; clustering each text to be clustered based on the similarity value between the category label keyword vector and text keyword vector of each text to be clustered and the category average keyword vector and category document average keyword vector of each cluster. The text clustering method provided in the embodiment of the present application clusters the text to be clustered through the knowledge graph and word vector, thereby improving the efficiency of text clustering.

Description

Technical Field

[0001] The present application relates to the field of text processing technology, and in particular to a text clustering method, device, electronic device, and computer program product. Background Art

[0002] Currently, the most commonly used clustering algorithm for text clustering is the hierarchical clustering algorithm. Hierarchical clustering algorithms cluster small clusters layer by layer, from bottom to top. Each time, they find the two clusters with the shortest distance, then merge them into a larger cluster until all clusters are merged into a single, final cluster. As can be seen from the above analysis, hierarchical clustering algorithms must calculate the pairwise distances of all data points within multiple clusters, resulting in a large computational load and low text clustering efficiency. Summary of the Invention

[0003] The present application provides a text clustering method, apparatus, electronic device, and computer program product, aiming to improve the efficiency of text clustering.

[0004] In a first aspect, the present application provides a text clustering method, comprising:

[0005] Calculate the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered;

[0006] Calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0007] The texts to be clustered are clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0008] In one embodiment, the step of clustering the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster includes:

[0009] Multiplying the category label keyword vector of each of the to-be-clustered texts by the category average keyword vector of each of the clusters to obtain a first calculation result;

[0010] Multiplying the text keyword vector of each of the to-be-clustered texts by the average keyword vector of the category documents of each of the clusters to obtain a second calculation result;

[0011] Calculating the similarity between each of the to-be-clustered texts and each of the clusters based on each of the first calculation results and the corresponding category weight value, and based on each of the second calculation results and the corresponding text weight value;

[0012] The texts to be clustered are clustered according to the similarity values ​​and a preset similarity threshold.

[0013] The step of clustering the texts to be clustered according to the similarity values ​​and the preset similarity threshold comprises:

[0014] Determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values;

[0015] If the target similarity value exists, the target similarity values ​​of the same text to be clustered are classified to obtain sets of target similarity values;

[0016] The to-be-clustered texts corresponding to each of the target similarity value sets are clustered with the cluster corresponding to the target similarity value with the largest value in each of the target similarity value sets.

[0017] After the step of determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values, the method further includes:

[0018] If the target similarity value does not exist, then creating new clusters for each text to be clustered in the preset knowledge graph;

[0019] Each text to be clustered is clustered with its corresponding newly added cluster, and each clustered text after clustering is added to the preset knowledge graph.

[0020] The step of calculating the category label keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keyword of each text to be clustered includes:

[0021] Segmenting each of the texts to be clustered according to the category label keywords to obtain each category label keyword of each of the texts to be clustered;

[0022] Calculate the word vector of each category label keyword of each of the texts to be clustered according to the preset training word vector;

[0023] The word vectors of each category label keyword of each of the texts to be clustered are summed and averaged to obtain the category label keyword vectors of each of the texts to be clustered.

[0024] The step of calculating the text keyword vector of each text to be clustered based on the pre-trained word vector set and the text keywords of each text to be clustered includes:

[0025] Performing word segmentation on each of the texts to be clustered and extracting text keywords to obtain each text keyword of each of the texts to be clustered;

[0026] Calculate the word vector of each text keyword of each of the texts to be clustered according to the preset training word vector;

[0027] The word vectors of each text keyword of each of the texts to be clustered are summed and averaged to obtain the text keyword vectors of each of the texts to be clustered.

[0028] After the step of clustering the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster, the method further includes:

[0029] Determine whether the number of clustered texts in a preset knowledge graph is greater than or equal to a preset value;

[0030] If the number is greater than or equal to the preset value, word vector training is performed using new text corpus to obtain a new set of pre-trained word vectors;

[0031] Each text to be clustered is clustered using the new pre-trained word vector set.

[0032] In a second aspect, the present application further provides a text clustering device, comprising:

[0033] A first calculation module is used to calculate the category label keyword vector and the text keyword vector of each text to be clustered according to the pre-trained word vector set and the category label keyword and text keyword of each text to be clustered;

[0034] A second calculation module is used to calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0035] The clustering module is used to cluster the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0036] In a third aspect, the present application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the text clustering method described in the first aspect are implemented.

[0037] In a fourth aspect, the present application further provides a computer program product, comprising a computer program, which implements the steps of the text clustering method described in the first aspect when executed by the processor.

[0038] The text clustering method, device, electronic device, and computer program product provided by this application cluster the texts to be clustered using the structured information of the knowledge graph and the contextual information of the word vectors during the text clustering process, thereby optimizing the clustering algorithm for text clustering. This allows text clustering to be achieved through a simple algorithm that combines the knowledge graph and word vectors, thereby improving the efficiency of text clustering. Furthermore, during the text clustering process, the distance between the texts to be clustered does not need to be calculated multiple times. Instead, the distance between the texts to be clustered is measured by the similarity values ​​between the word vectors, which reduces the amount of calculation and improves the efficiency of text clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0040] Figure 1 This is one of the flow charts of the text clustering method provided in this application;

[0041] Figure 2 This is the second flowchart of the text clustering method provided by this application;

[0042] Figure 3 This is the third flowchart of the text clustering method provided by this application;

[0043] Figure 4 It is a structural diagram of the text clustering device provided by this application;

[0044] Figure 5 It is a structural diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0046] The following combination Figures 1 to 5 The present invention describes the text clustering method, device, electronic device and computer program product provided by the present application.

[0047] Specifically, this application provides a text clustering method, referring to Figure 1 , Figure 1 This is one of the flow charts of the text clustering method provided in this application.

[0048] The embodiments of the present application provide embodiments of the text clustering method. It should be noted that although a logical order is shown in the flowchart, under certain data, the steps shown or described may be completed in an order different from that shown here.

[0049] The embodiment of the present application takes an electronic device as an example of an execution subject. The embodiment of the present application takes a text clustering system as one of the manifestations of the electronic device, but does not limit the electronic device.

[0050] The text clustering method provided in the embodiment of the present application includes:

[0051] Step S10 , calculating the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered.

[0052] It should be noted that, in this embodiment, before calculating the category label keyword vector and text keyword vector of each text to be clustered, the text clustering system needs to obtain a preset number of text corpora from a preset text corpus, wherein the preset text corpus includes but is not limited to an online text corpus (such as blogs, microblogs, and web pages), an NLP (Natural Language Processing) text corpus, and a BCC corpus. The preset number can be but is not limited to 1,000, 10,000, and 100,000. Then, the text clustering system performs word vector training on the text corpus to obtain a pre-trained word vector set Vectors.

[0053] The text clustering system obtains each text to be clustered and performs word segmentation on each text to be clustered based on the category label keywords and text keywords in the text to be clustered, thereby obtaining the category label keywords label_keywords and text keywords text_keywords of each text to be clustered. Next, the text clustering system inputs the category label keywords label_keywords and text keywords text_keywords of each text to be clustered into the trained pre-trained word vector set Vectors for calculation, thereby obtaining the category label keyword vector lv and text keyword vector tv of each text to be clustered. The specific calculation method of the category label keyword vector lv and text keyword vector tv is as described in steps S101 to S106.

[0054] Furthermore, the specific description of steps S101 to S106 is as follows:

[0055] Step S101, segmenting each of the texts to be clustered according to the category label keywords to obtain each category label keyword of each of the texts to be clustered;

[0056] Step S102, calculating the word vector of each category label keyword of each of the texts to be clustered based on the preset training word vector;

[0057] Step S103 , summing up the word vectors of the category label keywords of the documents to be clustered and taking the average value to obtain the category label keyword vectors of the documents to be clustered.

[0058] Specifically, the text clustering system segments each text to be clustered according to the category label keywords to obtain each category label keyword of each text to be clustered, and then inputs each category label keyword of each text to be clustered into the trained pre-trained word vector set to obtain the word vector of each category label keyword of each text to be clustered. Then, the text clustering system sums the word vectors of each category label keyword of each text to be clustered, and averages the sum obtained to obtain the category label keyword vector of each text to be clustered. In this embodiment, for example, there are 3 category label keywords in the text to be clustered 1, and the 3 category label keywords are category label keyword 1, category label keyword 2 and category label keyword 3, and the word vectors of the category label keywords corresponding to them are word vector lv1, word vector lv2 and word vector lv3, and the category label keyword vector lv of the text to be clustered 1 is calculated to be (lv1+lv2+lv3) / 3.

[0059] The embodiment of the present application calculates the category label keyword vector of each text to be clustered through a trained pre-trained word vector set. Since the pre-trained word vector set is obtained through training on a large amount of text corpus, the accuracy of the category label keyword vector is guaranteed.

[0060] Step S104, performing word segmentation on each of the texts to be clustered and extracting text keywords to obtain each text keyword of each of the texts to be clustered;

[0061] Step S105, calculating the word vector of each text keyword of each of the texts to be clustered according to the preset training word vector;

[0062] Step S106 , summing up the word vectors of the text keywords of the texts to be clustered and taking the average value to obtain the text keyword vectors of the texts to be clustered.

[0063] Furthermore, the text clustering system performs word segmentation on each text to be clustered and extracts text keywords to obtain each text keyword of each text to be clustered, and then inputs each text keyword of each text to be clustered into a trained pre-trained word vector set to obtain a word vector of each text keyword of each text to be clustered. Next, the text clustering system sums the word vectors of each text keyword of each text to be clustered, and averages the sum obtained to obtain a text keyword vector of each text to be clustered. In this embodiment, for example, there are 3 text keywords in text 1 to be clustered, and the 3 text keywords are text keyword 1, text keyword 2 and text keyword 3, and the word vectors of the text keywords corresponding to them are word vector tv1, word vector tv2 and word vector tv3, respectively. The text keyword vector tv of text 1 to be clustered is calculated to be (tv1+tv2+tv3) / 3.

[0064] The embodiment of the present application calculates the text keyword vectors of each text to be clustered through a trained pre-trained word vector set. Since the pre-trained word vector set is obtained through training on a large amount of text corpus, the accuracy of the text keyword vector is guaranteed.

[0065] Step S20, based on the pre-trained word vector set and each cluster document in the preset knowledge graph, calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph.

[0066] The text clustering system obtains each cluster document in a preset knowledge graph, wherein the preset knowledge graph is pre-set by a technician in the text clustering system, and then inputs each cluster document in the preset knowledge graph into a trained pre-trained word vector set for calculation to obtain the category average keyword vector cv and category document average keyword vector dv of each cluster in the preset knowledge graph. It should be noted that the calculation method of the category average keyword vector cv and category document average keyword vector dv of each cluster in the preset knowledge graph is the same as the calculation method of the category label keyword vector lv and the text keyword vector tv in the above S10, and therefore will not be repeated in this embodiment.

[0067] Step S30 , clustering the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0068] The text clustering system calculates the first result based on the category label keyword vector of each text to be clustered and the category average keyword vector of each cluster in the preset knowledge graph, and calculates the text keyword vector of each text to be clustered and the category document average keyword vector of each cluster in the preset knowledge graph to obtain a second result. Then, the text clustering system further calculates based on the first result and the second result to obtain the similarity value between each text to be clustered and each cluster in the preset knowledge graph. Finally, the text clustering system clusters each text to be clustered with each cluster in the preset knowledge graph based on the similarity value between each text to be clustered and each cluster in the preset knowledge graph to obtain each clustered text, as described in steps S301 to S304.

[0069] It should be noted that in the process of text clustering, before clustering, it is necessary to determine whether the number of clusters in the preset knowledge graph is 0. If the number of clusters in the preset knowledge graph is 0, that is, the category average keyword vector and the category document average keyword vector are both 0, then it is necessary to create new clusters corresponding to the current texts to be clustered, and cluster each text to be clustered with its corresponding new cluster to obtain each clustered text, and determine each clustered text as an initial cluster and add it to the preset knowledge graph.

[0070] In the case where clusters do not exist in the preset knowledge graph, the embodiment of the present application can create corresponding new clusters based on each text to be clustered, and cluster each text to be clustered with its corresponding new cluster, thereby making the clusters in the preset knowledge graph richer.

[0071] This embodiment provides a text clustering method. During the text clustering process, the texts to be clustered are clustered using the structured information of the knowledge graph and the contextual information of the word vectors. This optimizes the clustering algorithm for text clustering, enabling text clustering to be achieved through a simple algorithm that combines the knowledge graph and word vectors, thereby improving the efficiency of text clustering. Furthermore, during the text clustering process, the distance between the texts to be clustered does not need to be calculated multiple times. Instead, the distance between the texts to be clustered is measured using the similarity values ​​between the word vectors, reducing the amount of calculation and improving the efficiency of text clustering.

[0072] Further, refer to Figure 2 , Figure 2 This is the second flow chart of the text clustering method provided by this application, wherein step S30 includes:

[0073] Step S301, multiplying the category label keyword vector of each of the to-be-clustered texts by the category average keyword vector of each of the clusters to obtain first calculation results;

[0074] Step S302, multiplying the text keyword vector of each to-be-clustered text by the average keyword vector of the category documents of each cluster to obtain each second calculation result;

[0075] Step S303, calculating the similarity value between each of the to-be-clustered texts and each of the clusters based on each of the first calculation results and the corresponding category weight value, and based on each of the second calculation results and the corresponding text weight value;

[0076] Step S304: clustering the texts to be clustered according to the similarity values ​​and a preset similarity threshold.

[0077] The text clustering system multiplies the category label keyword vector of each to-be-clustered text with the category average keyword vector of each cluster in the preset knowledge graph to obtain each first calculation result, i.e., each first calculation result = the category label keyword vector lv of each to-be-clustered text * the category average keyword vector cv of each cluster. Next, the text clustering system multiplies the text keyword vector of each to-be-clustered text with the category document average keyword vector of each cluster to obtain each second calculation result, i.e., the second calculation result = the text keyword vector tv of each to-be-clustered text * the category document average keyword vector dv of each cluster. Next, the text clustering system multiplies each first calculation result and its corresponding category weight value w1 to obtain a first calculation value, and multiplies each second calculation result and its corresponding text weight value w2 to obtain a second calculation value. The first calculation value and the second calculation value are summed to obtain the similarity value between each text to be clustered and each cluster, that is, similarity value = (each category label keyword vector lv*each category average keyword vector cv)*category weight value w1+(each text keyword vector tv*each category document average keyword vector dv)*text weight value w2. Finally, the text clustering system compares the similarity value between each text to be clustered and each cluster with a preset similarity threshold to obtain a comparison result, and then clusters each text to be clustered with each cluster in the preset knowledge graph based on the comparison result to obtain each clustered text. Specifically, as described in steps S3041 to S3045, the comparison result can be that the similarity value is greater than or equal to the preset similarity threshold, or the comparison result can be that the similarity value is less than the preset similarity threshold.

[0078] This embodiment of the application calculates the similarity between each text to be clustered and each cluster by using word vectors and their weights, ensuring the accuracy of the calculated similarity values. Furthermore, text classification is performed based on the similarity between word vectors and a preset similarity threshold, reducing the amount of computation and improving the efficiency of text clustering.

[0079] Furthermore, the specific descriptions of steps S3041 to S3043 are as follows:

[0080] Step S3041, determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values;

[0081] Step S3042: if the target similarity value exists, classify the target similarity values ​​of the same text to be clustered to obtain a set of target similarity values;

[0082] Step S3043, clustering the to-be-clustered texts corresponding to each of the target similarity value sets with the cluster corresponding to the target similarity value with the largest value in each of the target similarity value sets;

[0083] Step S3044: if the target similarity value does not exist, then creating new clusters for each text to be clustered in the preset knowledge graph;

[0084] Step S3045: cluster each to-be-clustered text with its corresponding newly added cluster, and add each clustered text after clustering to the preset knowledge graph.

[0085] Specifically, the text clustering system determines whether there is a target similarity value greater than or equal to a preset similarity threshold value among the various similarity values. It should be noted that the target similarity value is a general term, rather than indicating that there is only one target similarity value. If it is determined that a target similarity value exists, the text clustering system classifies the target similarity values ​​of the same text to be clustered to obtain various target similarity value sets. In this embodiment, for example, there are target similarity value 1, target similarity value 2, target similarity value 3, and target similarity value 4. Target similarity value 1 and target similarity value 4 are both target similarity values ​​for text to be clustered 1, and target similarity value 2 and target similarity value 3 are both target similarity values ​​for text to be clustered 2. Then, target similarity value set 1 for text to be clustered 1 = {target similarity value 1, target similarity value 4}, and target similarity value set 2 for text to be clustered 2 = {target similarity value 2, target similarity value 3}.

[0086] Next, the text clustering system determines the cluster corresponding to the target similarity value with the largest value in each target similarity value set, and clusters the text to be clustered corresponding to each target similarity value set with the cluster corresponding to the target similarity value with the largest value in each target similarity value set, thereby obtaining each clustered text. It should be noted that if all target similarity values ​​in the target similarity value set are equal, then any target similarity value is selected. In this embodiment, for example, if target similarity value set 1 of text to be clustered 1 = {target similarity value 1, target similarity value 4}, and target similarity value set 2 of text to be clustered 2 = {target similarity value 2, target similarity value 3}, target similarity value 1 is greater than target similarity value 4, and target similarity value 3 is greater than target similarity value 2, then the cluster corresponding to target similarity value 1 is clustered with text to be clustered 1, thereby obtaining clustered text 1; and the cluster corresponding to target similarity value 3 is clustered with text to be clustered 2, thereby obtaining clustered text 2.

[0087] The embodiment of the present application performs text classification by using the similarity values ​​between word vectors and a preset similarity threshold, thereby reducing the amount of calculation and improving the efficiency of text clustering.

[0088] Furthermore, if it is determined that the target similarity value does not exist, the text clustering system needs to create each new cluster corresponding to each text to be clustered in the preset knowledge graph. Then, the text clustering system clusters each text to be clustered with its corresponding new cluster to obtain each clustered text, and adds each clustered text to the preset knowledge graph. In this embodiment, for example, the texts to be clustered include text to be clustered 1, text to be clustered 2, and text to be clustered 3. The new clusters created based on text to be clustered 1, text to be clustered 2, and text to be clustered 3 are new cluster 1, new cluster 2, and new cluster 3. The texts to be clustered include text to be clustered 1 and new cluster 1 to obtain cluster text 1; cluster text 2 and new cluster 2 to obtain cluster text 2; cluster text 3 and new cluster 3 to obtain cluster text 3. Finally, cluster text 1, cluster text 2, and cluster text 3 are added to the preset knowledge graph.

[0089] In the embodiment of the present application, when there are no clusters corresponding to each text to be clustered in the preset knowledge graph, corresponding new clusters can be created based on each text to be clustered, and each text to be clustered can be clustered with its corresponding new cluster, thereby enriching the clusters in the preset knowledge graph.

[0090] Further, refer to Figure 3 , Figure 3 This is the third flow chart of the text clustering method provided by this application. After step S30, the following steps are also included:

[0091] Step S40, determining whether the number of clustered texts in the preset knowledge graph is greater than or equal to a preset value;

[0092] Step S50: If the number is greater than or equal to the preset value, word vector training is performed using new text corpus to obtain a new pre-trained word vector set;

[0093] Step S60: clustering each text to be clustered using the new pre-trained word vector set.

[0094] After the text clustering system clusters each text to be clustered, it needs to determine the number of clustered texts in the preset knowledge graph at a preset interval, where the preset interval is set according to actual conditions, and determines whether the number of clustered texts in the preset knowledge graph is greater than or equal to a preset value. If it is determined that the number of clustered texts in the preset knowledge graph is greater than or equal to the preset value, the text clustering system obtains new text corpus from the preset corpus, performs word vector training on the new text corpus, obtains a new set of pre-trained word vectors, and clusters each text to be clustered using the new set of pre-trained word vectors.

[0095] This embodiment provides a text clustering method, which monitors the number of clustered texts in a preset knowledge graph in real time, trains a new pre-trained word vector set based on the number of clustered texts, improves the accuracy of the pre-trained word vector set, and thus improves the accuracy of the similarity value between each text to be clustered and each cluster.

[0096] Furthermore, the text clustering device provided in the present application is described below. The text clustering device described below and the text clustering method described above can be referred to in correspondence with each other.

[0097] like Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of the text clustering device provided by this application, which includes:

[0098] A first calculation module 401 is configured to calculate a category label keyword vector and a text keyword vector of each text to be clustered based on a pre-trained word vector set and the category label keywords and text keywords of each text to be clustered;

[0099] A second calculation module 402 is configured to calculate the category average keyword vector and the category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0100] The clustering module 403 is configured to cluster the texts to be clustered according to the similarity between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0101] Furthermore, the clustering module 403 is further configured to:

[0102] Multiplying the category label keyword vector of each of the to-be-clustered texts by the category average keyword vector of each of the clusters to obtain each first calculation result;

[0103] Multiplying the text keyword vector of each of the to-be-clustered texts by the average keyword vector of the category documents of each of the clusters to obtain each second calculation result;

[0104] Calculating the similarity between each of the to-be-clustered texts and each of the clusters based on each of the first calculation results and the corresponding category weight value, and based on each of the second calculation results and the corresponding text weight value;

[0105] The texts to be clustered are clustered according to the similarity values ​​and a preset similarity threshold.

[0106] Furthermore, the clustering module 403 is further configured to:

[0107] Determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values;

[0108] If the target similarity value exists, the target similarity values ​​of the same text to be clustered are classified to obtain sets of target similarity values;

[0109] The to-be-clustered texts corresponding to each of the target similarity value sets are clustered with the cluster corresponding to the target similarity value with the largest value in each of the target similarity value sets.

[0110] Furthermore, the text clustering device further includes a filling module, which is used to:

[0111] If the target similarity value does not exist, then creating new clusters for each text to be clustered in the preset knowledge graph;

[0112] Each text to be clustered is clustered with its corresponding newly added cluster, and each clustered text after clustering is added to the preset knowledge graph.

[0113] Furthermore, the first calculation module 401 is further configured to:

[0114] Segmenting each of the texts to be clustered according to the category label keywords to obtain each category label keyword of each of the texts to be clustered;

[0115] Calculate the word vector of each category label keyword of each of the texts to be clustered according to the preset training word vector;

[0116] The word vectors of each category label keyword of each of the texts to be clustered are summed and averaged to obtain the category label keyword vectors of each of the texts to be clustered.

[0117] Furthermore, the first calculation module 401 is further configured to:

[0118] Performing word segmentation on each of the texts to be clustered and extracting text keywords to obtain each text keyword of each of the texts to be clustered;

[0119] Calculate the word vector of each text keyword of each of the texts to be clustered according to the preset training word vector;

[0120] The word vectors of each text keyword of each of the texts to be clustered are summed and averaged to obtain the text keyword vectors of each of the texts to be clustered.

[0121] Furthermore, the clustering module 403 is further configured to:

[0122] Determine whether the number of clustered texts in a preset knowledge graph is greater than or equal to a preset value;

[0123] If the number is greater than or equal to the preset value, word vector training is performed using new text corpus to obtain a new set of pre-trained word vectors;

[0124] Each text to be clustered is clustered using the new pre-trained word vector set.

[0125] The specific embodiments of the text clustering device provided in this application are basically the same as the embodiments of the above-mentioned text clustering method, and will not be described in detail here.

[0126] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other via the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the text clustering method, which includes:

[0127] Calculate the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered;

[0128] Calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0129] The texts to be clustered are clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0130] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0131] On the other hand, the present application also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer can perform the text clustering method provided by the above methods, the method including:

[0132] Calculate the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered;

[0133] Calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0134] The texts to be clustered are clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0135] In another aspect, the present application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the above-mentioned text clustering method, the method comprising:

[0136] Calculate the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered;

[0137] Calculate the category average keyword vector and category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph;

[0138] The texts to be clustered are clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster.

[0139] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0140] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A text clustering method, characterized in that: include: Calculate the category label keyword vector and text keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keywords and text keywords of each text to be clustered; According to the pre-trained word vector set and each cluster document in the preset knowledge graph, the category average keyword vector and the category document average keyword vector of each cluster in the preset knowledge graph are calculated. Specifically, each cluster document in the preset knowledge graph is input into the trained pre-trained word vector set for calculation to obtain the category average keyword vector and the category document average keyword vector of each cluster in the preset knowledge graph; Based on the category label keyword vector and text keyword vector of each text to be clustered, and the category average keyword vector and category document average keyword vector of each cluster, the similarity value between each text to be clustered and each cluster is calculated, and each text to be clustered is clustered.

2. The text clustering method according to claim 1, characterized in that The step of clustering the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster comprises: Multiplying the category label keyword vector of each of the to-be-clustered texts by the category average keyword vector of each of the clusters to obtain a first calculation result; Multiplying the text keyword vector of each of the to-be-clustered texts by the average keyword vector of the category documents of each of the clusters to obtain a second calculation result; Calculating the similarity between each of the to-be-clustered texts and each of the clusters based on each of the first calculation results and the corresponding category weight value, and based on each of the second calculation results and the corresponding text weight value; The texts to be clustered are clustered according to the similarity values ​​and a preset similarity threshold.

3. The text clustering method according to claim 2, characterized in that The step of clustering the texts to be clustered according to the similarity values ​​and the preset similarity threshold comprises: Determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values; If the target similarity value exists, the target similarity values ​​of the same text to be clustered are classified to obtain sets of target similarity values; The to-be-clustered texts corresponding to each of the target similarity value sets are clustered with the cluster corresponding to the target similarity value with the largest value in each of the target similarity value sets.

4. The text clustering method according to claim 3, characterized in that After the step of determining whether there is a target similarity value greater than or equal to the preset similarity threshold among the similarity values, the method further includes: If the target similarity value does not exist, then creating new clusters for each text to be clustered in the preset knowledge graph; Each text to be clustered is clustered with its corresponding newly added cluster, and each clustered text after clustering is added to the preset knowledge graph.

5. The text clustering method according to claim 1, characterized in that The step of calculating the category label keyword vector of each text to be clustered based on the pre-trained word vector set and the category label keyword of each text to be clustered includes: Segmenting each of the texts to be clustered according to the category label keywords to obtain each category label keyword of each of the texts to be clustered; Calculate the word vector of each category label keyword of each of the texts to be clustered according to the pre-trained word vector set; The word vectors of each category label keyword of each of the texts to be clustered are summed and averaged to obtain the category label keyword vectors of each of the texts to be clustered.

6. The text clustering method according to claim 1, characterized in that The step of calculating the text keyword vector of each text to be clustered based on the pre-trained word vector set and the text keywords of each text to be clustered includes: Performing word segmentation on each of the texts to be clustered and extracting text keywords to obtain each text keyword of each of the texts to be clustered; Calculate the word vector of each text keyword of each of the texts to be clustered according to the pre-trained word vector set; The word vectors of each text keyword of each of the texts to be clustered are summed and averaged to obtain the text keyword vectors of each of the texts to be clustered.

7. The text clustering method according to any one of claims 1 to 6, characterized in that: After the step of clustering the texts to be clustered according to the similarity values ​​between the category label keyword vector and the text keyword vector of each text to be clustered and the category average keyword vector and the category document average keyword vector of each cluster, the method further includes: Determine whether the number of clustered texts in a preset knowledge graph is greater than or equal to a preset value; If the number is greater than or equal to the preset value, word vector training is performed using new text corpus to obtain a new set of pre-trained word vectors; Each text to be clustered is clustered using the new pre-trained word vector set.

8. A text clustering device, characterized in that: include: A first calculation module is used to calculate the category label keyword vector and the text keyword vector of each text to be clustered according to the pre-trained word vector set and the category label keyword and text keyword of each text to be clustered; A second calculation module is used to calculate the category average keyword vector and the category document average keyword vector of each cluster in the preset knowledge graph based on the pre-trained word vector set and each cluster document in the preset knowledge graph. Specifically, each cluster document in the preset knowledge graph is input into the trained pre-trained word vector set for calculation to obtain the category average keyword vector and the category document average keyword vector of each cluster in the preset knowledge graph; The clustering module is used to calculate the similarity value between each of the texts to be clustered and each of the clusters based on the category label keyword vector and text keyword vector of each of the texts to be clustered, the category average keyword vector and the category document average keyword vector of each of the clusters, and cluster each of the texts to be clustered.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the text clustering method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the text clustering method according to any one of claims 1 to 7 are implemented.

Images