A text clustering method, device, terminal equipment and storage medium

Abstract

The application is suitable for the technical field of data processing, and provides a text clustering method and device, terminal equipment and computer readable storage medium, the method comprises: obtaining a text to be clustered; determining a one-hot encoding corresponding to each word in the text to be clustered; determining a paragraph vector corresponding to the text to be clustered according to the text to be clustered and the one-hot encoding corresponding to each word; and performing clustering processing on the text to be clustered according to the paragraph vector to obtain a clustering result of the text to be clustered. Compared with the prior art of clustering each text according to a keyword in each text, the method makes the paragraph vectors corresponding to different texts with the same word closer to each other according to the one-hot encoding of each word and the paragraph vectors corresponding to different texts with the same word determined according to the one-hot encoding of each word and different texts, thereby improving the accuracy and clustering effect of text clustering.

Description

Technical Field

[0001] This application belongs to the field of data processing technology, and in particular relates to a text clustering method, apparatus, terminal device and computer-readable storage medium. Background Technology

[0002] Text clustering is an unsupervised learning process that involves grouping texts into classes based on their words (or word codes) to minimize inter-cluster similarity and maximize intra-cluster similarity, thereby achieving text classification.

[0003] Traditional text clustering methods typically extract features by using keywords as representative features, that is, by mining the words contained in the text to discover semantically meaningful words. After word mining, word clustering is performed, firstly grouping texts with the same words together. In other words, texts within the same cluster share the same topic, while texts in different clusters have significantly different topics, thus achieving the effect of text clustering. In other words, the above methods simply cluster by analyzing words in the text, resulting in low accuracy and effectiveness in text clustering. Summary of the Invention

[0004] This application provides a text clustering method, apparatus, terminal device, and computer-readable storage medium, which can improve the accuracy and clustering effect of text clustering.

[0005] In a first aspect, embodiments of this application provide a text clustering method, including:

[0006] Obtain the text to be clustered;

[0007] Determine the one-hot encoding corresponding to each word in the text to be clustered;

[0008] Based on the text to be clustered and the one-hot encoding corresponding to each word, determine the paragraph vector corresponding to the text to be clustered;

[0009] The text to be clustered is clustered based on the paragraph vector to obtain the clustering result of the text to be clustered.

[0010] Optionally, determining the one-hot encoding corresponding to each word in the text to be clustered includes:

[0011] Perform semantic analysis on each word in the text to be clustered to obtain the meaning of each word;

[0012] The semantic feature value of each word is determined based on the meaning of each word; wherein, the semantic feature values ​​of the first category of words that are synonyms are the same, and the semantic feature values ​​of the second category of words that are antonyms are different;

[0013] The one-hot encoding corresponding to each word is determined based on the semantic feature value.

[0014] Optionally, determining the one-hot encoding corresponding to each word based on the semantic feature value includes:

[0015] If the semantic feature value is the first feature value, then the word corresponding to the semantic feature value is determined to be a first type of word. When the first Euclidean distance between each word in the first type of word is determined to be less than the first threshold, the one-hot encoding corresponding to each word in the first type of word is determined according to the first Euclidean distance. The first feature value refers to the semantic feature value of words that have synonyms in the text to be clustered.

[0016] If the semantic feature value is the second feature value, then the word corresponding to the semantic feature value is determined to be a second type of word. When the second Euclidean distance between each word in the second type of word is determined to be greater than the second threshold, the one-hot encoding corresponding to each word in the second type of word is determined according to the second Euclidean distance. The second feature value refers to the semantic feature value of words that have antonyms in the text to be clustered. The second threshold is greater than the first threshold.

[0017] If the semantic feature value is the third feature value, then the word corresponding to the semantic feature value is determined to be a third type of word. When the third Euclidean distance between each word in the third type of word is greater than or equal to the first threshold and less than or equal to the second threshold, the one-hot encoding corresponding to each word in the third type of word is determined according to the third Euclidean distance. The third feature value refers to the semantic feature value of words that do not have antonyms and synonyms in the text to be clustered. The third type of word refers to words that do not have synonyms and antonyms in the text to be clustered.

[0018] Optionally, determining the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word includes:

[0019] The text to be clustered and the one-hot encoding corresponding to each word are input into the trained target paragraph vector model for processing to obtain the paragraph vector.

[0020] Optionally, determining the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word includes:

[0021] Obtain the initial word vector corresponding to each word in the text to be clustered; wherein, the initial word vector corresponding to each word is different;

[0022] The paragraph vector is determined based on the one-hot encoding corresponding to each word and the initial word vector;

[0023] Accordingly, determining the text to be clustered based on the one-hot encoding of each word and the initial word vector includes:

[0024] Each word's one-hot encoding is converted into a vector form to obtain the identifier vector for each word.

[0025] The initial word vector and the identifier vector corresponding to each word are summed to obtain the target word vector corresponding to each word.

[0026] The paragraph vector is determined based on the target word vector corresponding to each word.

[0027] Optionally, the texts to be clustered include multiple texts, each text corresponding to a paragraph vector; the step of clustering the texts to be clustered based on the paragraph vectors to obtain the clustering results includes:

[0028] Select multiple initial cluster centers from the paragraph vector;

[0029] For each remaining paragraph vector other than the initial cluster center, calculate the initial distance between the remaining paragraph vector and the multiple initial cluster centers, and cluster the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances;

[0030] Calculate the target cluster center for each cluster based on at least one segment vector contained in each cluster;

[0031] The clustering result is determined based on each of the target cluster centers.

[0032] Optionally, determining the clustering result based on each of the target cluster centers includes:

[0033] Accumulate the number of times the target cluster center of each cluster is calculated;

[0034] If the number of calculations exceeds the third threshold, clustering is stopped, and the cluster corresponding to each target cluster center is determined as the clustering result.

[0035] If the number of calculations is less than or equal to the third threshold, then the initial cluster center corresponding to each cluster is updated to the target cluster center corresponding to each cluster, and the steps of calculating the initial distance between each remaining paragraph vector and multiple initial cluster centers for each remaining paragraph vector other than the initial cluster center, and clustering the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances, and subsequent steps are returned.

[0036] Secondly, embodiments of this application provide a text clustering apparatus, including:

[0037] The first acquisition unit is used to acquire the text to be clustered.

[0038] The first encoding determination unit is used to determine the one-hot encoding corresponding to each word in the text to be clustered;

[0039] The first vector determination unit is used to determine the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word.

[0040] The first processing unit is used to perform clustering processing on the text to be clustered based on the paragraph vector to obtain the clustering result of the text to be clustered.

[0041] Thirdly, embodiments of this application provide a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the text clustering method as described in any one of the first aspects above.

[0042] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the text clustering method as described in any one of the first aspects above.

[0043] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, enables the terminal device to execute the text clustering method described in any one of the first aspects.

[0044] The beneficial effects of the embodiments in this application compared with the prior art are:

[0045] This application provides a text clustering method that involves: acquiring the text to be clustered; determining the one-hot encoding of each word in the text; determining the paragraph vector corresponding to the text based on the text and the one-hot encoding of each word; and performing clustering processing on the text based on the paragraph vectors to obtain the clustering result. Compared with existing technologies that cluster texts solely based on keywords, this method, because words with the same one-hot encoding have similar one-hot encodings, makes the paragraph vectors corresponding to different texts with the same word more similar. Furthermore, the paragraph vectors accurately express the meaning of their corresponding texts to be clustered, thereby improving the accuracy and effectiveness of text clustering. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the implementation of a text clustering method provided in an embodiment of this application;

[0048] Figure 2 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0049] Figure 3 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0050] Figure 4 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0051] Figure 5 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0052] Figure 6 This is a schematic diagram illustrating the determination of paragraph vectors provided in an embodiment of this application;

[0053] Figure 7 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0054] Figure 8 This is a flowchart illustrating the implementation of a text clustering method provided in another embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the structure of a text clustering device provided in an embodiment of this application;

[0056] Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0057] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0058] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0059] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0060] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0061] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0062] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0063] In practical applications, a paragraph vector, also known as a document vector, can be considered as the main idea of ​​a text to be clustered. Therefore, in the text clustering methods provided in all embodiments of this application, the paragraph vectors obtained based on the text to be clustered and the one-hot encoding of each word in the text can more accurately express the meaning of the corresponding text to be clustered, thereby improving the accuracy and effect of text clustering. Simultaneously, the text clustering methods provided in all embodiments of this application separate the vector representation of the text to be clustered from the clustering process, greatly reducing the number of parameters during clustering and avoiding omissions or errors due to excessive parameters, thereby improving the accuracy of calculating the text correlation between the texts to be clustered.

[0064] Please see Figure 1 , Figure 1 This is a flowchart illustrating the implementation of a text clustering method according to an embodiment of this application. In this embodiment, the execution subject of the text clustering method is a terminal device. The terminal device can be a smartphone, laptop, or computer, etc.

[0065] like Figure 1 As shown, a text clustering method provided in one embodiment of this application may include S101 to S104, which are described in detail below:

[0066] In S101, obtain the text to be clustered.

[0067] It should be noted that the text to be clustered includes multiple texts.

[0068] In practical applications, when a user needs to perform clustering processing on the text to be clustered, the user can send a text clustering request to the terminal device.

[0069] In this embodiment, the terminal device receiving a text clustering request from a user can be achieved by detecting that the user has triggered a preset operation on the terminal device's display interface. The preset operation can be determined according to actual needs and is not limited here. For example, the preset operation could be clicking a preset control; that is, if the terminal device detects that the user has clicked its own preset control, it considers that a preset operation has been triggered, i.e., it considers that the user has sent a text clustering request.

[0070] In one embodiment of this application, the terminal device can obtain the text to be clustered in the following manner, as detailed below:

[0071] Get the text to be processed;

[0072] The text to be processed is disambiguated based on a pre-built knowledge base to obtain the text to be clustered.

[0073] It should be noted that the text to be processed includes multiple texts.

[0074] In one implementation of this embodiment, the terminal device can obtain the text to be processed in real time from a server connected to it via wireless / wired communication. The server can be a cloud server, a desktop computer, a computer, or other similar device.

[0075] In another implementation of this embodiment, the terminal device can pre-acquire the text to be processed and store it in its own memory. When it detects that it needs to acquire the text to be processed, it retrieves the text to be processed from its own memory.

[0076] In this embodiment, since the text to be processed usually includes common high-frequency words, stop words, punctuation marks, emoticons, and a mixture of simplified and traditional Chinese characters, in order to avoid the influence of the above words on the subsequently obtained paragraph vectors and reduce the accuracy of the paragraph vectors, the terminal device needs to perform disambiguation processing on the text to be processed to obtain the text to be clustered.

[0077] Specifically, the terminal device can perform disambiguation processing on the text to be processed based on a pre-built knowledge base. This pre-built knowledge base can be CNKI or the English lexicon WordNet.

[0078] In S102, the one-hot encoding corresponding to each word in the text to be clustered is determined.

[0079] In practical applications, one-hot encoding, also known as one-bit valid encoding, uses an N-bit state register to encode N states. Each state has its own independent register bit, and at any given time, only one bit is valid.

[0080] It is understandable that each word in the text to be clustered has its own one-hot code, and the one-hot codes of the same word are the same, while the one-hot codes of different words are different.

[0081] In one embodiment of this application, the terminal device can randomly determine the one-hot encoding corresponding to each word in the text to be clustered.

[0082] Specifically, in this embodiment, the terminal device can determine the one-hot encoding corresponding to each word in the text to be clustered based on a random function. The random function can be rand().

[0083] In another embodiment of this application, the terminal device can specifically be implemented through, as shown in the example below. Figure 2 S201 to S203, as shown, determine the one-hot encoding corresponding to each word in the text to be clustered, as detailed below:

[0084] In S201, semantic analysis is performed on each word in the text to be clustered to obtain the semantic meaning of each word.

[0085] In this embodiment, the terminal device can import the text to be clustered into a trained neural network model for word meaning analysis, thereby determining the meaning of each word in the text to be clustered. The trained neural network model can be an existing semantic analysis model, such as a Deep Structured Semantic Model (DSSM).

[0086] In S202, the semantic feature value of each word is determined according to the meaning of each word; wherein, the semantic feature values ​​of the first category of words that are synonyms are the same, and the semantic feature values ​​of the second category of words that are antonyms are different.

[0087] In this embodiment, the semantic feature value consists of a semantic identifier and a numerical feature value. The semantic identifier refers to the attribute of each word in the text to be clustered, and this semantic identifier can be represented by a letter. For example, assuming a word is "SUV," and its attribute is car type, the terminal device can set the semantic identifier corresponding to car type to A; therefore, the semantic identifier of "SUV" is A.

[0088] Among them, the numerical feature values ​​in the semantic feature values ​​can be represented by the numbers 1, 2, 3, etc.

[0089] In this embodiment, the terminal device can determine the semantic feature value containing the number 1 as the first feature value, the semantic feature value containing the number 2 as the second feature value, and the semantic feature value containing the number 3 as the third feature value. Here, the first feature value refers to the semantic feature value of words that have synonyms in the text to be clustered, the second feature value refers to the semantic feature value of words that have antonyms in the text to be clustered, and the third feature value refers to the semantic feature value of words that do not have antonyms or synonyms in the text to be clustered.

[0090] It should be noted that the semantic feature values ​​of the first category of words that are synonyms are the same, while the semantic feature values ​​of the second category of words that are antonyms are different.

[0091] For example, suppose the semantic feature value of “SUV” is A1. Since this semantic feature value contains the number 1, it means that “SUV” has a synonym in the text to be clustered.

[0092] Based on this, taking the automotive field as an example, suppose the text to be clustered is feedback information from car buyers. For example, the text to be clustered contains words such as "SUV", "Haval", and "mythical beast". Since these words are all used to describe car types, they are synonyms and belong to the first category of words. Moreover, the semantic feature value of "SUV" is A1. Therefore, the terminal device can determine that the semantic feature values ​​of words such as "Haval" and "mythical beast" are also A1.

[0093] For example, suppose the text to be clustered contains words such as "like", "love", and "dislike", and the semantic feature value of "like" is S1. Since "like" and "love" are synonyms and "like" and "dislike" are antonyms, the terminal device can determine that the semantic feature value of "love" is also S1 and the semantic feature value of "dislike" is S2.

[0094] In S203, the one-hot code corresponding to each word is determined based on the semantic feature value.

[0095] In this embodiment, the terminal device needs to determine the number of bits in the one-hot encoding of each word based on the number of words in the text to be clustered. For example, assuming there are 200 words in the text to be clustered, the one-hot encoding of each word is 200 bits.

[0096] After determining the number of bits for the one-hot encoding, the terminal device can determine the one-hot encoding corresponding to each word based on the semantic feature value of each word in the text to be clustered.

[0097] In one embodiment of this application, in order to ensure that the vector values ​​of first-category words that are synonyms are more similar, and the vector values ​​of second-category words that are antonyms differ more, the terminal device can specifically achieve this by, as follows: Figure 3S301 to S303, as shown, determine the one-hot encoding corresponding to each word in the text to be clustered, as detailed below:

[0098] In S301, if the semantic feature value is the first feature value, then the word corresponding to the semantic feature value is determined to be a first type of word, and when the first Euclidean distance between each word in the first type of word is determined to be less than the first threshold, the one-hot encoding corresponding to each word in the first type of word is determined according to the first Euclidean distance; the first feature value refers to the semantic feature value of words that have synonyms in the text to be clustered.

[0099] In this embodiment, in conjunction with S202, for any word in the text to be clustered, when the terminal device detects that the word's semantic feature value is the first feature value, it indicates that the word has a synonym. Therefore, the terminal device can identify the word, as well as other words with the same semantic feature value as the word, as the first category of words. Here, "with the same semantic feature value as the word" means that the semantic identifier and numerical feature value in the semantic feature value of the other words are the same as those in the semantic identifier and numerical feature value of the word.

[0100] Based on this, the terminal device can determine that the first Euclidean distance between each word in the first category of word meanings is less than a first threshold. The first threshold can be set according to actual needs and is not restricted here.

[0101] In one implementation of this embodiment, since the number of bits in one-hot encoding is determined based on the number of words in the text to be clustered, the terminal device can determine a first threshold based on the number of words in the first category of words in order to minimize the first Euclidean distance between synonyms. For example, assuming the number of words in the first category is 10, the first threshold can be determined to be 10.

[0102] In S302, if the semantic feature value is the second feature value, then the word corresponding to the semantic feature value is determined to be a second type of word, and when the second Euclidean distance between each word in the second type of word is determined to be greater than the second threshold, the one-hot encoding corresponding to each word in the second type of word is determined according to the second Euclidean distance; the second feature value refers to the semantic feature value of words that have antonyms in the text to be clustered; the second threshold is greater than the first threshold.

[0103] In this embodiment, in conjunction with S202, for any word in the text to be clustered, when the terminal device detects that the semantic feature value of the word is the second feature value, it indicates that the word has an antonym. Therefore, the terminal device can identify the word, as well as other words whose semantic feature values ​​differ from the word's, as words in the second category. Here, "different from the word's semantic feature value" means that the semantic identifier in the semantic feature values ​​of other words is the same as that of the word, but the numerical feature values ​​in the semantic feature values ​​of those other words are different from those of the word.

[0104] Based on this, the terminal device can determine that the second Euclidean distance between each word in the second category of word meanings is greater than the second threshold. The second threshold can be set according to actual needs and is not restricted here.

[0105] It should be noted that the second threshold is greater than the first threshold.

[0106] In one implementation of this embodiment, since the number of bits in one-hot encoding is determined based on the number of words in the text to be clustered, in order to maximize the second Euclidean distance between antonyms, the terminal device can determine the second threshold based on the number of words in the second category of words and the number of words in the text to be clustered.

[0107] Specifically, the terminal device can determine the second threshold based on the difference between the number of words in the text to be clustered and the number of words in the second category of words mentioned above. For example, assuming that the number of words in the second category of words mentioned above is 10 and the number of words in the text to be clustered is 200, then the second threshold can be determined as 200-10=190.

[0108] In S303, if the semantic feature value is the third feature value, then the word corresponding to the semantic feature value is determined to be a third type of word. When the third Euclidean distance between each word in the third type of word is determined to be greater than or equal to the first threshold and less than or equal to the second threshold, the one-hot encoding corresponding to each word in the third type of word is determined according to the third Euclidean distance. The third feature value refers to the semantic feature value of words that do not have antonyms and synonyms in the text to be clustered. The third type of word refers to words that do not have synonyms and antonyms in the text to be clustered.

[0109] In this embodiment, in conjunction with S202, for any word in the text to be clustered, when the terminal device detects that the word's semantic feature value is the third feature value, it indicates that the word has neither antonyms nor synonyms. Therefore, the terminal device can identify the word, as well as other words whose semantic feature values ​​are completely different from the word's, as third-category words. Here, "completely different from the word's semantic feature value" means that the semantic identifiers in the semantic feature values ​​of other words are different from those in the word's semantic feature value, and the numerical feature values ​​in the semantic feature values ​​of other words are also different from those in the word's semantic feature value.

[0110] Based on this, the terminal device can determine that the third Euclidean distance between each word in the third category of word meanings is greater than or equal to the first threshold and less than or equal to the second threshold.

[0111] In S103, the paragraph vector corresponding to the text to be clustered is determined based on the text to be clustered and the one-hot encoding corresponding to each word.

[0112] In one embodiment of this application, the terminal device can specifically be configured as follows: Figure 4 S401 to S402, as shown, determine the paragraph vectors corresponding to the text to be clustered, as detailed below:

[0113] In S401, the initial word vector corresponding to each word in the text to be clustered is obtained; wherein, the initial word vector corresponding to each word is different.

[0114] In this embodiment, the initial word vector corresponding to each word in the text to be clustered can be obtained according to the existing word vector (word2vec) model. The word vectors of each word in the text to be clustered are all different; that is, the word vectors of the same word are also different.

[0115] In S402, the paragraph vector is determined based on the one-hot encoding corresponding to each word and the initial word vector.

[0116] In some possible embodiments, the terminal device specifically uses, for example... Figure 5 S4021 to S4023, as shown, determine the segment vector, as detailed below:

[0117] In S4021, the one-hot encoding corresponding to each word is converted into vector form to obtain the identifier vector corresponding to each word.

[0118] In this embodiment, in order to improve the working efficiency of the terminal device and keep the one-hot encoding of each word in the same format as the word vector, the terminal device can convert the one-hot encoding of each word into vector form to obtain the identifier vector corresponding to each word.

[0119] In S4022, for each word, the initial word vector and the identifier vector corresponding to each word are summed to obtain the target word vector corresponding to each word.

[0120] In this embodiment, in order to improve the accuracy of the paragraph vectors of the determined text to be clustered, the terminal device can perform a summation operation on the initial word vector and the identifier vector corresponding to each word to obtain the target word vector corresponding to each word.

[0121] As can be seen from the above, in this embodiment, the target word vector of each word is based on the initial word vector of the word, with the addition of the fixed identifier vector of the word. In other words, the target word vector of the word is no longer simply the initial word vector determined by the context of the word, but also needs to be determined based on the initial word vector of the word and the fixed identifier vector of the word itself, so that the vector values ​​of the target word vectors of the same words are closer, which can further improve the subsequent clustering accuracy.

[0122] In S4023, the paragraph vector is determined based on the target word vector corresponding to each word.

[0123] In this embodiment, after obtaining the target word vector corresponding to each word in the text to be clustered, the terminal device can calculate the weight value of each word in the text to be clustered based on the term frequency–inverse document frequency (TF-IDF) algorithm, and obtain the paragraph vector of the text to be clustered based on the target word vector of each word and the weight value.

[0124] In one embodiment of this application, the paragraph vector of the text to be clustered can be specifically calculated according to the following formula:

[0125]

[0126] Where PV represents the paragraph vector of the text to be clustered, and ω j W represents the weight value of the j-th word in the text to be clustered. j Let represent the target word vector of the j-th word in the text to be clustered, n represent the number of words in the text to be clustered, and m represent the dimension of the target word vector of each word in the text to be clustered. Here, m can be 300 dimensions.

[0127] Please see Figure 6 , Figure 6 This is a schematic diagram of determining the paragraph vector provided in an embodiment of this application.

[0128] In practical applications, since the Doc2vec model can output paragraph vectors, in another embodiment of this application, the terminal device can also obtain the paragraph vectors corresponding to the text to be clustered in the following way:

[0129] The text to be clustered and the one-hot encoding of each word are input into the trained target paragraph vector model for processing to obtain the paragraph vector.

[0130] In this embodiment, the training method of the target paragraph vector model is the same as that of the paragraph vector model in the prior art. Each time, a fixed length of words is sampled from the text to be clustered, and one word is selected as the prediction word, while the others are used as input words. The word vectors corresponding to the input words and the initial paragraph vectors corresponding to the text to be clustered are used as input to the input layer. The initial paragraph vector of the text to be clustered and the word vectors of the sampled input words are accumulated to form a new vector X. This vector X is then used to predict the word vector of the predicted word within the current window. In other words, the word vectors of each word in the text to be clustered are different; even the word vectors of the same word are different.

[0131] In one implementation of this embodiment, the terminal device can determine the initial paragraph vector corresponding to the text to be clustered based on the average value of the word vectors corresponding to the output words.

[0132] In another implementation of this embodiment, the terminal device may also randomly select multiple words from the text to be clustered according to a random function, and determine the initial paragraph vector of the text to be clustered based on the average of the word vectors of the multiple words. The random function may be Rand().

[0133] It should be noted that, unlike the paragraph vector model in the existing technology, the target word vector model obtains the target word vector of each word by combining the word vector and the one-hot encoding of the word, making the vector values ​​of the same words in the training samples more similar.

[0134] In practical applications, the paragraph vector model uses paragraph vectors in the input layer as a kind of memory, recording missing content from the current context or the text theme of the text to be clustered. Each training iteration involves sliding through a small subset of words from the document, and this paragraph vector is shared across multiple training iterations of the clustered text. Therefore, the text to be clustered undergoes multiple training iterations, with each iteration's input including the paragraph vector. Thus, during each training iteration, as a subset of words from the text to be clustered is selected for training, the shared paragraph vector, as part of the input layer for each training iteration, becomes increasingly accurate in expressing the main idea. After training, the paragraph vectors corresponding to the text to be clustered are obtained.

[0135] For example, assuming there are N texts to be clustered in the corpus and M words in the dictionary, we need to learn paragraph vectors. Each paragraph vector is mapped to p dimensions, and each word is mapped to q dimensions. The model then has a total of N*p + M*q parameters. Excluding the softmax parameters, even when N is large, the number of parameters is significant, but updates during training are usually infrequent, making it quite efficient. After training, each paragraph vector is considered a feature of a particular text to be clustered.

[0136] A key advantage of paragraph vectors is that they are an unsupervised method, thus performing well even on tasks with insufficient labeled data. Not only do paragraph vectors inherit important features from word vectors, but they also contain semantic information about words. Furthermore, at least in a smaller context, paragraph vectors, like n-gram models with larger n, consider word order.

[0137] In S104, the text to be clustered is clustered according to the paragraph vector to obtain the clustering result of the text to be clustered.

[0138] In this embodiment of the application, the text to be clustered may include multiple texts, and each text to be clustered corresponds to a paragraph vector. Therefore, the terminal device can perform clustering processing on multiple texts to be clustered according to the paragraph vector corresponding to each text to be clustered and the preset clustering algorithm to obtain the clustering results of the multiple texts to be clustered.

[0139] It should be noted that the preset clustering algorithm can be k-means, Birch, or DBSCAN; there are no restrictions here.

[0140] In one embodiment of this application, taking the k-means algorithm as the preset clustering algorithm as an example, the terminal device can specifically use, as follows: Figure 7The clustering results of the text to be clustered, shown in S501 to S504, are as follows:

[0141] In S501, multiple initial cluster centers are selected from the paragraph vector.

[0142] In this embodiment, since there are multiple texts to be clustered, there are also multiple paragraph vectors.

[0143] In one implementation of this embodiment, the terminal device can randomly select multiple initial cluster centers from the paragraph vectors. The initial cluster centers are the paragraph vectors themselves.

[0144] It should be noted that in practical applications, one cluster center corresponds to one cluster, and a text to be clustered can only be in one cluster.

[0145] In S502, for each remaining paragraph vector other than the initial cluster center, the initial distance between the remaining paragraph vector and the multiple initial cluster centers is calculated, and the remaining paragraph vector is clustered into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances.

[0146] In this embodiment, after determining multiple initial cluster centers, the terminal device needs to calculate the initial distance between each remaining segment vector and the multiple initial cluster centers for each remaining segment vector other than the multiple initial cluster centers. Here, the initial distance refers to the Euclidean distance.

[0147] After calculating the initial distances between a remaining paragraph vector and multiple initial cluster centers, the terminal device needs to determine the minimum initial distance among the initial distances and cluster the text to be clustered corresponding to the remaining paragraph vector into the cluster corresponding to the initial cluster center corresponding to the minimum initial distance.

[0148] For example, suppose the distances from a certain remaining paragraph vector to multiple initial cluster centers are: distances = [5, 6, 2, 1, 3, 9]. Then the terminal device can determine that the text to be clustered corresponding to the remaining paragraph vector needs to be clustered into the cluster where the 4th initial cluster center is located.

[0149] In S503, the target cluster center of each cluster is calculated based on at least one segment vector contained in each cluster.

[0150] In this embodiment, in order to improve the clustering accuracy and clustering effect, after the terminal device clusters the text to be clustered corresponding to each of the above-mentioned multiple remaining paragraph vectors into the corresponding clusters, it needs to calculate the target cluster center of each cluster based on at least one paragraph vector contained in each cluster.

[0151] Specifically, the terminal device can recalculate the target cluster center of the cluster using the average value of at least one segment vector contained in each cluster.

[0152] For example, suppose a cluster contains 3 texts to be clustered, and the paragraph vectors corresponding to the 3 texts to be clustered are: [[1,4],[2,5],[3,6]]. Therefore, the terminal device can obtain the target cluster center of the cluster as: [(1+2+3) / 3,(4+5+6) / 3].

[0153] In S504, the clustering result is determined based on each of the target cluster centers.

[0154] In one embodiment of this application, the terminal device can specifically be configured as follows: Figure 8 The clustering results of the text to be clustered, shown in S601 to S603, are detailed below:

[0155] In S601, the number of times the target cluster center of each cluster is calculated is accumulated.

[0156] In S602, if the number of calculations exceeds the third threshold, clustering is stopped, and the cluster corresponding to each target cluster center is determined as the clustering result.

[0157] In S603, if the number of calculations is less than or equal to the third threshold, the initial cluster center corresponding to each cluster is updated to the target cluster center corresponding to each cluster, and the process of calculating the initial distance between each remaining paragraph vector and multiple initial cluster centers for each remaining paragraph vector other than the initial cluster center, and clustering the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances, and subsequent steps are returned.

[0158] In this embodiment, the terminal device needs to accumulate the number of calculations for the target cluster center of each cluster and compare the accumulated number of calculations with a third threshold. The third threshold can be determined according to actual needs and is not limited here.

[0159] When the terminal device detects that the number of calculations exceeds the third threshold, it indicates that the number of clustering operations has reached the preset requirement. Therefore, the terminal device can stop clustering and determine the cluster corresponding to each target cluster center as the clustering result of multiple texts to be clustered.

[0160] If the terminal device detects that the number of calculations is less than or equal to the third threshold, it indicates that the number of clustering operations has not reached the preset requirement. Therefore, the terminal device can update the initial cluster center corresponding to each cluster to the target cluster center corresponding to each cluster and return to the execution steps S502 to S504.

[0161] In another embodiment of this application, the terminal device may further obtain the clustering result of the text to be clustered through the following steps, detailed below:

[0162] Calculate the difference between the initial cluster center and the target cluster center for each cluster;

[0163] If the difference is less than or equal to the fourth threshold, then clustering is stopped, and the cluster corresponding to each target cluster center is determined as the clustering result;

[0164] If the difference is greater than the fourth threshold, then the initial cluster center corresponding to each cluster is updated to the target cluster center corresponding to each cluster, and the steps of calculating the initial distance between each remaining paragraph vector and multiple initial cluster centers for each remaining paragraph vector other than the initial cluster center, and clustering the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among multiple initial distances, and subsequent steps are returned.

[0165] In this embodiment, the terminal device needs the difference between the initial cluster center and the target cluster center of each cluster, and compares these multiple differences with a fourth threshold. The fourth threshold can be determined according to actual needs and is not limited here.

[0166] When the terminal device detects a difference greater than the fourth threshold among the above differences, it indicates that the clustering effect has reached the preset condition. Therefore, the terminal device can stop clustering and determine the cluster corresponding to each target cluster center as the clustering result of multiple texts to be clustered.

[0167] When the terminal device detects that there is no difference less than or equal to the fourth threshold among the above differences, it indicates that the clustering effect has not reached the preset condition. Therefore, the terminal device can update the initial cluster center corresponding to each cluster to the target cluster center corresponding to each cluster and return to the execution steps S502 to S504.

[0168] Exemplarily, assume that the multiple initial clusters are: [[1,1],[2,2],[3,3]], and the target cluster centers corresponding to each of the multiple initial cluster centers are: [[1.5,1.5],[4,4],[6,6]]. The terminal device can calculate the change distance sequence of different cluster centers as: [np.sqrt(0.5),np.sqrt(8),np.sqrt(18)] = [0.71,2.83,4.24]. The terminal device can determine the maximum value of the change distances of all cluster centers as: max([0.71,2.83,4.24]) = 4.24. If maxDistance = 5, since 4.24 < maxDistance, the terminal device can stop clustering. If maxDistance = 1, since 4.24 > maxDistance, the terminal device can update the initial cluster center corresponding to each cluster to the target cluster center corresponding to each cluster, and return to execute steps S502 - S504.

[0169] As can be seen from the above, a text clustering method provided by an embodiment of the present application includes obtaining the text to be clustered; determining the one - hot encoding corresponding to each word in the text to be clustered; determining the paragraph vector corresponding to the text to be clustered according to the text to be clustered and the one - hot encoding corresponding to each word; and clustering the text to be clustered according to the paragraph vector to obtain the clustering result of the text to be clustered. Compared with the prior art in which only the keywords in each text are used to cluster each text, by using this method, since the one - hot encoding of the same word is the same, the paragraph vectors corresponding to different texts with the same word determined according to the one - hot encoding of each word and different texts are closer. At the same time, the paragraph vector can accurately express the meaning of the corresponding text to be clustered, thereby improving the accuracy and clustering effect of text clustering.

[0170] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0171] Corresponding to the text clustering method described in the above embodiments, Figure 9 The structural block diagram of a text clustering device provided by an embodiment of the present application is shown. For the sake of convenience of description, only the parts related to the embodiments of the present application are shown. Referring to Figure 9 The text clustering device 900 includes: a first acquisition unit 91, a first encoding determination unit 92, a first vector determination unit 93, and a first processing unit 94. Among them:

[0172] The first acquisition unit 91 is used to acquire the text to be clustered.

[0173] The first encoding determination unit 92 is used to determine the one-hot encoding corresponding to each word in the text to be clustered.

[0174] The first vector determination unit 93 is used to determine the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word.

[0175] The first processing unit 94 is used to perform clustering processing on the text to be clustered according to the paragraph vector, and obtain the clustering result of the text to be clustered.

[0176] In one embodiment of this application, the first encoding determination unit 92 specifically includes: an analysis unit, a feature value determination unit, and a second encoding determination unit. Wherein:

[0177] The analysis unit is used to perform semantic analysis on each word in the text to be clustered, and to obtain the semantic meaning of each word.

[0178] The feature value determination unit is used to determine the semantic feature value of each word based on the meaning of each word; wherein, the semantic feature values ​​of the first category of words that are synonyms are the same, and the semantic feature values ​​of the second category of words that are antonyms are different.

[0179] The second processing unit is used to determine the one-hot encoding corresponding to each word based on the semantic feature value.

[0180] In one embodiment of this application, the second encoding determination unit specifically includes: a first encoding determination subunit, a second encoding determination subunit, and a third encoding determination subunit. Wherein:

[0181] The first encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a first type of word if the word meaning feature value is a first feature value, and to determine the one-hot encoding corresponding to each word in the first type of word according to the first Euclidean distance when the first Euclidean distance between each word in the first type of word is less than a first threshold; the first feature value refers to the word meaning feature value of words that have synonyms in the text to be clustered.

[0182] The second encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a second type of word if the word meaning feature value is a second feature value, and to determine the one-hot encoding of each word in the second type of word according to the second Euclidean distance when the second Euclidean distance between each word in the second type of word is greater than the second threshold; the second feature value refers to the word meaning feature value of words that have antonyms in the text to be clustered; the second threshold is greater than the first threshold.

[0183] The third encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a third type of word if the word meaning feature value is the third feature value, and to determine the one-hot encoding of each word in the third type of word according to the third Euclidean distance when the third Euclidean distance between each word in the third type of word is greater than or equal to the first threshold and less than or equal to the second threshold; the third feature value refers to the word meaning feature value of words that do not have antonyms and synonyms in the text to be clustered; the third type of word refers to words that do not have synonyms and antonyms in the text to be clustered.

[0184] In one embodiment of this application, the first vector determination unit 93 specifically includes: a third processing unit.

[0185] The third processing unit is used to input the text to be clustered and the one-hot encoding corresponding to each word into the trained target paragraph vector model for processing, so as to obtain the paragraph vector.

[0186] In one embodiment of this application, the first vector determination unit 93 specifically includes: a second acquisition unit and a second vector determination unit. Wherein:

[0187] The second acquisition unit is used to acquire the initial word vector corresponding to each word in the text to be clustered; wherein, the initial word vector corresponding to each word is different.

[0188] The second vector determination unit is used to determine the paragraph vector based on the one-hot encoding corresponding to each word and the initial word vector.

[0189] Accordingly, the second vector determination unit specifically includes: a transformation unit, a summation unit, and a third vector determination unit. Wherein:

[0190] The conversion unit is used to convert the one-hot encoding corresponding to each word into vector form to obtain the identifier vector corresponding to each word.

[0191] The summation unit is used to perform a summation operation on the initial word vector and the identifier vector corresponding to each word for each word, so as to obtain the target word vector corresponding to each word.

[0192] The third vector determination unit is used to determine the paragraph vector based on the target word vector corresponding to each word.

[0193] In one embodiment of this application, the texts to be clustered include multiple texts, each text corresponding to a paragraph vector; the first processing unit 94 specifically includes: a selection unit, a first calculation unit, a second calculation unit, and a clustering result determination unit. Wherein:

[0194] The selection unit is used to select multiple initial cluster centers from the paragraph vector.

[0195] The first calculation unit is used to calculate the initial distance between each remaining paragraph vector and the plurality of initial cluster centers for each remaining paragraph vector other than the initial cluster centers, and to cluster the remaining paragraph vectors into the cluster corresponding to the initial cluster center corresponding to the smallest initial distance among the plurality of initial distances.

[0196] The second calculation unit is used to calculate the target cluster center of each cluster based on at least one segment vector contained in each cluster.

[0197] The clustering result determination unit is used to determine the clustering result based on each of the target cluster centers.

[0198] In one embodiment of this application, the clustering result determination unit specifically includes: an accumulation unit, a stopping unit, and a return unit. Wherein:

[0199] The accumulation unit is used to accumulate the number of times the target cluster center of each cluster is calculated.

[0200] The stopping unit is used to stop clustering if the number of calculations exceeds a third threshold, and to determine the cluster corresponding to each target cluster center as the clustering result.

[0201] The return unit is used to update the initial cluster center corresponding to each cluster to the target cluster center corresponding to each cluster if the number of calculations is less than or equal to the third threshold, and return to the steps of calculating the initial distance between each remaining paragraph vector and multiple initial cluster centers for each remaining paragraph vector other than the initial cluster center, and clustering the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances, as well as subsequent steps.

[0202] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0203] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0204] Figure 10 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 10 As shown, the terminal device 1 of this embodiment includes: at least one processor 10 ( Figure 10 (Only one is shown in the diagram) a processor, a memory 11, and a computer program 12 stored in the memory 11 and executable on the at least one processor 10, wherein the processor 10 executes the computer program 12 to implement the steps in any of the above-described text clustering method embodiments.

[0205] The terminal device may include, but is not limited to, a processor 10 and a memory 11. Those skilled in the art will understand that... Figure 10 This is merely an example of terminal device 1 and does not constitute a limitation on terminal device 1. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0206] The processor 10 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0207] In some embodiments, the memory 11 may be an internal storage unit of the terminal device 1, such as the RAM of the terminal device 1. In other embodiments, the memory 11 may be an external storage device of the terminal device 1, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 1. Furthermore, the memory 11 may include both internal and external storage units of the terminal device 1. The memory 11 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 11 can also be used to temporarily store data that has been output or will be output.

[0208] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0209] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0210] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0211] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0212] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A text clustering method, characterized in that, include: Obtain the text to be clustered; Determine the one-hot encoding corresponding to each word in the text to be clustered; Based on the text to be clustered and the one-hot encoding corresponding to each word, determine the paragraph vector corresponding to the text to be clustered; The text to be clustered is clustered based on the paragraph vector to obtain the clustering result of the text to be clustered. Determining the one-hot encoding corresponding to each word in the text to be clustered includes: Perform semantic analysis on each word in the text to be clustered to obtain the meaning of each word; The semantic feature value of each word is determined based on the meaning of each word; wherein, the semantic feature values ​​of the first category of words that are synonyms are the same, and the semantic feature values ​​of the second category of words that are antonyms are different; Determine the one-hot encoding corresponding to each word based on the semantic feature values; The step of determining the one-hot encoding corresponding to each word based on the semantic feature value includes: If the semantic feature value is the first feature value, then the word corresponding to the semantic feature value is determined to be a first type of word. When the first Euclidean distance between each word in the first type of word is determined to be less than the first threshold, the one-hot encoding corresponding to each word in the first type of word is determined according to the first Euclidean distance. The first feature value refers to the semantic feature value of words that have synonyms in the text to be clustered. If the semantic feature value is the second feature value, then the word corresponding to the semantic feature value is determined to be a second type of word. When the second Euclidean distance between each word in the second type of word is determined to be greater than the second threshold, the one-hot encoding corresponding to each word in the second type of word is determined according to the second Euclidean distance. The second feature value refers to the semantic feature value of words that have antonyms in the text to be clustered. The second threshold is greater than the first threshold. If the semantic feature value is the third feature value, then the word corresponding to the semantic feature value is determined to be a third type of word. When the third Euclidean distance between each word in the third type of word is greater than or equal to the first threshold and less than or equal to the second threshold, the one-hot encoding corresponding to each word in the third type of word is determined according to the third Euclidean distance. The third feature value refers to the semantic feature value of words that do not have antonyms and synonyms in the text to be clustered. The third type of word refers to words that do not have synonyms and antonyms in the text to be clustered.

2. The text clustering method as described in claim 1, characterized in that, The step of determining the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word includes: The text to be clustered and the one-hot encoding corresponding to each word are input into the trained target paragraph vector model for processing to obtain the paragraph vector.

3. The text clustering method as described in claim 1, characterized in that, The step of determining the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word includes: Obtain the initial word vector corresponding to each word in the text to be clustered; wherein, the initial word vector corresponding to each word is different; The paragraph vector is determined based on the one-hot encoding corresponding to each word and the initial word vector; Accordingly, determining the text to be clustered based on the one-hot encoding of each word and the initial word vector includes: Each word's one-hot encoding is converted into a vector form to obtain the identifier vector for each word. For each word, the initial word vector and the identifier vector corresponding to each word are summed to obtain the target word vector corresponding to each word. The paragraph vector is determined based on the target word vector corresponding to each word.

4. The text clustering method according to any one of claims 1-3, characterized in that, The texts to be clustered include multiple texts, each text corresponding to a paragraph vector; the clustering process based on the paragraph vectors to obtain the clustering results includes: Select multiple initial cluster centers from the paragraph vector; For each remaining paragraph vector other than the initial cluster center, calculate the initial distance between the remaining paragraph vector and the multiple initial cluster centers, and cluster the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances; Calculate the target cluster center for each cluster based on at least one segment vector contained in each cluster; The clustering result is determined based on each of the target cluster centers.

5. The text clustering method as described in claim 4, characterized in that, Determining the clustering result based on each of the target cluster centers includes: Accumulate the number of times the target cluster center of each cluster is calculated; If the number of calculations exceeds the third threshold, clustering is stopped, and the cluster corresponding to each target cluster center is determined as the clustering result. If the number of calculations is less than or equal to the third threshold, then the initial cluster center corresponding to each cluster is updated to the target cluster center corresponding to each cluster, and the steps of calculating the initial distance between each remaining paragraph vector and multiple initial cluster centers for each remaining paragraph vector other than the initial cluster center, and clustering the remaining paragraph vector into the cluster corresponding to the initial cluster center with the smallest initial distance among the multiple initial distances, and subsequent steps are returned.

6. A text clustering device, characterized in that, include: The first acquisition unit is used to acquire the text to be clustered. The first encoding determination unit is used to determine the one-hot encoding corresponding to each word in the text to be clustered; The first vector determination unit is used to determine the paragraph vector corresponding to the text to be clustered based on the text to be clustered and the one-hot encoding corresponding to each word. The first processing unit is used to perform clustering processing on the text to be clustered according to the paragraph vector to obtain the clustering result of the text to be clustered. The first encoding determination unit specifically includes: The analysis unit is used to perform semantic analysis on each word in the text to be clustered, and to obtain the semantic meaning of each word; The feature value determination unit is used to determine the semantic feature value of each word based on the meaning of each word; wherein, the semantic feature values ​​of the first category of words that are synonyms are the same, and the semantic feature values ​​of the second category of words that are antonyms are different; The second encoding determination unit is used to determine the one-hot encoding corresponding to each word based on the semantic feature value; The second encoding determination unit specifically includes: The first encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a first type of word if the word meaning feature value is a first feature value, and to determine the one-hot encoding corresponding to each word in the first type of word according to the first Euclidean distance when the first Euclidean distance between each word in the first type of word is less than a first threshold; the first feature value refers to the word meaning feature value of words that have synonyms in the text to be clustered; The second encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a second type of word if the word meaning feature value is a second feature value, and to determine the one-hot encoding corresponding to each word in the second type of word according to the second Euclidean distance when the second Euclidean distance between each word in the second type of word is greater than the second threshold; the second feature value refers to the word meaning feature value of words that have antonyms in the text to be clustered; the second threshold is greater than the first threshold; The third encoding determination subunit is used to determine the word corresponding to the word meaning feature value as a third type of word if the word meaning feature value is the third feature value, and to determine the one-hot encoding of each word in the third type of word according to the third Euclidean distance when the third Euclidean distance between each word in the third type of word is greater than or equal to the first threshold and less than or equal to the second threshold; the third feature value refers to the word meaning feature value of words that do not have antonyms and synonyms in the text to be clustered; the third type of word refers to words that do not have synonyms and antonyms in the text to be clustered.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the text clustering method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the text clustering method as described in any one of claims 1 to 5.

Images