Text clustering method and apparatus based on artificial intelligence (AI), and electronic device

Abstract

This application relates to a text clustering method, apparatus, and electronic device based on artificial intelligence (AI). The method includes: segmenting each text in a text set to be clustered to obtain multiple keywords contained in each text; for each text, concatenating the multiple keywords according to a preset method to obtain at least one corresponding candidate word group; grouping the text set to be clustered based on the at least one candidate word group corresponding to each text to obtain multiple text sets; and performing clustering processing on the multiple text sets separately according to a preset clustering method to obtain multiple target clusters. By grouping the text set to be clustered based on multiple candidate word groups to obtain multiple text sets, and then performing clustering processing on the multiple text sets separately, the efficiency of text clustering is improved, thereby improving the efficiency of building chatbots. This application can also combine RPA and AI to achieve IA text clustering, thereby reducing the manual cost of text clustering.

Description

Technical Field

[0001] This application relates to the fields of robotic process automation and artificial intelligence, and in particular to a text clustering method, apparatus and electronic device based on artificial intelligence (AI). Background Technology

[0002] Robotic Process Automation (RPA) uses specific "robot software" to simulate human operations on a computer and automatically execute process tasks according to rules.

[0003] Artificial intelligence (AI) is a technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence.

[0004] Intelligent Automation (IA) is a collective term for a series of technologies ranging from Robotic Process Automation (RPA) to Artificial Intelligence (AI). It combines RPA with various AI technologies such as Optical Character Recognition (OCR), Intelligent Character Recognition (ICR), Process Mining, Deep Learning (DL), Machine Learning (ML), Natural Language Processing (NLP), Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Computer Vision (CV) to create end-to-end business processes that can think, learn, and adapt. It covers the entire process from process discovery and automation to managing and optimizing business processes through automatic and continuous data collection, understanding the meaning of data, and using data.

[0005] Currently, chatbots, widely used in daily life, can provide answers to users' questions according to a pre-set question-and-answer correspondence. However, in practice, different users may express the same question in different ways. To ensure that the chatbot can accurately provide the corresponding answer based on the pre-set question-and-answer correspondence regardless of how users express the same question, multiple identical or similar questions can be treated as a single question during the chatbot's development process, and a unified answer can be assigned to it. This requires text clustering of a large amount of question text, grouping identical or similar question texts into the same category. The efficiency of text clustering directly affects the efficiency of chatbot development. Summary of the Invention

[0006] This application provides a text clustering method, apparatus, and electronic device based on artificial intelligence (AI) to solve the technical problem of low clustering efficiency when clustering a large amount of question text during the construction of chatbots in related technologies.

[0007] The first aspect of this application provides a text clustering method based on artificial intelligence (AI). The method includes: performing word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text; for each text, concatenating the multiple keywords contained therein according to a preset method to obtain at least one corresponding candidate word group; grouping the text set to be clustered based on the at least one candidate word group corresponding to each text to obtain multiple text sets; and performing clustering processing on the multiple text sets respectively according to a preset clustering method to obtain multiple target clusters.

[0008] In some embodiments, the text set to be clustered is grouped based on at least one candidate word group corresponding to each text to obtain multiple text sets, including: determining the statistical value of each candidate word group under a preset statistical indicator, and filtering each candidate word group according to the statistical value to obtain multiple target candidate word groups; for each target candidate word group, the set of texts in the text set to be clustered that contains each keyword in the target candidate word group is determined as the corresponding text set.

[0009] In some embodiments, multiple text sets are clustered according to a preset clustering method to obtain multiple target clusters, including: performing initial clustering on multiple text sets to obtain a cluster set including multiple initial clusters; and using the singlepass clustering algorithm on the cluster set to perform secondary clustering on multiple text sets to obtain multiple target clusters.

[0010] In some embodiments, the cluster set includes a portion of text from multiple text sets; based on the cluster set, a single-pass clustering algorithm is used to perform secondary clustering on the multiple text sets to obtain multiple target clusters, including: for target text in the multiple text sets that is not assigned to a cluster set, determining a first similarity between the target text and the cluster centers of each initial cluster currently included in the cluster set, and obtaining the highest similarity among the first similarities; if the highest similarity is greater than a first preset threshold, assigning the target text to the initial cluster corresponding to the highest similarity; if the highest similarity is not greater than the first preset threshold, creating a new initial cluster, assigning the target text to the newly created initial cluster, and adding the newly created initial cluster to the cluster set to update the cluster set; when all texts in the multiple text sets are assigned to the cluster set, each initial cluster included in the cluster set is determined as multiple target clusters.

[0011] In some embodiments, after classifying the target text into the initial cluster corresponding to the highest similarity, the method further includes: updating the cluster center of the initial cluster corresponding to the highest similarity; determining a second similarity between each text in the initial cluster corresponding to the highest similarity and the updated cluster center; and removing texts whose second similarity with the updated cluster center is not greater than a first preset threshold from the initial cluster corresponding to the highest similarity.

[0012] In some embodiments, for each text, multiple keywords contained therein are concatenated in a preset manner to obtain at least one corresponding candidate word group, including: for each text, concatenating any two adjacent keywords among the multiple keywords contained therein in sequence to obtain at least one corresponding candidate word group.

[0013] In some embodiments, before concatenating multiple keywords contained in each text according to a preset method to obtain at least one corresponding candidate word group, the method further includes: performing named entity recognition on multiple keywords contained in each text to obtain the category to which each keyword belongs; and replacing keywords belonging to a preset category with the corresponding preset category to update the keywords.

[0014] In some embodiments, after clustering multiple text sets according to a preset clustering method to obtain multiple target clusters, the method further includes: for each target cluster, determining the third similarity between each text within it; and deleting texts in the target cluster whose third similarity with each other text is less than a second preset threshold.

[0015] In some embodiments, before performing word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text, the method further includes: calling an RPA robot to trigger an upload instruction for the text set to be clustered; and obtaining the text set to be clustered based on the upload instruction.

[0016] A second aspect of this application provides a text clustering device based on artificial intelligence (AI), comprising: a word segmentation module for segmenting each text in a text set to be clustered to obtain multiple keywords contained in each text; a concatenation module for concatenating the multiple keywords contained in each text according to a preset method to obtain at least one corresponding candidate word group; a grouping module for grouping the text set to be clustered based on the at least one candidate word group corresponding to each text to obtain multiple text sets; and a clustering module for performing clustering processing on the multiple text sets respectively according to a preset clustering method to obtain multiple target clusters.

[0017] In some embodiments, the grouping module includes: a filtering unit, configured to determine the statistical value of each candidate word group under a preset statistical indicator, and filter each candidate word group according to the statistical value to obtain multiple target candidate word groups; and a determining unit, configured to, for each target candidate word group, determine the set of texts containing each keyword in the target candidate word group in the text set to be clustered as the corresponding text set.

[0018] In some embodiments, the clustering module includes: a first clustering unit, configured to perform initial clustering on multiple text sets respectively to obtain a cluster set including multiple initial clusters; and a second clustering unit, configured to perform secondary clustering on multiple text sets based on the cluster set using the Singlepass clustering algorithm to obtain multiple target clusters.

[0019] In some embodiments, the AI-based text clustering device further includes: a recognition module for performing named entity recognition on multiple keywords contained in each text to obtain the category to which each keyword belongs; and a processing module for replacing keywords belonging to a preset category with the corresponding preset category to update the keywords.

[0020] A third aspect of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method described in the first aspect of this application described above.

[0021] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect of this application described above.

[0022] A fifth aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect of this application above.

[0023] The technical solutions provided in this application embodiment may include the following beneficial effects:

[0024] By grouping the text set to be clustered based on multiple candidate word groups, multiple text sets are obtained. Then, according to the preset clustering method, the multiple text sets are clustered separately to obtain multiple target clusters. This reduces the time consumption of text clustering, improves the efficiency of text clustering, and thus improves the efficiency of building a chatbot.

[0025] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0026] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0027] Figure 1 This is a flowchart illustrating the text clustering method based on artificial intelligence (AI) according to the first embodiment of this application;

[0028] Figure 2 This is a flowchart illustrating the text clustering method based on artificial intelligence (AI) according to the second embodiment of this application;

[0029] Figure 3 This is a flowchart illustrating the secondary clustering process of multiple text sets according to the second embodiment of this application;

[0030] Figure 4 This is a flowchart illustrating the text clustering method based on artificial intelligence (AI) according to the third embodiment of this application;

[0031] Figure 5 This is a flowchart illustrating a text clustering apparatus based on artificial intelligence (AI) according to the fourth embodiment of this application;

[0032] Figure 6 This is a block diagram of an electronic device used to implement the AI-based text clustering method of the embodiments of this application. Detailed Implementation

[0033] The embodiments of this application / disclosure are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application / disclosure, and should not be construed as limiting this application / disclosure.

[0034] These and other aspects of the embodiments of this application / disclosure will become clear from the following description and accompanying drawings. In these descriptions and drawings, some specific implementations of the embodiments of this application / disclosure are specifically disclosed to illustrate some ways of carrying out the principles of the embodiments of this application / disclosure; however, it should be understood that the scope of the embodiments of this application / disclosure is not limited thereto. Rather, the embodiments of this application / disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0035] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution of this disclosure comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0036] This application provides a text clustering method based on artificial intelligence (AI). The method includes: segmenting each text in the text set to be clustered to obtain multiple keywords contained in each text; for each text, concatenating the multiple keywords according to a preset method to obtain at least one corresponding candidate word group; grouping the text set to be clustered based on the at least one candidate word group corresponding to each text to obtain multiple text sets; and performing clustering processing on the multiple text sets separately according to a preset clustering method to obtain multiple target clusters. Therefore, by grouping the text set to be clustered based on multiple candidate word groups to obtain multiple text sets, and then performing clustering processing on the multiple text sets separately according to a preset clustering method to obtain multiple target clusters, the time consumption of text clustering is reduced, the efficiency of text clustering is improved, and thus the efficiency of building chatbots is improved.

[0037] To clearly illustrate the various embodiments of the present invention, the technical terms involved in the embodiments of the present invention will first be explained.

[0038] In the description of this application / disclosure, the term "multiple" refers to two or more.

[0039] In the description of this application, "RPA robot" refers to a software robot that can automatically perform business processing by combining AI technology and RPA technology. RPA robots have two characteristics: "connector" and "non-intrusive". By simulating human operation methods, they extract, integrate and connect data from different systems in a non-intrusive manner without changing the information system.

[0040] In the description of this application, "text set to be clustered" refers to a collection of multiple texts that need to be clustered. Each text can be a statement containing at least two keywords, or an article, etc., and this application does not impose any restrictions. Furthermore, each text can be Chinese text, English text, or text in other languages, and this application does not impose any restrictions. This application uses Chinese characters as an example for illustration in its embodiments. The multiple texts included in the text set to be clustered can be the question texts to be clustered during the chatbot setup process, i.e., the texts of questions posed by users to the chatbot.

[0041] In the description of this application, "candidate phrase" refers to a phrase composed of two or more keywords.

[0042] In the description of this application, "preset statistical indicators" refers to pre-set statistical indicators, such as word frequency, cohesion, and degrees of freedom corresponding to candidate word groups, which are not limited in this application. "Statistical value" refers to the value corresponding to the statistical indicator.

[0043] Here, cohesion represents the tightness between characters in a candidate word group. Degrees of freedom represent the degree to which a candidate word group can operate independently. Term frequency represents the frequency with which a candidate word group appears in the text set to be clustered.

[0044] In the description of this application, "similarity" is used to indicate the degree of similarity.

[0045] In the description of this application, "preset category" refers to a category that is set in advance as needed, such as a category for personal names, place names, organization names, etc.

[0046] In the description of this application, "cluster" refers to a collection of multiple texts, in which the texts within each cluster have a high degree of similarity, and the texts between different clusters have a low degree of similarity.

[0047] The following describes, in conjunction with the accompanying drawings, an AI-based text clustering method, apparatus, electronic device, and storage medium according to embodiments of this application / disclosure.

[0048] First, with reference to the accompanying drawings, the text clustering method based on artificial intelligence (AI) in the embodiments of this application will be described.

[0049] Figure 1 This is a flowchart of the text clustering method based on artificial intelligence (AI) according to the first embodiment of this application. Figure 1 As shown, the method may include the following steps:

[0050] Step 101: Perform word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text.

[0051] It should be noted that the AI-based text clustering method in this application embodiment can be executed by an AI-based text clustering device, which will be referred to as a text clustering device below. This text clustering device can be implemented by software and / or hardware. It can be an electronic device or configured within an electronic device to improve the efficiency of text clustering, thereby improving the efficiency of building a chatbot. The electronic device can include, but is not limited to, terminal devices, servers, etc., and this embodiment does not specifically limit the electronic device. The text clustering device can be a chatbot configured within a terminal device.

[0052] In an example embodiment, after obtaining an initial text set consisting of multiple texts to be clustered, the initial text set can be preprocessed to obtain the text set to be clustered. The preprocessing of the initial text set may include: removing stop characters from the initial text set; using regular expressions to determine whether the number of characters in each text in the initial text set is greater than a preset number of characters, and deleting texts with a fewer than the preset number of characters as noise data from the initial text set.

[0053] The stop characters can be characters that do not need to be clustered, such as ":" and "!". The preset number of characters can be set arbitrarily as needed. For example, if the text is Chinese, the preset number of characters can be set to 2, 3, 5, etc.

[0054] In the example embodiment, a dictionary can be pre-defined as needed to perform word segmentation on each text in the text set to be clustered, based on the customized dictionary, to obtain multiple keywords contained in each text.

[0055] In an example embodiment, word segmentation can be performed on each text in the text set to be clustered based on a word segmentation model in related technologies to obtain multiple keywords contained in each text.

[0056] Alternatively, other methods can be used to segment each text in the clustered text set to obtain multiple keywords contained in each text; this application does not impose any restrictions on this.

[0057] Step 102: For each text, the multiple keywords contained therein are concatenated according to a preset method to obtain at least one corresponding candidate word group.

[0058] The preset method is a pre-defined method for concatenating at least two keywords.

[0059] In the example embodiment, for each text, any predetermined number of adjacent keywords from among the multiple keywords contained therein can be concatenated according to their corresponding order in the text to obtain at least one candidate word group corresponding to that text. Thus, for all texts in the text set to be clustered, multiple candidate word groups can be obtained.

[0060] For example, assuming the preset quantity is 2, for a certain text "Ask why medical insurance in City A was credited separately", by performing word segmentation on the text, we can obtain multiple keywords contained in the text, including at least: "ask", "below", "ask below", "City A", "of", "City A's", "medical insurance", "for", "medical insurance for", "what", "why", "separate", "to", "separate to", "account", "credited", "credited".

[0061] By concatenating any two adjacent keywords from these keywords in the corresponding order in the text, we can obtain at least the following candidate phrases: "ask below", "ask below City A", "City A", "City A's", "medical insurance", "medical insurance for", "why", "why alone", "what alone", "what alone to", "alone to account", "to account", "to account".

[0062] Step 103: Based on at least one candidate word group corresponding to each text, the text set to be clustered is grouped to obtain multiple text sets.

[0063] Step 104: Cluster the multiple text sets according to the preset clustering method to obtain multiple target clusters.

[0064] In the example embodiment, after obtaining multiple text sets, each text set can be clustered separately according to a preset clustering method, thereby dividing the text set to be clustered into multiple target clusters.

[0065] The preset clustering method can be set as needed, such as density-based clustering or K-means clustering. This application does not impose any restrictions on this.

[0066] The time complexity of clustering multiple texts is typically on the order of the square of the number of texts, resulting in low efficiency when dealing with a large number of texts. This solution addresses this by first grouping the text set to be clustered based on multiple candidate word groups, obtaining multiple text sets, and then performing clustering processing on each text set separately. Since the number of texts in each text set is reduced, the time consumption for clustering each text set is significantly reduced, thereby reducing the time spent clustering the text sets to be clustered and improving the efficiency of text clustering. By employing the AI-based text clustering method provided in this application during the chatbot construction process, a large number of question texts can be clustered, quickly classifying multiple identical or similar questions into the same category. This allows for the setting of a unified answer for questions belonging to the same category, enabling the chatbot to accurately provide the corresponding answer to the question based on the pre-set correspondence between questions and answers, even when users ask the same question in different ways. This improves the efficiency of chatbot construction.

[0067] In summary, the AI-based text clustering method provided in this application performs word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text. For each text, the multiple keywords are concatenated according to a preset method to obtain at least one corresponding candidate word group. Based on the at least one candidate word group corresponding to each text, the text set to be clustered is grouped to obtain multiple text sets. Then, according to a preset clustering method, the multiple text sets are clustered separately to obtain multiple target clusters. Therefore, by grouping the text set to be clustered based on multiple candidate word groups to obtain multiple text sets, and then performing clustering processing on the multiple text sets separately according to a preset clustering method to obtain multiple target clusters, the time consumption of text clustering is reduced, the efficiency of text clustering is improved, and thus the efficiency of building a chatbot is improved.

[0068] The following is combined Figure 2 The present application provides a further explanation of the process of grouping the text set to be clustered into multiple text sets based on at least one candidate word group corresponding to each text in the text clustering method provided in the embodiments of this application, and the process of clustering multiple text sets according to a preset clustering method.

[0069] Figure 2 This is a flowchart of a text clustering method based on artificial intelligence (AI) according to the second embodiment of this application, as shown below. Figure 2 As shown, the method includes:

[0070] Step 201: Perform word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text.

[0071] The specific implementation process and principle of step 201 can be referred to the description of the above embodiments, and will not be repeated here.

[0072] In an example embodiment, the text clustering device can be combined with an RPA robot to obtain the text set to be clustered, thereby reducing the manual cost of text clustering.

[0073] Specifically, the text clustering device can provide an interactive interface, including a button for uploading data. The text clustering device can then invoke an RPA robot to trigger an upload command, which can carry the text set to be clustered. Based on the upload command, the text clustering device can then obtain the text set to be clustered. Correspondingly, before step 201, the process may also include: invoking an RPA robot to trigger an upload command for the text set to be clustered; and obtaining the text set to be clustered based on the upload command.

[0074] Alternatively, the upload command can carry an initial text set, allowing the text clustering device to obtain the initial text set based on the upload command, and then preprocess the initial text set to obtain the text set to be clustered. Correspondingly, before step 201, the process may also include: invoking the RPA robot to trigger the upload command for the initial text set; obtaining the initial text set based on the upload command; and preprocessing the initial text set to obtain the text set to be clustered.

[0075] Step 202: For each text, the multiple keywords contained therein are concatenated according to a preset method to obtain at least one corresponding candidate word group.

[0076] In an example embodiment, for each text, the multiple keywords contained therein are concatenated in a preset manner to obtain at least one corresponding candidate word group. This may include: for each text, concatenating any two adjacent keywords from the multiple keywords contained therein in sequence to obtain at least one corresponding candidate word group.

[0077] For each text, any two adjacent keywords from the multiple keywords it contains can be concatenated according to their corresponding order in the text to obtain at least one candidate word group for that text. Thus, for all texts in the set to be clustered, multiple candidate word groups can be obtained.

[0078] Step 203: Determine the statistical value of each candidate word group under the preset statistical indicators, and filter each candidate word group according to the statistical value to obtain multiple target candidate word groups.

[0079] The preset statistical indicators may include at least one of the following: degrees of freedom, cohesion, word frequency, etc.

[0080] In the example embodiment, taking preset statistical indicators including degrees of freedom, cohesion, and word frequency as an example, the thresholds corresponding to these three preset statistical indicators can be preset, and the statistical values ​​corresponding to the degrees of freedom, cohesion, and word frequency of each candidate word group can be determined. Then, the candidate word groups that meet the following three conditions are determined as target candidate word groups, thereby obtaining multiple target candidate word groups.

[0081] Step 204: For each target candidate word group, determine the set of texts in the text set to be clustered that contain each keyword in the target candidate word group as the corresponding text set.

[0082] Taking the candidate phrase "single arrival" formed by combining the keywords "single" and "arrival" as an example, the set of all texts in the text set to be clustered that simultaneously contain the keywords "single" and "arrival" can be determined as the text set corresponding to "single arrival".

[0083] It should be noted that if there are texts in the text set to be clustered that are not assigned to a certain target candidate word group, then such texts can be considered as noise data and will not participate in the subsequent clustering process.

[0084] Therefore, meaningful and representative target candidate word groups can be selected from multiple candidate word groups, and based on the target candidate word groups, the text set to be clustered can be grouped to obtain multiple text sets.

[0085] Step 205: Perform initial clustering on multiple text sets to obtain a cluster set including multiple initial clusters.

[0086] In this application, any clustering method from related technologies can be used to perform initial clustering on multiple text sets, and this application does not impose any restrictions on this.

[0087] In the example embodiment, taking the HDBSCAN (Hierarchical Density-based Spatial Clustering of Applications with Noise) algorithm as an example, step 205 can be implemented in the following way:

[0088] For each text set, determine the fourth similarity between the texts within it;

[0089] Based on the fourth similarity, HDBSCAN is used to perform initial clustering on the text set to obtain the corresponding initial clustering results. The initial clustering results may include one, one or more initial clusters.

[0090] The initial clustering results of multiple text sets are merged to obtain a cluster set that includes multiple initial clusters.

[0091] Step 206: Based on the cluster set, the Singlepass clustering algorithm is used to perform secondary clustering on multiple text sets to obtain multiple target clusters.

[0092] The Singlepass clustering algorithm, a simple and efficient clustering method, suffers from unstable clustering performance due to the influence of the initial clusters. In this embodiment, multiple initial clusters from the cluster set obtained in step 205 can be used as the initial clusters for the Singlepass clustering algorithm. A single-pass Singlepass clustering algorithm is then applied to perform secondary clustering on multiple text sets, thereby improving the clustering performance of the Singlepass algorithm.

[0093] In the example embodiment, the cluster set may only include a portion of the text from multiple text sets. In this embodiment, based on the target texts from multiple text sets that are not included in the cluster set and the initial clusters in the cluster set, the Singlepass clustering algorithm can be used to perform secondary clustering on the multiple text sets to obtain multiple target clusters. Accordingly, step 206 can be implemented in the following way:

[0094] For target texts in multiple text sets that are not assigned to a cluster set, determine the first similarity between the target text and the cluster centers of each initial cluster currently included in the cluster set, and obtain the highest similarity among the first similarities; if the highest similarity is greater than a first preset threshold, assign the target text to the initial cluster corresponding to the highest similarity; if the highest similarity is not greater than the first preset threshold, create a new initial cluster, assign the target text to the new initial cluster, and add the new initial cluster to the cluster set to update the cluster set; when all texts in multiple text sets are assigned to the cluster set, each initial cluster included in the cluster set is determined as multiple target clusters.

[0095] Specifically, for each target text in multiple text sets that is not assigned to a cluster set, the above process can be followed to determine the first similarity between the target text and the cluster centers of each initial cluster currently included in the cluster set, and obtain the highest similarity among the first similarities. If the highest similarity is greater than a first preset threshold, the target text is assigned to the initial cluster corresponding to the highest similarity. If the highest similarity is not greater than the first preset threshold, a new initial cluster is created, and the target text is assigned to the new initial cluster. The new initial cluster is also added to the cluster set to update the cluster set. This process continues until all texts in multiple text sets are assigned to the cluster set. At this point, each initial cluster included in the cluster set is determined as a multiple target cluster.

[0096] It should be noted that after updating the cluster set based on a target text, when determining the first similarity for subsequent target texts, the initial clusters currently included in the cluster set are the multiple initial clusters included in the updated cluster set.

[0097] The first similarity between the target text and the cluster center of an initial cluster can be obtained in the following way: the target text is vectorized to obtain the first vector corresponding to the target text; any text is randomly selected from the initial cluster and vectorized, and the resulting vector is used as the second vector corresponding to the cluster center of the initial cluster; or, all texts in the initial cluster are vectorized, and the average of the resulting vectors is used as the second vector corresponding to the cluster center of the initial cluster; the similarity between the first vector and the second vector is determined, and this similarity is used as the first similarity between the target text and the cluster center of the initial cluster.

[0098] The first preset threshold can be set arbitrarily as needed, and this application does not impose any restrictions on it.

[0099] The following is combined Figure 3 The specific implementation process of step 206 above will be explained.

[0100] refer to Figure 3The system can obtain multiple text sets from step 204 (step 301) and the initial clusters currently included in the cluster set from step 205 (step 302). For any target text in the multiple text sets that is not assigned to a cluster set, the first similarity between the target text and the cluster centers of the initial clusters currently included in the cluster set can be determined (step 303), and the highest similarity among the first similarities can be obtained (step 304). Further, it can be determined whether the highest similarity is greater than a first preset threshold (step 305). If the highest similarity is greater than the first preset threshold, the target text can be assigned to the initial cluster corresponding to the highest similarity (step 306). If the highest similarity is not greater than the first preset threshold, a new initial cluster can be created, and the target text can be assigned to the newly created initial cluster (step 307). The newly created initial cluster can also be added to the cluster set to update the cluster set (step 308). Furthermore, it can be determined whether there are still target texts in the multiple text sets that have not been assigned to a cluster set (step 309). If they exist, for the target text, steps 303-308 can be returned to be executed. When executing steps 303-308, each initial cluster in the updated cluster set is taken as the current initial clusters included in the cluster set, until there are no target texts in the multiple text sets that have not been assigned to a cluster set. Then, each initial cluster currently included in the cluster set is determined as a multiple target cluster (step 310).

[0101] In the example embodiment, reference Figure 3 After assigning the target text to the initial cluster corresponding to the highest similarity, it can also include:

[0102] Step 311: Update the cluster center of the initial cluster corresponding to the highest similarity.

[0103] Step 312: For each text in the initial cluster corresponding to the highest similarity, determine the third similarity with the updated cluster center;

[0104] Step 313: Remove texts whose third similarity with the updated cluster center is not greater than the first preset threshold from the initial cluster corresponding to the highest similarity.

[0105] Among them, the text removed from the initial cluster corresponding to the highest similarity can be used as the target text, and the above process 303-309 is continued to be performed on the target text until all texts in multiple text sets are classified into the cluster set.

[0106] By assigning the target text to the initial cluster corresponding to the highest similarity, updating the cluster center of the initial cluster corresponding to the highest similarity, and determining the third similarity between each text in the initial cluster corresponding to the highest similarity and the updated cluster center, and removing texts whose third similarity with the updated cluster center is not greater than a first preset threshold from the initial cluster corresponding to the highest similarity, dynamic adjustment of the text in the initial cluster can be achieved, so that the similarity between the texts in the adjusted initial cluster is higher, thus improving the effect of text clustering.

[0107] In summary, the AI-based text clustering method provided in this application group the text set to be clustered based on multiple candidate word groups to obtain multiple text sets. Then, it performs initial clustering on each of these text sets to obtain a cluster set containing multiple initial clusters. Based on these cluster sets, it employs the Singlepass clustering algorithm to perform secondary clustering on the multiple text sets to obtain multiple target clusters. This reduces the time consumption of text clustering, improves its efficiency, and consequently enhances the efficiency of building chatbots. Furthermore, by combining RPA robots to automatically acquire the text set to be clustered, it achieves text clustering using both RPA and AI, further reducing the manual cost of text clustering.

[0108] The following is combined Figure 4 The text clustering method based on artificial intelligence (AI) provided in the embodiments of this application will be further explained.

[0109] Figure 4 This is a flowchart of a text clustering method based on artificial intelligence (AI) according to a third embodiment of this application. Figure 4 As shown, the method includes:

[0110] Step 401: Perform word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text.

[0111] The specific implementation process and principle of step 401 can be referred to the description of the above embodiments, and will not be repeated here.

[0112] Step 402: Perform named entity recognition on multiple keywords contained in each text to obtain the category to which each keyword belongs.

[0113] Step 403: Replace the keywords belonging to the preset category with the corresponding preset category to update the keywords.

[0114] The preset categories can be set as needed, such as personal names, place names, and organization names.

[0115] It is understandable that the texts in the text set to be clustered may contain words such as personal names, place names, and organization names, which may introduce noise into the text clustering. In this embodiment, the keywords in each text that belong to a preset category can be replaced with the preset category corresponding to that keyword to update the keyword, and then the updated keyword can be used to perform subsequent steps.

[0116] For example, suppose the preset categories include personal names, place names, and organization names. For the text "Zhang San asked whether her personal income tax was paid in City A or City B," by segmenting the text, we can obtain at least the following keywords: "Zhang San," "asked," "she," "asked her," "of," "personal income tax," "is," "pay," "in," "City A," "or," and "City B." By performing named entity recognition on each keyword, we can determine that "Zhang San" belongs to the category of personal names, and "City A" and "City B" belong to the category of place names. Therefore, we can replace "Zhang San" with "[person name]", "City A" with "[place name]", and "City B" with "[place name]".

[0117] Step 404: For each text, the updated keywords contained therein are concatenated according to a preset method to obtain at least one corresponding candidate word group.

[0118] Step 405: Based on at least one candidate word group corresponding to each text, the text set to be clustered is grouped to obtain multiple text sets.

[0119] Step 406: Cluster the multiple text sets according to the preset clustering method to obtain multiple target clusters.

[0120] The specific implementation process and principle of steps 404-406 can be referred to the description of the above embodiments, and will not be repeated here.

[0121] It should be noted that for any text, after replacing the keywords belonging to the preset category with the corresponding preset category to update the keywords, steps 404-406 can be implemented using the updated keywords and the text after updating the keywords.

[0122] For example, continuing the example above, when obtaining candidate phrases, you can concatenate "[person's name]" and "ask" to get the candidate phrase "[person's name] ask".

[0123] Assuming that the candidate word group is the target candidate word group, then for the target candidate word group, the set of all texts in the text set to be clustered that contain keywords belonging to people's names (such as Zhang San, Li Si, etc.) and the keyword "ask" can be determined as the text set corresponding to the target candidate word group.

[0124] When performing the initial clustering of the text "Zhang San asked her whether her personal income tax was paid in City A or City B", we can cluster "[Name] asked her whether her personal income tax was paid in [Place Name] or [Place Name]". When determining the first similarity between the text and the cluster center of a certain initial cluster, we can determine it based on the first vector corresponding to "[Name] asked her whether her personal income tax was paid in [Place Name] or [Place Name]" and the second vector corresponding to the cluster center of the initial cluster.

[0125] This avoids the influence of words such as personal names, place names, and organizations on clustering, thus improving the accuracy of text clustering.

[0126] Step 407: For each target cluster, determine the third similarity between the texts within it.

[0127] Step 408: Delete the texts in the target cluster whose third similarity with all other texts is less than the second preset threshold.

[0128] The second preset threshold can be set as needed, and this application does not impose any restrictions on it.

[0129] In the example embodiment, after obtaining multiple target clusters, the third similarity between each text in each target cluster can be determined, and the texts in the target cluster whose third similarity with each other text is less than the second preset threshold can be deleted.

[0130] Therefore, texts with low similarity to other texts in the target cluster can be deleted, thereby improving the accuracy of text clustering.

[0131] To achieve the above embodiments, this application also proposes a text clustering device based on artificial intelligence (AI). Figure 5 This is a schematic diagram of the structure of a text clustering device based on artificial intelligence (AI) according to the fourth embodiment of this application.

[0132] like Figure 5 As shown, the AI-based text clustering device 500 includes: a word segmentation module 501, a splicing module 502, a grouping module 503, and a clustering module 504.

[0133] Among them, the word segmentation module 501 is used to perform word segmentation on each text in the text set to be clustered, so as to obtain multiple keywords contained in each text;

[0134] The splicing module 502 is used to splice multiple keywords contained in each text according to a preset method to obtain at least one corresponding candidate word group.

[0135] Grouping module 503 is used to group the text set to be clustered based on at least one candidate word group corresponding to each text to obtain multiple text sets;

[0136] Clustering module 504 is used to perform clustering processing on multiple text sets according to a preset clustering method to obtain multiple target clusters.

[0137] It should be noted that the AI-based text clustering device 500 of this application embodiment can execute the AI-based text clustering method provided in the above embodiments. The AI-based text clustering device 500 can be implemented by software and / or hardware. This AI-based text clustering device can be an electronic device, or it can be configured in an electronic device to improve the efficiency of text clustering, thereby improving the efficiency of building the chatbot. The electronic device can include, but is not limited to, terminal devices, servers, etc., and this embodiment does not specifically limit the electronic device. The text clustering device can be a chatbot configured in a terminal device.

[0138] In one embodiment of this application, the grouping module 503 includes:

[0139] The filtering unit is used to determine the statistical value of each candidate word group under the preset statistical indicators, and to filter each candidate word group according to the statistical value to obtain multiple target candidate word groups;

[0140] The determination unit is used to determine the set of texts in the text set to be clustered that contain each keyword in the target candidate word group as the corresponding text set for each target candidate word group.

[0141] In one embodiment of this application, the clustering module 504 includes:

[0142] The first clustering unit is used to perform initial clustering on multiple text sets to obtain a cluster set including multiple initial clusters;

[0143] The second clustering unit is used to perform secondary clustering on multiple text sets based on the cluster set, using the Singlepass clustering algorithm to obtain multiple target clusters.

[0144] In one embodiment of this application, the cluster set includes partial texts from multiple text sets; the second clustering unit is used for:

[0145] For target texts that are not classified into a cluster set in multiple text sets, determine the first similarity between the target text and the cluster centers of each initial cluster currently included in the cluster set, and obtain the highest similarity among the first similarities.

[0146] If the highest similarity is greater than the first preset threshold, the target text will be classified into the initial cluster corresponding to the highest similarity.

[0147] If the highest similarity is no greater than the first preset threshold, a new initial cluster is created, and the target text is assigned to the new initial cluster. The new initial cluster is also added to the cluster set to update the cluster set.

[0148] When all texts in multiple text sets are classified into a cluster set, the initial clusters included in the cluster set are determined as multiple target clusters.

[0149] In one embodiment of this application, the second clustering unit is further configured to:

[0150] Update the cluster center of the initial cluster corresponding to the highest similarity;

[0151] For each text in the initial cluster corresponding to the highest similarity, determine the second similarity with the updated cluster center;

[0152] Texts whose second similarity to the updated cluster center is not greater than the first preset threshold are removed from the initial cluster corresponding to the highest similarity.

[0153] In one embodiment of this application, the splicing module 502 includes: a combination unit, used to sequentially splice any two adjacent keywords from the plurality of keywords contained in each text to obtain at least one corresponding candidate word group.

[0154] In one embodiment of this application, the AI-based text clustering device 500 further includes:

[0155] The recognition module is used to perform named entity recognition on multiple keywords contained in each text to obtain the category to which each keyword belongs;

[0156] The processing module is used to replace keywords belonging to a preset category with the corresponding preset category in order to update the keywords.

[0157] In one embodiment of this application, the AI-based text clustering device 500 further includes:

[0158] The determination module is used to determine the third similarity between texts within each target cluster;

[0159] The deletion module is used to delete texts in the target cluster whose third similarity with all other texts is less than the second preset threshold.

[0160] In one embodiment of this application, the AI-based text clustering device 500 further includes:

[0161] The calling module is used to invoke the RPA robot to trigger the upload command for the text set to be clustered;

[0162] The acquisition module is used to acquire the text set to be clustered based on the upload command.

[0163] It should be noted that the foregoing explanation of the AI-based text clustering method embodiment also applies to the AI-based text clustering device of this embodiment. Details not disclosed in the AI-based text clustering device embodiment of this application will not be repeated here.

[0164] In summary, the AI-based text clustering device of this application performs word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text. For each text, the multiple keywords are concatenated according to a preset method to obtain at least one corresponding candidate word group. Based on the at least one candidate word group corresponding to each text, the text set to be clustered is grouped to obtain multiple text sets. Then, according to a preset clustering method, the multiple text sets are clustered separately to obtain multiple target clusters. Therefore, by grouping the text set to be clustered based on multiple candidate word groups to obtain multiple text sets, and then performing clustering processing on the multiple text sets separately according to a preset clustering method to obtain multiple target clusters, the time consumption of text clustering is reduced, the efficiency of text clustering is improved, and thus the efficiency of building a chatbot is improved.

[0165] To implement the above embodiments, this application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the text clustering method based on artificial intelligence (AI) as described in any of the foregoing method embodiments.

[0166] To implement the above embodiments, this application also proposes a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the AI-based text clustering method as described in any of the foregoing method embodiments.

[0167] To implement the above embodiments, this application also proposes a computer program product that, when executed by an instruction processor, implements the AI-based text clustering method as described in any of the foregoing method embodiments.

[0168] Figure 6 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown. Figure 6 The electronic device 10 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0169] like Figure 6 As shown, the electronic device 10 is presented in the form of a general-purpose computing device. The components of the electronic device 10 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including memory 28 and processing unit 16).

[0170] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. Examples of these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0171] Electronic device 10 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 10, including volatile and non-volatile media, removable and non-removable media.

[0172] Memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 10 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disc drive for reading and writing to a removable non-volatile optical disc (e.g., a compact disc read-only memory (CD-ROM), a digital video disc read-only memory (DVD-ROM), or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0173] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this application.

[0174] Electronic device 10 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the electronic device 10, and / or with any device that enables the electronic device 10 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, electronic device 10 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. Figure 6 As shown, network adapter 20 communicates with other modules of electronic device 10 via bus 18. It should be understood that, although... Figure 6 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 10, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0175] The processing unit 16 executes various functional applications and data processing by running programs stored in the memory 28, such as implementing the methods mentioned in the foregoing embodiments.

[0176] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0177] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0178] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0179] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0180] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0181] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0182] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0183] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A text clustering method based on artificial intelligence (AI), characterized in that, include: Each text in the clustered text set is segmented to obtain multiple keywords contained in each text; For each text, the multiple keywords contained therein are concatenated according to a preset method to obtain at least one corresponding candidate word group, including: for each text, any two adjacent keywords are concatenated sequentially to obtain the at least one corresponding candidate word group, wherein, before concatenating the multiple keywords contained therein in each text according to a preset method to obtain the at least one corresponding candidate word group, the method further includes: performing named entity recognition on the multiple keywords contained in each text to obtain the category to which each keyword belongs, and replacing the keywords belonging to the preset category with the corresponding preset category to update the keywords; Based on the at least one candidate word group corresponding to each of the texts, the text set to be clustered is grouped to obtain multiple text sets; According to a preset clustering method, multiple text sets are clustered to obtain multiple target clusters, including: performing initial clustering on each of the multiple text sets to obtain a cluster set including multiple initial clusters; and using the singlepass clustering algorithm on the cluster set to perform secondary clustering on the multiple text sets to obtain the multiple target clusters. The cluster set includes a portion of the text from the plurality of text sets; based on the cluster set, a single-pass clustering algorithm is used to perform secondary clustering on the plurality of text sets to obtain the plurality of target clusters, including: For target texts in multiple text sets that are not classified into the cluster set, determine the first similarity between the target text and the cluster center of each initial cluster currently included in the cluster set, and obtain the highest similarity among the first similarities; If the highest similarity is greater than a first preset threshold, the target text is assigned to the initial cluster corresponding to the highest similarity. If the highest similarity is not greater than the first preset threshold, a new initial cluster is created, and the target text is assigned to the newly created initial cluster. The newly created initial cluster is also added to the cluster set to update the cluster set. When each text in multiple text sets is assigned to the cluster set, each initial cluster included in the cluster set is determined as the multiple target clusters; After updating the cluster set, when determining the first similarity for subsequent target text, the initial clusters currently included in the cluster set are multiple initial clusters included in the updated cluster set. The method for obtaining the first similarity between the target text and the cluster centers of the initial clusters includes: vectorizing the target text, obtaining the first vector corresponding to the target text, arbitrarily obtaining a text from the initial clusters and vectorizing the text, using the obtained vector as the second vector corresponding to the cluster center of the initial cluster, determining the similarity between the first vector and the second vector, and using the similarity as the first similarity between the target text and the cluster centers of the initial clusters. The step of grouping the text set to be clustered based on the at least one candidate word group corresponding to each text to obtain multiple text sets includes: The statistical values ​​of each candidate word group under a preset statistical indicator are determined, and each candidate word group is screened according to the statistical values ​​to obtain multiple target candidate word groups. The preset statistical indicator includes at least one of degree of freedom, cohesion, and word frequency. For each target candidate word group, the set of texts in the text set to be clustered that contains each keyword in the target candidate word group is determined as the corresponding text set; After classifying the target text into the initial cluster corresponding to the highest similarity, the method further includes: Update the cluster center of the initial cluster corresponding to the highest similarity; For each text in the initial cluster corresponding to the highest similarity, determine the second similarity with the updated cluster center; Texts whose second similarity to the updated cluster center is not greater than the first preset threshold are removed from the initial cluster corresponding to the highest similarity.

2. The method according to claim 1, characterized in that, After performing clustering processing on the multiple text sets according to a preset clustering method to obtain multiple target clusters, the method further includes: For each target cluster, determine the third similarity between the texts within it; Delete the texts in the target cluster whose third similarity with each other text is less than the second preset threshold.

3. The method according to claim 1, characterized in that, Before performing word segmentation on each text in the text set to be clustered to obtain multiple keywords contained in each text, the method further includes: Invoke the RPA robot to trigger the upload command for the text set to be clustered; Based on the upload command, the text set to be clustered is obtained.

4. A text clustering device based on artificial intelligence (AI), characterized in that, include: The word segmentation module is used to segment each text in the text set to be clustered, so as to obtain multiple keywords contained in each text. A concatenation module is used to concatenate multiple keywords contained in each text to obtain at least one corresponding candidate word group. This includes: for each text, sequentially concatenating any two adjacent keywords from the multiple keywords contained to obtain the at least one corresponding candidate word group. Before concatenating the multiple keywords contained in each text according to a preset method to obtain the at least one corresponding candidate word group, the module further includes: performing named entity recognition on the multiple keywords contained in each text to obtain the category to which each keyword belongs, and replacing keywords belonging to a preset category with the corresponding preset category to update the keywords. A grouping module is used to group the text set to be clustered based on the at least one candidate word group corresponding to each text, so as to obtain multiple text sets; The clustering module is used to perform clustering processing on multiple text sets according to a preset clustering method to obtain multiple target clusters; The clustering module includes: The first clustering unit is used to perform initial clustering on the multiple text sets respectively to obtain a cluster set including multiple initial clusters; The second clustering unit is used to perform secondary clustering on the multiple text sets based on the cluster set, using the Singlepass clustering algorithm, to obtain the multiple target clusters; The cluster set includes a portion of the text from the plurality of text sets; based on the cluster set, a single-pass clustering algorithm is used to perform secondary clustering on the plurality of text sets to obtain the plurality of target clusters, including: For target texts in multiple text sets that are not classified into the cluster set, determine the first similarity between the target text and the cluster center of each initial cluster currently included in the cluster set, and obtain the highest similarity among the first similarities; If the highest similarity is greater than a first preset threshold, the target text is assigned to the initial cluster corresponding to the highest similarity. If the highest similarity is not greater than the first preset threshold, a new initial cluster is created, and the target text is assigned to the newly created initial cluster. The newly created initial cluster is also added to the cluster set to update the cluster set. When each text in multiple text sets is assigned to the cluster set, each initial cluster included in the cluster set is determined as the multiple target clusters; After updating the cluster set, when determining the first similarity for subsequent target text, the initial clusters currently included in the cluster set are multiple initial clusters included in the updated cluster set. The method for obtaining the first similarity between the target text and the cluster centers of the initial clusters includes: vectorizing the target text, obtaining the first vector corresponding to the target text, arbitrarily obtaining a text from the initial clusters and vectorizing the text, using the obtained vector as the second vector corresponding to the cluster center of the initial cluster, determining the similarity between the first vector and the second vector, and using the similarity as the first similarity between the target text and the cluster centers of the initial clusters. The grouping module includes: A filtering unit is used to determine the statistical value of each candidate word group under a preset statistical indicator, and to filter each candidate word group according to the statistical value to obtain multiple target candidate word groups, wherein the preset statistical indicator includes at least one of degree of freedom, cohesion, and word frequency; The determining unit is configured to, for each of the target candidate word groups, determine the set of texts in the text set to be clustered that contains each keyword in the target candidate word group as the corresponding text set; After classifying the target text into the initial cluster corresponding to the highest similarity, the method further includes: Update the cluster center of the initial cluster corresponding to the highest similarity; For each text in the initial cluster corresponding to the highest similarity, determine the second similarity with the updated cluster center; Texts whose second similarity to the updated cluster center is not greater than the first preset threshold are removed from the initial cluster corresponding to the highest similarity.

5. The apparatus according to claim 4, characterized in that, Also includes: The recognition module is used to perform named entity recognition on the multiple keywords contained in each of the texts to obtain the category to which each keyword belongs; The processing module is used to replace keywords belonging to a preset category with the corresponding preset category in order to update the keywords.

6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-3.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-3.

Images