A hot issue discovery method and system based on machine learning
Through machine learning technology, 12366 hot issues are preprocessed and analyzed, and TF-IDF, Word2vec, k-means and TextRank algorithms are used to solve the accuracy and lag of manual statistical methods, realizing accurate automatic discovery of hot issues and Classification analysis improves service efficiency.
Patent Information
- Application Number
- CN202410903268.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-08
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-07-08
AI Technical Summary
The existing 12366 hot-spot problem analysis methods rely on manual statistics, and there are inaccurate labels, lags and semantic deviations, making it difficult to accurately detect high-frequency hot-spot problems.
Machine learning technology, including TF-IDF, Word2vec, k-means clustering and TextRank algorithms, preprocessing, vector representation, clustering and weight analysis of incoming call consultation issues, and automatically discover and classify hot issues.
It realizes more accurate and scientific automatic discovery and classified analysis of hot issues, and improves service efficiency.
Smart Images

Figure CN118445403B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine learning, and in particular to a method and system for discovering hot issues based on machine learning. Background Art
[0002] At present, the analysis method of 12366 hot issues is mainly based on manual statistics. The operators put preset labels on the inquiries of the callers and extract the questions with the same labels through the background for statistical analysis. In the context of the current tax system vigorously promoting tax modernization, it is necessary to introduce machine learning technology to scientifically extract the caller questions, find out the high-frequency hot issues every month, and provide reference for accurate push and relevant departments to formulate targeted management and service measures. Summary of the invention
[0003] The 12366 hot issue discovery method is mainly a manual statistical analysis method, which has limitations: first, due to differences in personal experience, business understanding, etc., the labels that operators put on incoming calls may not be accurate, and there are also cases where the same issue involves multiple labels; second, there is a lag in the labels, and the update of the labels lags behind the issues, and new issues may not have suitable labels; third, the manual statistical analysis method has limitations, and the text description is prone to semantic deviations, and the discovery and analysis of hot issues is difficult. Based on this, the present invention provides a method and system for discovering hot issues based on machine learning.
[0004] In the first aspect, the present invention provides a method for discovering hot issues based on machine learning. The technical solution adopted to solve the above technical problems is as follows:
[0005] A method for discovering hot issues based on machine learning, comprising the following steps:
[0006] S1. Obtain the problem summary text of the service hotline caller consultation, and pre-process and segment the problem summary text;
[0007] S2. For the question summary text after preprocessing and word segmentation, the TF-IDF algorithm is used to calculate the importance of each word, and the K candidate words with the highest scores are selected as hot words;
[0008] S3, using the segmentation results of the selected hot words as input, training the word vector model through the Word2vec method, obtaining the vector form of the segmentation described in step S1, and then obtaining the vector representation of each sentence in the question summary text;
[0009] S4, clustering the vectorized sentences in step S3 using the k-means clustering algorithm to group similar questions into the same cluster;
[0010] S5. Combine the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster;
[0011] S6. Associate the person who raises the question with the person who performs the corresponding obligation, and display hot issues and drill down to detailed issues according to different dimensions.
[0012] Optionally, execute step S1. First, preprocess the question summary text, including: removing duplicates, converting traditional Chinese to simplified Chinese, removing stop words, clearing symbols, converting uppercase to lowercase, and converting full-width to half-width. Then, use the jieba word segmentation tool to segment the preprocessed question summary text.
[0013] Optionally, step S4 is performed to cluster the vectorized sentences in step S3 using a k-means clustering algorithm to group similar questions into the same cluster, specifically including:
[0014] First, a range of cluster numbers is preset, and the K-means clustering algorithm is run for each preset range of cluster numbers to obtain clustering results of different clusters;
[0015] Calculate and compare the silhouette coefficient of each clustering result, and select the cluster number with the highest silhouette coefficient as the final number of clusters;
[0016] Based on the selected number of clusters, the K-means clustering algorithm is run to group similar questions into the same cluster.
[0017] Optionally, step S5 is performed to merge the vectorized sentences in each cluster into one text, and the text of each cluster is analyzed using the TextRank algorithm to find the text sentence with the highest weight as the hot issue of the cluster, which specifically includes:
[0018] Merge all vectorized sentences in each cluster into one text;
[0019] Using the TextRank algorithm, each sentence in the text is treated as a node in the graph. When two sentences have common keywords or phrases, an edge is created between the corresponding nodes, and the weight of the edge is determined by the TF-IDF value of the common vocabulary. The score of each node is calculated iteratively until the score is stable, and the final score of the node is used to know its importance in the text.
[0020] For each cluster, the top three sentences with the highest weights calculated by the TextRank algorithm are selected as candidate description sentences for hot issues;
[0021] Weights are set for the candidate description sentence's length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs. The weights of the candidate description sentence in terms of sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs are added together to obtain a comprehensive weight. The candidate description sentence with the highest comprehensive weight is selected as the hot issue of the cluster.
[0022] Preferably, when setting weights for the candidate description sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs:
[0023] Evaluate the sentence lengths of the three candidate description sentences, set three different weights based on the sentence lengths, and the sum of the weights of the three candidate description sentences is 1;
[0024] Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1;
[0025] Check the similarity between the three candidate description sentences and the remaining sentences in their clusters, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1.
[0026] In a second aspect, the present invention provides a hot issue discovery system based on machine learning. The technical solution adopted to solve the above technical problems is as follows:
[0027] A hot issue discovery system based on machine learning, comprising:
[0028] A preprocessing and word segmentation module is used to preprocess and word segment the question summary text obtained from the service hotline call consultation;
[0029] The calculation and selection module is used to calculate the importance of each word using the TF-IDF algorithm and select the K candidate words with the highest scores as hot words;
[0030] The vector processing module is used to select the segmentation results of hot words as input, train the word vector model through the Word2vec method, obtain the vector form of the segmentation output by the preprocessing and segmentation modules, and then obtain the vector representation of each sentence in the question summary text;
[0031] The clustering grouping module is used to cluster the vectorized sentences output by the vector processing module using the k-means clustering algorithm to group similar questions into the same cluster;
[0032] The merge analysis module is used to merge the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster;
[0033] The association display module is used to associate the person who raises the question with the person who performs the corresponding obligation, and to display hot issues and drill down to detailed issues according to different dimensions.
[0034] Optionally, the preprocessing and word segmentation module performs preprocessing and word segmentation on the obtained question summary text of the service hotline call consultation, specifically including:
[0035] Preprocess the text of the problem summary by removing duplicates, converting traditional Chinese to simplified Chinese, removing stop words, clearing symbols, converting uppercase to lowercase, and converting full-width to half-width;
[0036] The jieba word segmentation tool is used to segment the preprocessed question summary text.
[0037] Optionally, the clustering grouping module uses a k-means clustering algorithm to cluster the vectorized sentences output by the vector processing module to group similar questions into the same cluster, specifically including:
[0038] First, a range of cluster numbers is preset, and the K-means clustering algorithm is run for each preset range of cluster numbers to obtain clustering results of different clusters;
[0039] Calculate and compare the silhouette coefficient of each clustering result, and select the cluster number with the highest silhouette coefficient as the final number of clusters;
[0040] Based on the selected number of clusters, the K-means clustering algorithm is run to group similar questions into the same cluster.
[0041] Optionally, the merge analysis module merges the vectorized sentences in each cluster into one text, uses the TextRank algorithm to analyze the text of each cluster, and finds the text sentence with the highest weight as the hot issue of the cluster, specifically including:
[0042] Merge all vectorized sentences in each cluster into one text;
[0043] Using the TextRank algorithm, each sentence in the text is treated as a node in the graph. When two sentences have common keywords or phrases, an edge is created between the corresponding nodes, and the weight of the edge is determined by the TF-IDF value of the common vocabulary. The score of each node is calculated iteratively until the score is stable, and the final score of the node is used to know its importance in the text.
[0044] For each cluster, the top three sentences with the highest weights calculated by the TextRank algorithm are selected as candidate description sentences for hot issues;
[0045] Weights are set for the candidate description sentence's length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs. The weights of the candidate description sentence in terms of sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs are added together to obtain a comprehensive weight. The candidate description sentence with the highest comprehensive weight is selected as the hot issue of the cluster.
[0046] Preferably, when setting weights for the candidate description sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs:
[0047] Evaluate the sentence lengths of the three candidate description sentences, set three different weights based on the sentence lengths, and the sum of the weights of the three candidate description sentences is 1;
[0048] Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1;
[0049] Check the similarity between the three candidate description sentences and the remaining sentences in their clusters, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1.
[0050] Compared with the prior art, the hot issue discovery method and system based on machine learning of the present invention have the following beneficial effects:
[0051] The present invention adopts machine learning algorithms such as Word2Vec, TF-IDF, K-means and TextRank. By cleaning and converting the consultation questions of incoming calls, it automatically extracts hot consultation questions every month, realizes more accurate and scientific automatic discovery and classification analysis of hot consultation questions, and improves service efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] Attached Figure 1 is a flow chart of a method according to Embodiment 1 of the present invention;
[0053] Attached Figure 2 This is a module connection block diagram of the second embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to make the technical solution, the technical problem solved and the technical effect of the present invention more clearly understood, the technical solution of the present invention is clearly and completely described below in conjunction with specific embodiments.
[0055] The English words in this embodiment are explained as follows:
[0056] Word2Vec: The full name is word to vector, a word vector method that represents words as real-valued vectors. The models used are CBOW (Continuous Bag-Of-Words) and Skip-Gram.
[0057] TF-IDF: Term Frequency-Inverse Document Frequency, full name Term Frequency-Inverse Document Frequency, is a commonly used weighting technology used in information retrieval and text mining.
[0058] K-means: K-means clustering algorithm, the full name is k-means clustering algorithm, the most commonly used clustering algorithm based on Euclidean distance, which believes that the closer the distance between two targets, the greater the similarity.
[0059] TextRank: It is a graph-based ranking algorithm for keyword extraction and document summarization, which is improved from Google's page importance ranking algorithm PageRank.
[0060] Embodiment 1: Combining with the attached Figure 1 This embodiment proposes a method for discovering hot issues based on machine learning. Taking the 12366 tax service hotline as an example, it includes the following steps:
[0061] S1. Obtain the problem summary text of the 12366 tax service hotline callers on a monthly basis, and pre-process and segment the problem summary text, including:
[0062] Preprocessing operations include: deduplication, conversion from traditional Chinese to simplified Chinese, removal of stop words, removal of symbols, conversion from uppercase to lowercase, and conversion from full-width to half-width;
[0063] The jieba word segmentation tool is used to segment the preprocessed question summary text.
[0064] S2. For the question summary text after preprocessing and word segmentation, the TF-IDF algorithm is used to calculate the importance of each word, and the K candidate words with the highest scores are selected as hot words.
[0065] S3. Using the segmentation results of the selected hot words as input, the word vector model is trained by the Word2vec method to obtain the vector form of the segmentation described in step S1, and then the vector representation of each sentence in the question summary text is obtained.
[0066] S4. Cluster the vectorized sentences in step S3 using the k-means clustering algorithm to group similar questions into the same cluster, including:
[0067] First, a range of cluster numbers is preset, and the K-means clustering algorithm is run for each preset range of cluster numbers to obtain clustering results of different clusters;
[0068] Calculate and compare the silhouette coefficient of each clustering result, and select the cluster number with the highest silhouette coefficient as the final number of clusters;
[0069] Based on the selected number of clusters, the K-means clustering algorithm is run to group similar questions into the same cluster.
[0070] S5. Combine the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster, including:
[0071] Merge all vectorized sentences in each cluster into one text;
[0072] Using the TextRank algorithm, each sentence in the text is treated as a node in the graph. When two sentences have common keywords or phrases, an edge is created between the corresponding nodes, and the weight of the edge is determined by the TF-IDF value of the common vocabulary. The score of each node is calculated iteratively until the score is stable, and the final score of the node is used to know its importance in the text.
[0073] For each cluster, the top three sentences with the highest weights calculated by the TextRank algorithm are selected as candidate description sentences for hot issues;
[0074] Weights are set for the sentence length of the candidate description sentence, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs: ① Evaluate the sentence length of the three candidate description sentences, set three different weights based on the sentence length, and the sum of the weights of the three candidate description sentences is 1; ② Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1; ③ Check the similarity of the three candidate description sentences with the remaining sentences in their cluster, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1;
[0075] The weights of the candidate description sentences in terms of sentence length, number of content words contained, and similarity with the remaining sentences in the cluster to which they belong are added to obtain a comprehensive weight, and the candidate description sentence with the highest comprehensive weight is selected as the hot issue of the cluster.
[0076] S6. Associate the person who raises the question with the taxpayer, and display hot issues and drill down to detailed issues according to different dimensions.
[0077] Embodiment 2: Combining with Figure 2This embodiment proposes a hot issue discovery system based on machine learning, taking the 12366 tax service hotline as an example, which includes:
[0078] The preprocessing and word segmentation module is used to preprocess and word segment the question summary text obtained from the 12366 tax service hotline call consultation;
[0079] The calculation and selection module is used to calculate the importance of each word using the TF-IDF algorithm and select the K candidate words with the highest scores as hot words;
[0080] The vector processing module is used to select the segmentation results of hot words as input, train the word vector model through the Word2vec method, obtain the vector form of the segmentation output by the preprocessing and segmentation modules, and then obtain the vector representation of each sentence in the question summary text;
[0081] The clustering grouping module is used to cluster the vectorized sentences output by the vector processing module using the k-means clustering algorithm to group similar questions into the same cluster;
[0082] The merge analysis module is used to merge the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster;
[0083] The associated display module is used to associate the person who asks the question with the taxpayer, and to display hot issues and drill down to detailed issues according to different dimensions.
[0084] In this embodiment, the preprocessing and word segmentation module performs preprocessing and word segmentation on the obtained question summary text of the service hotline call consultation, specifically including:
[0085] Preprocess the text of the problem summary by removing duplicates, converting traditional Chinese to simplified Chinese, removing stop words, clearing symbols, converting uppercase to lowercase, and converting full-width to half-width;
[0086] The jieba word segmentation tool is used to segment the preprocessed question summary text.
[0087] In this embodiment, the clustering grouping module uses the k-means clustering algorithm to cluster the vectorized sentences output by the vector processing module to group similar questions into the same cluster, specifically including:
[0088] First, a range of cluster numbers is preset, and the K-means clustering algorithm is run for each preset range of cluster numbers to obtain clustering results of different clusters;
[0089] Calculate and compare the silhouette coefficient of each clustering result, and select the cluster number with the highest silhouette coefficient as the final number of clusters;
[0090] Based on the selected number of clusters, the K-means clustering algorithm is run to group similar questions into the same cluster.
[0091] In this embodiment, the merge analysis module merges the vectorized sentences in each cluster into one text, uses the TextRank algorithm to analyze the text of each cluster, and finds the text sentence with the highest weight as the hot issue of the cluster, which specifically includes:
[0092] Merge all vectorized sentences in each cluster into one text;
[0093] Using the TextRank algorithm, each sentence in the text is treated as a node in the graph. When two sentences have common keywords or phrases, an edge is created between the corresponding nodes, and the weight of the edge is determined by the TF-IDF value of the common vocabulary. The score of each node is calculated iteratively until the score is stable, and the final score of the node is used to know its importance in the text.
[0094] For each cluster, the top three sentences with the highest weights calculated by the TextRank algorithm are selected as candidate description sentences for hot issues;
[0095] Weights are set for the sentence length of the candidate description sentence, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs: ① Evaluate the sentence length of the three candidate description sentences, set three different weights based on the sentence length, and the sum of the weights of the three candidate description sentences is 1; ② Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1; ③ Check the similarity of the three candidate description sentences with the remaining sentences in their cluster, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1;
[0096] The weights of the candidate description sentences in terms of sentence length, number of content words contained, and similarity with the remaining sentences in the cluster to which they belong are added to obtain a comprehensive weight, and the candidate description sentence with the highest comprehensive weight is selected as the hot issue of the cluster.
[0097] In summary, the use of a hot issue discovery method and system based on machine learning of the present invention can achieve more accurate and scientific automatic discovery and classification analysis of hot issues, thereby improving service efficiency.
[0098] The above specific examples are used to explain the principles and implementation methods of the present invention in detail. These examples are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made by technicians in this technical field without departing from the principles of the present invention should fall within the scope of patent protection of the present invention.
Claims
1. A hot issue discovery method based on machine learning, It is characterized in that The steps include: S1. Obtain the problem summary text of the service hotline caller consultation, and pre-process and segment the problem summary text; S2. For the question summary text after preprocessing and word segmentation, the TF-IDF algorithm is used to calculate the importance of each word, and the K candidate words with the highest scores are selected as hot words; S3, using the segmentation results of the selected hot words as input, training the word vector model through the Word2vec method, obtaining the vector form of the segmentation described in step S1, and then obtaining the vector representation of each sentence in the question summary text; S4, clustering the vectorized sentences in step S3 using the k-means clustering algorithm to group similar questions into the same cluster, specifically comprising: first presetting a range of cluster numbers, running the K-means clustering algorithm for each of the preset ranges of cluster numbers to obtain clustering results of different clusters; calculating and comparing the silhouette coefficient of each clustering result, and selecting the cluster number with the highest silhouette coefficient as the final cluster number; running the K-means clustering algorithm according to the selected cluster number to group similar questions into the same cluster; S5. Merge the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster, which specifically includes: merge all the vectorized sentences in each cluster into one text; use the TextRank algorithm to treat each sentence in the text as a node of the graph; when two sentences have common keywords or phrases, create an edge between the corresponding nodes, and the weight of the edge is determined according to the TF-IDF value of the common vocabulary; iteratively calculate the score of each node until the score is stable, and the importance of the node in the text is known through the final score of the node; for each cluster, select the top three sentences with the highest weight calculated by the TextRank algorithm as candidate description sentences for hot issues; set weights for the candidate description sentence’s sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs, add the weights of the candidate description sentence in terms of sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs, and obtain the comprehensive weight, and select the candidate description sentence with the highest comprehensive weight as the hot issue of the cluster; S6. Associate the person who raises the question with the person who performs the corresponding obligation, and display hot issues and drill down to detailed issues according to different dimensions.
2. According to the method for discovering hot issues based on machine learning in claim 1, It is characterized in that Execute step S1. First, preprocess the question summary text, including: removing duplicates, converting traditional Chinese to simplified Chinese, removing stop words, clearing symbols, converting uppercase to lowercase, and converting full-width to half-width. Then, use the jieba word segmentation tool to segment the preprocessed question summary text.
3. According to the method for discovering hot issues based on machine learning in claim 1, It is characterized in that When setting weights for the candidate description sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs: Evaluate the sentence lengths of the three candidate description sentences, set three different weights based on the sentence lengths, and the sum of the weights of the three candidate description sentences is 1; Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1; Check the similarity between the three candidate description sentences and the remaining sentences in their clusters, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1.
4. A hot issue discovery system based on machine learning, It is characterized in that It includes: A preprocessing and word segmentation module is used to preprocess and word segment the question summary text obtained from the service hotline call consultation; The calculation and selection module is used to calculate the importance of each word using the TF-IDF algorithm and select the K candidate words with the highest scores as hot words; The vector processing module is used to select the segmentation results of hot words as input, train the word vector model through the Word2vec method, obtain the vector form of the segmentation output by the preprocessing and segmentation modules, and then obtain the vector representation of each sentence in the question summary text; The clustering grouping module is used to cluster the vectorized sentences output by the vector processing module using the k-means clustering algorithm to group similar questions into the same cluster. The specific process includes: firstly presetting a range of cluster numbers, and running the K-means clustering algorithm for each cluster number in the preset range to obtain clustering results of different clusters; calculating and comparing the silhouette coefficient of each clustering result, and selecting the cluster number with the highest silhouette coefficient as the final cluster number; and running the K-means clustering algorithm according to the selected cluster number to group similar questions into the same cluster; The merging analysis module is used to merge the vectorized sentences in each cluster into one text, use the TextRank algorithm to analyze the text of each cluster, and find the text sentence with the highest weight as the hot issue of the cluster. The specific process includes: merging all the vectorized sentences in each cluster into one text; using the TextRank algorithm, each sentence in the text is used as a node in the graph; when two sentences have common keywords or phrases, an edge is created between the corresponding nodes, and the weight of the edge is determined according to the TF-IDF value of the common vocabulary; the score of each node is calculated iteratively until the score is stable, and its importance in the text is known by the final score of the node; for each cluster, the top three sentences with the highest weight calculated by the TextRank algorithm are selected as candidate description sentences for hot issues; weights are set for the candidate description sentence’s sentence length, the number of content words contained, and the similarity with the remaining sentences in the cluster to which it belongs, and the weights of the candidate description sentence in terms of sentence length, the number of content words contained, and the similarity with the remaining sentences in the cluster to which it belongs are added to obtain a comprehensive weight, and the candidate description sentence with the highest comprehensive weight is selected as the hot issue of the cluster; The association display module is used to associate the person who raises the question with the person who performs the corresponding obligation, and to display hot issues and drill down to detailed issues according to different dimensions.
5. A hot issue discovery system based on machine learning according to claim 4, It is characterized in that The preprocessing and word segmentation module preprocesses and word segments the obtained question summary text of the service hotline call consultation, specifically including: Preprocess the text of the problem summary by removing duplicates, converting traditional Chinese to simplified Chinese, removing stop words, clearing symbols, converting uppercase to lowercase, and converting full-width to half-width; The jieba word segmentation tool is used to segment the preprocessed question summary text.
6. A hot issue discovery system based on machine learning according to claim 4, It is characterized in that When setting weights for the candidate description sentence length, the number of content words it contains, and the similarity with the remaining sentences in the cluster to which it belongs: Evaluate the sentence lengths of the three candidate description sentences, set three different weights based on the sentence lengths, and the sum of the weights of the three candidate description sentences is 1; Analyze the parts of speech of the three candidate description sentences, set the largest weight for the candidate description sentence containing the most content words, and the sum of the weights of the three candidate description sentences is 1; Check the similarity between the three candidate description sentences and the remaining sentences in their clusters, set the largest weight for the candidate description sentence with the highest similarity to the remaining sentences in the cluster, and the sum of the weights of the three candidate description sentences is 1.
