Government text classification and hot issue mining method and system based on machine learning

Abstract

The present invention discloses a method and system for classifying government texts and mining hot issues based on machine learning. The classification method includes: obtaining multiple training government text data and corresponding labels, and constructing a coding dictionary; obtaining vector representations of the multiple training government text data based on the coding dictionary; encoding the label data to obtain vector representations of each label; using a machine learning model to train a government text classification model based on the vector representations of the text data and the corresponding labels; the government text classification model is used for government text classification. The present invention constructs a dictionary through government texts, and performs text encoding and vector representation based on the dictionary, which can improve the accuracy of government text classification. On the basis of classification, the problems in each category are clustered, and the number of problem categories calculated by similarity can further improve the effect of government text clustering.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text data mining, and in particular to a method and system for government text classification and hot issue mining based on machine learning. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] With the development of network technology, people can get the latest information through online government inquiry platforms such as Weibo, WeChat, Mayor's Mailbox, and Sunshine Hotline, and express their ideas or suggestions at any time. This has also broadened the channels for people to respond to problems, and relevant departments can also grasp people's wishes at any time, so as to provide better services. However, the continuous increase in the amount of text data related to various social conditions and public opinions has brought great challenges to the work of relevant departments that used to rely mainly on manual division of messages and hot spot sorting. At the same time, with the development of technologies such as big data, cloud computing, and artificial intelligence, the establishment of a smart government system based on natural language processing technology has become a new trend in the innovative development of social governance, which has a great driving force for improving the government's management level and governance efficiency. Therefore, the application of network text analysis and data mining technology to the application of online government inquiry platforms is of great significance.

[0004] According to the inventors, the current text classification technology includes classification based on machine learning and clustering based on clustering algorithms. Among them, the current text classification method usually uses a general dictionary to encode words. This method ignores the contextual relationship of the text. Each word is independent of each other and cannot represent semantic information, resulting in insufficient model performance. Clustering algorithms usually specify several categories subjectively, and the classification results obtained lack objectivity. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for government text classification and hot issue mining based on machine learning. The classification method constructs a coding dictionary based on the government text dictionary, combines the dictionary for text encoding and vector representation, and embeds it into the machine learning model, which can improve the accuracy of government text classification.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0007] A government text classification method based on machine learning includes the following steps:

[0008] Acquire multiple pieces of training government text data and corresponding labels, and construct a coding dictionary; the coding dictionary includes each word in the training government text data and the corresponding code;

[0009] Obtaining vector representations of the plurality of training government text data based on a coding dictionary;

[0010] Encode the label data to obtain the vector representation of each label;

[0011] According to the vector representation of text data and corresponding labels, a machine learning model is used to train a government text classification model; the government text classification model is used for government text classification.

[0012] One or more embodiments provide a method for mining hot issues in government texts, including the following steps:

[0013] Acquire multiple pieces of government text data, and obtain the category to which each piece of government text data belongs based on the classification method;

[0014] For a certain type of government text data:

[0015] Determine the number of categories of message data in this type of government text data based on a similarity analysis method;

[0016] Use the number of categories as the number of cluster categories to cluster the message data;

[0017] Combined with the evaluation of the message data, the popularity of the message data is calculated.

[0018] One or more embodiments provide a government text classification system based on machine learning, including:

[0019] A data acquisition module is configured to acquire a plurality of training government text data and corresponding labels;

[0020] A dictionary construction module is configured to construct a coding dictionary; the coding dictionary includes each word in the training government text data and the corresponding code;

[0021] A vector representation module is configured to obtain vector representations of the plurality of training government text data based on a coding dictionary; encode the label data to obtain vector representations of each label;

[0022] The model training module is configured to train a government text classification model using a machine learning model based on vector representations of text data and corresponding labels; the government text classification model is used for government text classification.

[0023] One or more embodiments provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the government text classification method or the government text hot issue mining method is implemented.

[0024] One or more embodiments provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the government text classification method or the government text hot issue mining method.

[0025] One or more of the above technical solutions have the following beneficial effects:

[0026] During the training stage of government text classification, a dictionary is built through government texts instead of using the dictionary commonly used in the industry. Encoding and word embedding based on the dictionary are input into the machine learning model to improve the accuracy of government text classification.

[0027] When mining hot issues, we do not cluster the issues directly, but cluster them in each type of government text, which can improve clustering efficiency and accuracy. Moreover, when clustering, the number of cluster categories K is not randomly selected, but is calculated based on the similarity between each issue. Issues with similarities greater than a set threshold are classified into one category. The resulting number of issue categories can further improve the clustering effect of the issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0029] Figure 1 This is a flow chart of a method for classifying government documents based on machine learning in an embodiment of the present invention;

[0030] Figure 2 This is a diagram of the machine learning model architecture in an embodiment of the present invention;

[0031] Figure 3 This is a flow chart of a method for mining hot issues in government affairs texts in an embodiment of the present invention;

[0032] Figure 4 This is a flow chart of an evaluation method for message replies in an embodiment of the present invention;

[0033] Figure 5 It is a schematic diagram of the result that the similarity is 0 in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0035] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0036] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0037] Embodiment 1

[0038] This embodiment discloses a method for classifying government documents based on machine learning. Figure 1 As shown, including:

[0039] S1: Acquire multiple pieces of training government text data and corresponding label data, and construct a dictionary of the training government text, wherein the dictionary includes each word in the training government text data and its corresponding code;

[0040] The government text data records the message number, message user, message subject, message time, and message details of each message user. There is also a first-level label for training data, but not for test data. According to the content in the government text document, the user's message details are extracted and data preprocessing, word segmentation, and stop word removal are performed. There are 7 categories of labels in the data, namely urban and rural construction, environmental protection, transportation, education, culture and sports, labor and social security, commerce and tourism, and health and family planning.

[0041] The initial data set used in this embodiment includes message information of 9210 message-leaving users, and the data set format is csv format, as shown in Table 1.

[0042] Table 1 User message classification information table

[0043]

[0044]

[0045] The specific steps of constructing a training dictionary of government affairs texts include:

[0046] For the government text used for training, all non-repeated Chinese characters in the text are extracted to obtain a vocabulary, and then encoded. These Chinese characters and corresponding codes are regarded as a dictionary. In this embodiment, there are 5000 characters in the government text used for training, and a vocabulary of 5000 characters is constructed, one character per line, and encoded from 0 to 4999 to obtain the code corresponding to each Chinese character. The following is an example of some dictionaries constructed in this embodiment:

[0047] dict={' <pad>':0,',':1,'of':2,'. ':3,'one':4,'is':5,'in':6,......,'small':122,'min':203,'yang',659,'yin':793,......,}.

[0048] S2: Matching the plurality of training government text data with the dictionary to obtain a vector representation of each text data; encoding the label data to obtain a vector representation of each label;

[0049] Each training government text is matched with the dictionary to obtain multiple text vectors. In this embodiment, the length of each text vector is set to 600. If the length is less than 600, 0 is added. If the length is more than 600, the last 600 is taken. For example, a training government text is "Noise disturbs residents in Sunshine Community". After matching, cont_to_id = [659, 429, 122, 213, 2612, 793, 1478, 203, ...., 0, 0, 0, 0].

[0050] Encoding the label data includes: first, setting a unique number id for each category. In this embodiment, the six categories of labels are represented by numbers from 0 to 6: label_to_id = {'Urban and Rural Construction': 0, 'Environmental Protection': 1, 'Transportation': 2, 'Education and Culture': 3, 'Labor and Social Security': 4, 'Commerce and Tourism': 5, 'Health and Family Planning': 6}; then, one-hot encoding is performed on each label data to obtain the encoded representation of each label, for example: Urban and Rural Construction = [1, 0, 0, 0, 0, 0], Environmental Protection = [0, 1, 0, 0, 0, 0, 0].

[0051] S3: Input the text data vector and the corresponding label vector into the machine learning model, train the machine learning model, and obtain a trained machine learning model;

[0052] The text data vector of the training government text data is used as the input of the machine learning model, and the corresponding label vector is used as the output to train the model parameters of the machine learning model. Before inputting the model, the text data vector and the corresponding label vector are shuffled. After the shuffle, the parameters are not easy to fall into the local optimum, and the model can converge more easily.

[0053] The machine learning model used in this embodiment is a convolutional neural network model CNN. The convolutional neural network model includes: an embedding layer, a convolution layer, a pooling layer, a fully connected layer, and a softmax layer. Specifically, the number of neurons in the fully connected layer is 128, followed by dropout and relu activation functions, and Adam optimized CNN is used.

[0054] Embedding layer, first input data (a text data), after preprocessing in step S2, so that each sample is filled with 600 words, and converted into word vectors according to the dictionary, enter the embedding layer, randomly initialize the embedding vector of each word, and represent each word with a 64-dimensional vector, that is, the embedding vector dimension of each word is 64. All text data are processed according to the above method to obtain a three-dimensional embedding matrix, which consists of the number of texts, the length of each text, and the dimension of each word. In this embodiment, the three-dimensional embedding matrix can be expressed as: shape = (64,600,64), the first parameter 64 is the size of the batch data, indicating 64 sentences per batch, the second parameter 600 indicates 600 words per sentence, and the third parameter 64 indicates that each word is represented by a 64-dimensional vector. The obtained three-dimensional embedding matrix is ​​input to the convolutional layer.

[0055] Convolution layer, the number of convolution kernels is 256, the convolution kernel size is 5, and then connected to the pooling layer; the vector obtained after convolution is of shape = (64,596,256), the first parameter 64 is the size of the batch data, the second parameter 596 is the length of the vector after the vertical convolution on the feature layer, and the third parameter 256 means that there are 256 convolution kernels, and 256 column vectors will be obtained.

[0056] In the pooling layer, the maximum value of the 256 column vectors is obtained, and shape = (64, 256) is obtained. The first parameter 64 is the size of the batch data, and the second parameter is the maximum value of the 256 column vectors.

[0057] The fully connected layer concatenates all the pooled feature values ​​together to form a single feature map, which is then fully connected to a softmax layer for classification.

[0058] Entering the fully connected layer is the mapping from the hidden layer to the output layer. The fully connected layer has 128 neurons. The shape after full connection is (64, 128). The first parameter 64 is the size of the batch data, and the second parameter 128 is the dimension we specified.

[0059] After drop, it is put into the activation function relu function.

[0060] After another fully connected layer, we get shape = (64,7). The first parameter 64 is the size of the batch data, and the second parameter 7 is the number of categories.

[0061] Then, the softmax function is used to get the probability of each category. The value with the largest probability is taken to get the predicted category.

[0062] Use the categories predicted by the model and the actual categories to calculate the cross entropy loss function, optimize the loss function, and minimize the loss function.

[0063] Compare the predicted classification results with the known label categories. If the accuracy of the classification results reaches the set threshold, stop training, and the machine learning model corresponding to the highest accuracy of the output classification results is the trained machine learning model; otherwise, continue training.

[0064] Figure 2 This is the structure diagram of the CNN for text classification. The classifier accuracy can reach 94%.

[0065] S4: Obtain government text data to be classified, input the government text data to be classified into the machine learning model, and output the classification result of the government text data.

[0066] During the training stage of government text classification, a dictionary is built through government texts instead of using the dictionary commonly used in the industry. Encoding and word embedding based on the dictionary are input into the machine learning model to improve the accuracy of government text classification.

[0067] Embodiment 2

[0068] This embodiment provides a government document clustering method based on machine learning. Figure 3 As shown, including:

[0069] S1: Obtain the message dataset for clustering and classify the data according to the classification method described in Example 1;

[0070] The government text data records the message number, message user, message subject, message time, message details, number of objections and number of likes of each message user; based on the content in the government text document, the user's message details are extracted and data preprocessing, word segmentation, stop word removal and other operations are performed.

[0071] This example includes the message information of 4326 message users, which constitutes the initial data set. The data set format is csv format, as shown in Table 2.

[0072] Table 2 User message heat evaluation information table

[0073]

[0074] After obtaining the message data set for clustering in S1, the message data is first preprocessed according to the methods in steps S1 and S2 in Example 1 to obtain a vector representation of each message, and then classified based on the classification model obtained in Example 1.

[0075] S2: Preprocess the classified data and perform word segmentation and stop word removal. Calculate the similarity between questions by category, cluster questions with a similarity greater than 0.5 into one category, and obtain the number of question categories.

[0076] The specific steps of calculating similarity by category and obtaining the number of question categories include:

[0077] The cosine value of the angle between two vectors in a vector space is used to measure the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0, indicating that the two vectors are more similar. The closer the cosine value is to 0, the closer the angle is to 90 degrees, indicating that the two vectors are less similar.

[0078] First, preprocess the two text data whose similarity needs to be calculated, remove duplicate data and empty data in the data, remove empty data and some symbols in the data including '\t', '\n', ','\r', '\xa0', '\xa9', '\u3000', and perform word segmentation and stop word removal.

[0079] After using the Jieba word segmentation to segment the sentences in the two texts, we get two lists respectively, for example:

[0080] ListA = ['A3', 'District', 'One meter', 'Sunshine', 'Wedding Art', 'Photography', 'Tax']

[0081] ListB = ['A', 'City', 'Subway', 'Violation', 'Employment', 'Problem', 'Question']

[0082] List all the words, put listA and listB in a set, and get:

[0083] Set = {'A3', 'District', 'One meter', 'Sunshine', 'Wedding Art', 'Photography', 'Tax', 'A', 'City', 'Subway', 'Violation', 'Employment', 'Problem', 'Question'}

[0084] Convert the above set to a dict, where the key is the word in the set and the value is the position where the word appears in the set, that is, in the form of 'region':1:

[0085] Dict1 = {'A3': 0, 'District': 1, 'One meter': 2, 'Sunshine': 3, 'Wedding Art': 4, 'Photography': 5, 'Tax': 6, 'A': 7, 'City': 8, 'Subway': 9, 'Violation': 10, 'Employment': 11, 'Problem': 12, 'Question': 13}

[0086] Encode listA and listB, and one-hot encode each word according to its position in dict1:

[0087] listAcodeOneHot=[1,1,1,1,1,1,1,0,0,0,0,0,0,0]

[0088] listBcodeOneHot=[0,0,0,0,0,0,0,1,1,1,1,1,1,1]

[0089] After obtaining the word frequency vectors of the two sentences, it becomes a matter of calculating the cosine value of the angle between the two vectors. The larger the value, the higher the similarity:

[0090]

[0091] S3: Use K-Means clustering again, and set K to the number of question categories obtained in S2;

[0092] The specific steps of using K-Means clustering include:

[0093] (1) First, the text is segmented and stop words are removed. Specifically, the text is preprocessed using the Jieba segmentation algorithm, and the stop word documents on the Internet are used to remove stop words.

[0094] (2) Calculate the text features of each word segment and construct a vector space model.

[0095] Specifically, TF-IDF, or inverse text frequency index, is used to calculate text features. TF-IDF is a statistical method used to evaluate the importance of a word to a document in a corpus. The importance of a word increases in proportion to the number of times it appears in a document, and decreases inversely with the frequency of its appearance in other documents in the corpus. In other words, if a word appears more frequently in a document and not in other documents, it means that the word is very important for the classification of the document. However, if it also appears more frequently in other documents, it means that the word is not very discriminative, and IDF is used to reduce the weight of the word. TF-IDF is proportional to the number of times a word appears in a document, and inversely proportional to the number of times the word appears in the entire language.

[0096] In this embodiment, for a certain type of text, assuming that there are i messages in this type of text, and the message data of this type of text is processed by word segmentation to obtain j words, a matrix A with i rows and j columns is created, each row represents the text feature vector of a message, and each column represents the TF-IDF weight of the word in each message. Matrix A is the text feature vector space model. Specifically, for each message, the word frequency of each word in the message is calculated respectively; then the TF-IDF weight of each word in each message is counted and assigned to the corresponding position in matrix A.

[0097] For each type of government text, the number of problem categories in the type of government text is obtained by the method of step S3, and the number of problem categories is used as the number of clustering categories. The K-Means clustering method is used to cluster the message data in the type of government text.

[0098] The K-Means algorithm is an unsupervised classification algorithm that assumes there is an unlabeled data set:

[0099] x=(x (1) , x (2) , ..., x (m) ) T (7)

[0100] Among them, vector x is matrix A, x (m) Represents the mth word in the message data.

[0101] Cluster the data set into K clusters c = C1, C2, ..., C k , the minimized loss function is:

[0102]

[0103] where μ i Cluster C i The center point:

[0104]

[0105] To find the optimal solution to the above problem, it is necessary to traverse all possible cluster divisions. The K-Mmeans algorithm uses a greedy strategy to obtain an approximate solution. The specific steps are as follows:

[0106] (1) Randomly select K sample points from the sample to serve as the center of each cluster {μ1, μ2, ..., μ k }

[0107] (2) Calculate the distance dist(x) between all sample points and the center of each cluster (i) , μ j ), and then assign the sample point to the nearest cluster x (i) ∈μ nearest

[0108] (3) Recalculate the cluster center based on the existing sample points in the cluster

[0109]

[0110] (4) Repeat steps 2 and 3

[0111] S4: After clustering is completed, the outliers of the number of likes and dislikes are processed, and the popularity of each question is calculated by grouping by the number of question categories;

[0112] The specific steps of obtaining the heat of each type of question include:

[0113] (1) Extract the number of likes and dislikes of the question. If there are abnormal values ​​in the number of likes and dislikes that exceed the normal range, we use a box plot to count the number of likes and dislikes, and set the number less than 0 and the number exceeding the upper quartile plus 1.5 times the interquartile range to 0;

[0114] (2) Calculate the popularity of this type of question:

[0115] In this embodiment, the sum of the number of likes and dislikes plus one is used to represent the popularity of the question. That is:

[0116] Likes + Dislikes +1 (1)

[0117] In essence, it is expressed by the number of participants in the issue. One like or dislike means one person's participation, but the person who posts the message is also counted as one person, so one is added to the sum of the number of likes and dislikes.

[0118] S5: Sort the questions by message time, and finally sort each question by question popularity;

[0119] The specific steps of sorting each type of questions according to question popularity include:

[0120] The clustered questions are first sorted by time within the category, and then sorted from large to small according to the popularity of the category of questions. The top few questions are the ones that the public is most concerned about, and the duration of the question can also be obtained.

[0121] S6: You can get any of the top hot issues, and you can get the issues that are of concern to a specific place or a specific group of people in a certain period of time, or the issues that urgently need to be resolved by the relevant departments, so as to discover problems in time and solve the issues that citizens are most concerned about in time.

[0122] We can get any of the top hot issues, and we can get the issues that are of concern to specific places or specific groups of people within a certain period of time, or the issues that urgently need to be resolved by the relevant departments, so that government workers can grasp the needs of the people and serve the people better and more promptly.

[0123] Furthermore, this embodiment also provides an evaluation method for the relevant departments to respond to the message, including:

[0124] We obtain the message subject, message time, message details, reply opinions and reply time for evaluation; we evaluate the reply opinions of relevant departments to the message based on the relevance and timeliness of the reply; we also perform data preprocessing operations such as deduplication and empty space removal, Chinese word segmentation and stop word filtering on the data in the text;

[0125] We removed stop words from the two texts to be compared and expressed them as vectors after word segmentation. We calculated the cosine similarity between the message details and the reply opinions. A cosine value of 0 means that the message and the reply opinions are completely dissimilar. We also compared the message time and reply time, and regarded the reply time exceeding 1 month from the message time as an indicator of low efficiency. Then we defined evaluation indicators for other correlations greater than 0 and timely replies within 1 month, namely:

[0126]

[0127] h1: message time

[0128] h2: Response time

[0129] text1: message details

[0130] text2: Reply to comments

[0131] Convert the message time and reply time into hours, then divide by the number of hours in a month 744 (a month is calculated as 31 days) to get the percentage decimal plus the sum of the cosine values, and then calculate the average to get the evaluation index of relevance and timeliness. The higher the index, the better.

[0132] After obtaining the major category to which the message data belongs, this embodiment calculates the text similarity between each message and other messages, and obtains the number of question categories based on the similarity threshold. For verification, we also use the K-Means clustering method, setting K as the number of question categories we obtained previously, and displaying all the questions after clustering them. The number of categories K calculated by the similarity can carefully cluster all the questions under this major category.

[0133] When mining hot issues, we do not cluster the issues directly, but cluster them in each type of government text, which can improve clustering efficiency and accuracy. Moreover, when clustering, the number of cluster categories K is not randomly selected, but is calculated based on the similarity between each issue. Issues with similarities greater than a set threshold are classified into one category. The resulting number of issue categories can further improve the clustering effect of the issues.

[0134] After clustering, we defined a heat evaluation index. We can sort hot issues according to the heat evaluation index we defined, and also sort them according to the time of each type of issue, so as to more intuitively reflect the heat and duration of each issue.

[0135] The responses of relevant departments to messages are evaluated in terms of relevance and timeliness, and our evaluation indicators are defined, which can better reflect the quality of the responses of relevant departments by combining relevance and timeliness.

[0136] Embodiment 3

[0137] The purpose of this embodiment is to provide an electronic device.

[0138] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the government text classification method of embodiment 1 or the government text hot issue mining method of embodiment 2 is implemented.

[0139] Embodiment 4

[0140] The purpose of this embodiment is to provide a computer-readable storage medium.

[0141] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the government text classification method of embodiment one or the government text hot issue mining method of embodiment two.

[0142] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0143] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0144] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0145] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.< / pad>

Claims

1. A method for government text classification and hot issue mining based on machine learning, characterized in that: The following steps are involved: Acquire multiple pieces of training government text data and corresponding labels, and construct a coding dictionary; the coding dictionary includes each word in the training government text data and the corresponding code; Obtaining vector representations of the plurality of training government text data based on a coding dictionary; Encode the label data to obtain the vector representation of each label; According to the vector representation of the text data and the corresponding labels, a machine learning model is used to train a government text classification model; the government text classification model is used for government text classification; Acquire multiple pieces of government text data, and obtain the category to which each piece of government text data belongs based on the government text classification model; For a certain type of government text data: Determine the number of categories of message data in the type of government text data based on a similarity analysis method; wherein determining the number of categories of message data in the type of government text data based on a similarity analysis method includes: respectively calculating the pairwise similarity of message details and setting the message details with a similarity greater than a threshold as one category; The message data is clustered using the number of categories as the number of cluster categories, including: clustering using the obtained number of question categories as the K value of K-Means clustering; Combined with the evaluation of the message data, the popularity of the message data is calculated.

2. The method for classifying government texts and mining hot issues based on machine learning as claimed in claim 1, characterized in that: According to the number n of non-repeated characters in the training government text, the characters are encoded from 0 to n-1 to obtain a coding dictionary.

3. The method for classifying government texts and mining hot issues based on machine learning as claimed in claim 1, characterized in that: Encoding the label data involves: Set a unique number for each label, perform one-hot encoding, and obtain the encoding representation of each label.

4. The method for classifying government texts and mining hot issues based on machine learning as claimed in claim 1, characterized in that: Similarity analysis of message data includes: The two messages for which similarity is to be calculated are deduplicated, stop-word removed, and word segmented. The words obtained by word segmentation of the two message data are concatenated together to obtain a concatenated set; Encode the two message data according to their positions in the set to obtain vector representation; The similarity is calculated based on the vector representation of the two message data.

5. The method for classifying government texts and mining hot issues based on machine learning as claimed in claim 1, characterized in that: Clustering message data includes: Construct a text feature matrix with i rows and j columns, where i represents the number of messages in this type of text, j represents the number of words in this type of text that have been segmented, and the mth row and nth column in the matrix represent the TF-IDF weight of the nth word in the mth message; This number of categories is used as the number of clustering categories, and the K-means clustering method is used to cluster the message data according to the text features of each row of message data in the text feature matrix.

6. The method for classifying government texts and mining hot issues based on machine learning as claimed in claim 1, characterized in that: Combined with the evaluation of the message data, the calculation of the popularity of the message data includes: Extract the number of likes and dislikes of the question, and calculate the popularity of the question by adding the sum of the number of likes and dislikes plus one.

7. A government text classification and hot issue mining system based on machine learning using the method according to any one of claims 1 to 6, characterized in that: include: A data acquisition module is configured to acquire a plurality of training government text data and corresponding labels; A dictionary construction module is configured to construct a coding dictionary; the coding dictionary includes each word in the training government text data and the corresponding code; A vector representation module is configured to obtain vector representations of the plurality of training government text data based on a coding dictionary; encode the label data to obtain vector representations of each label; The model training module is configured to train a government text classification model using a machine learning model based on vector representations of text data and corresponding labels; the government text classification model is used for government text classification.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the government text classification and hot issue mining method based on machine learning as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the government text classification and hot issue mining method based on machine learning as described in any one of claims 1 to 6.

Images