Event text classification method, device and equipment oriented to social governance field
By using word embedding interpretable model and social governance professional dictionary for orthogonal rotation in event text classification in the field of social governance, interpretable word embeddings are generated and attention-weighted, the problems of inefficient and difficult to guarantee the accuracy of event text classification in the prior art are solved, and efficient, accurate and credible event text classification is achieved.
Patent Information
- Application Number
- CN202510132826.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-06
- Publication Date
- 2025-05-30
AI Technical Summary
The prior art is inefficient and difficult to guarantee the classification of event texts in the field of social governance, and the uninterpretation of word embedding makes it difficult to verify and rely on classification results in scenarios requiring high trust.
By obtaining event text data sets in the field of social governance, the word embedding interpretable model is used to extract original word embeddings, and orthogonal rotation is combined with the social governance professional dictionary to generate interpretable word embeddings. Then the interpretable word embedding is focused on, the document embedding is obtained, and the event text classification model is trained using the document embedding and its corresponding original category label.
The generated interpretable word embedding has clear semantics, which improves the efficiency and accuracy of event text classification and ensures the credibility and verifiability of classification results.
Smart Images
Figure CN120067319A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular, to an event text classification method, device, and equipment for the field of social governance. Background Art
[0002] With the rapid development of the urbanization process and the continuous progress of technology, the concept of smart cities has gradually emerged. Smart cities rely on advanced technological means to improve urban management efficiency and the convenience of residents' lives, especially showing broad application prospects at the level of social governance. In today's big data era, the Internet and numerous information management systems generate a large amount of text data every day. Therefore, the application of natural language processing technology has become crucial. Applying technologies such as natural language processing and deep learning to the texts in the field of social governance can significantly improve the operation efficiency of social governance, thereby better serving society and residents.
[0003] Currently, the data processing of social governance events mainly relies on manual classification. This method is not only time-consuming and inefficient but also occupies a large amount of human resources. Moreover, since social event data usually lacks clear labels and is highly specialized, traditional text classification methods are inefficient and difficult to guarantee accuracy when processing this data. In addition, the quality of word embeddings has a great impact on text classification results. Although currently, word embeddings generated by models such as word2vec, BERT (Bidirectional Encoder Representations from Transformers), and GPT (Chat Generative Pre-trained Transformer) can capture rich semantic information and can be used to improve classification accuracy, word embeddings face the problem of non-interpretability. People cannot understand the meaning of each dimension of the embedding vector, which makes it difficult to fully verify and rely on the obtained classification results in practical applications, especially in social governance scenarios that require a high degree of trust. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide an event text classification method, device, and equipment for the field of social governance, which can generate interpretable word embeddings with clear semantics and at the same time improve the efficiency and accuracy of event text classification.
[0005] In a first aspect, the present invention provides an event text classification method for the field of social governance, including:
[0006] Obtain an event text data set in the field of social governance, where the event text data set includes multiple event text data;
[0007] Extract the original word embeddings contained in the event text data through a word embedding interpretable model, and perform orthogonal rotation on the original word embeddings in combination with the social governance professional dictionary to obtain the interpretable word embeddings contained in the event text data;
[0008] Perform attention weighting on the interpretable word embeddings contained in the event text data to obtain the document embedding corresponding to the event text data;
[0009] Determine the original category label corresponding to the document embedding;
[0010] Use the document embedding and its corresponding original category label to train the event text classification model, and the trained event text classification model is used to predict the category to which the new event text data belongs.
[0011] In one implementation, performing orthogonal rotation on the original word embeddings in combination with the social governance professional dictionary to obtain the interpretable word embeddings contained in the event text data includes:
[0012] Determine the target broad category concept to which the original word embeddings contained in the event text data belong among the broad category concepts included in the social governance professional dictionary;
[0013] In combination with the target knowledge base, generate the broad category embedding corresponding to the target broad category concept based on the original word embeddings included in the target broad category concept;
[0014] Through the pre-learned orthogonal rotation matrix, transform the original word embeddings contained in the event text data into the broad category embedding corresponding to the target broad category concept to which the original word embeddings belong, so as to obtain the interpretable word embeddings contained in the event text data.
[0015] In one implementation, in combination with the target knowledge base, generating the broad category embedding corresponding to the target broad category concept based on the original word embeddings included in the target broad category concept includes:
[0016] Generate the broad category embedding corresponding to the target broad category concept according to the following formula:
[0017]
[0018] where L i is the broad category embedding corresponding to the i-th target broad category concept, J is the number of original word embeddings included in the i-th target broad category concept, V ij is the j-th original word embedding in the i-th target broad category concept, Neig(v ij ) is the near synonym of V ij in the target knowledge base, V ijk is the k-th approximate semantic concept of V ij in the target knowledge base, Kij is the number of Neig(v ij ), and α is a constant used to control the contribution strength of the original word embedding and its approximate semantic concepts in the broad category embedding.
[0019] In one implementation, attention weighting is performed on the interpretable word embeddings included in the event text data to obtain a document embedding corresponding to the event text data, including:
[0020] Performing a linear transformation on the interpretable word embeddings included in the event text data through a weight matrix to convert the interpretable word embeddings into key vectors;
[0021] Based on the context vectors and key vectors corresponding to the interpretable word embeddings included in the event text data, determining the attention weights corresponding to the interpretable word embeddings included in the event text data;
[0022] Using the attention weights, performing attention weighting on the interpretable word embeddings included in the event text data to obtain a document embedding corresponding to the event text data.
[0023] In one implementation, the expression of the document embedding is:
[0024]
[0025] where h (d) is the document embedding corresponding to the d-th event text data, n is the total number of interpretable word embeddings included in the document embedding, is the weight of the u-th interpretable word embedding in the document embedding, q is the context vector of the interpretable word embedding, W e is the weight matrix, e u and e v are the u-th and v-th interpretable word embeddings in the document embedding, We e e u and W e e v are the key vectors of the u-th and v-th interpretable word embeddings in the document embedding.
[0026] In one implementation, determining the original category label corresponding to the document embedding includes:
[0027] Performing clustering processing on the document embedding respectively according to multiple alternative clustering numbers to obtain clustering clusters corresponding to each alternative clustering number;
[0028] For each clustering cluster corresponding to each alternative clustering number, perform the following operations: taking the sum of the squares of the distances between all document embeddings and the centers of their respective clustering clusters as the clustering effect evaluation index corresponding to this alternative clustering number;
[0029] Determine the target number of clusters from each alternative number of clusters based on the clustering effect evaluation index;
[0030] Cluster the document embeddings according to the target number of clusters to determine the target cluster to which the document embeddings belong, and use the category of the target cluster to which the document embeddings belong as the original category label corresponding to the document embeddings.
[0031] In one implementation, use the document embeddings and their corresponding original category labels to train the event text classification model, including:
[0032] Extract the word frequency distribution information and context information corresponding to the document embeddings through the convolutional kernel generation network in the event text classification model, and adjust the parameters of the convolutional layer in the event text classification model based on the word frequency distribution information and context information;
[0033] Perform a convolution operation on the document embeddings through the adjusted convolutional layer in the event text classification model;
[0034] Perform global max pooling on the feature information output by the convolutional layer through the global max pooling layer in the event text classification model;
[0035] Determine the category to which the document embeddings belong based on the feature information output by the global max pooling layer through the fully connected layer in the event text classification model;
[0036] Train the event text classification model based on the category to which the document embeddings belong and their corresponding original category labels, and randomly discard neurons in the event text classification model at a preset ratio during the training process until the training stop condition is met.
[0037] In a second aspect, the present invention also provides an event text classification device for the field of social governance, including:
[0038] A data acquisition module for acquiring an event text data set in the field of social governance, where the event text data set contains multiple event text data;
[0039] An interpretable word embedding generation module for extracting the original word embeddings contained in the event text data through a word embedding interpretable model, and performing orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings contained in the event text data;
[0040] A document embedding generation module for performing attention weighting on the interpretable word embeddings contained in the event text data to obtain document embeddings corresponding to the event text data;
[0041] A label determination module for determining the original category label corresponding to the document embeddings;
[0042] A model training module is used to train an event text classification model by using document embeddings and their corresponding original category labels. The trained event text classification model is used to predict the category to which new event text data belongs.
[0043] In a third aspect, the present invention also provides an electronic device, including a processor and a memory. The memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the method according to any one of the first aspect.
[0044] In a fourth aspect, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the method according to any one of the first aspect.
[0045] An event text classification method, device, and equipment provided by the present invention for the field of social governance first obtain an event text data set in the field of social governance, where the event text data set includes multiple event text data; then, through a word embedding interpretable model, extract the original word embeddings included in the event text data, and perform orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings included in the event text data; then perform attention weighting on the interpretable word embeddings included in the event text data to obtain the document embeddings corresponding to the event text data, and determine the original category labels corresponding to the document embeddings; finally, use the document embeddings and their corresponding original category labels to train the event text classification model, and the trained event text classification model is used to predict the category to which new event text data belongs. The above method performs orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings with clear semantics in each dimension, and on this basis, generates document embeddings by using the attention weighting method to capture the deep semantic information of the event text data, which helps to improve the accuracy of event text classification. Finally, use the document embeddings and their corresponding original category labels to train the event text classification model, and the trained event text classification model can efficiently and accurately implement event text classification.
[0046] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims, and drawings.
[0047] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Description of the Drawings
[0048] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0049] Figure 1 It is a schematic flowchart of an event text classification method for the field of social governance provided by an embodiment of the present invention;
[0050] Figure 2 It is an overall flowchart of an event text classification method for the field of social governance provided by an embodiment of the present invention;
[0051] Figure 3 It is a schematic flowchart of obtaining an interpretable word embedding provided by an embodiment of the present invention;
[0052] Figure 4 It is a process diagram of document clustering provided by an embodiment of the present invention;
[0053] Figure 5 It is a structural diagram of an improved TextCNN model provided by an embodiment of the present invention;
[0054] Figure 6 It is a schematic structural diagram of an event text classification device for the field of social governance provided by an embodiment of the present invention;
[0055] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific Embodiments
[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions of the present invention in conjunction with the embodiments. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.
[0057] Currently, traditional text classification methods are inefficient and difficult to guarantee accuracy when processing social event data, and word embeddings face the problem of being non-interpretable, resulting in difficulty in fully verifying and relying on the obtained classification results in social governance scenarios that require a high degree of trust. Based on this, the embodiments of the present invention provide an event text classification method, device, and equipment for the field of social governance, which can generate interpretable word embeddings with clear semantics and at the same time improve the efficiency and accuracy of event text classification.
[0058] For the convenience of understanding this embodiment, first, a method for classifying event texts in the field of social governance disclosed in the embodiments of the present invention will be introduced in detail. Refer to Figure 1 the schematic flowchart of a method for classifying event texts in the field of social governance shown in the figure. This method mainly includes the following steps S102 to S110:
[0059] Step S102, obtain an event text data set in the field of social governance.
[0060] Among them, the event text data set contains multiple event text data. In one example, event text data reported by users can be collected from a social governance platform to form an event text data set.
[0061] Step S104, through a word embedding interpretable model, extract the original word embeddings contained in the event text data, and perform orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings contained in the event text data.
[0062] Among them, the word embedding interpretable model can select the WoBERF pre-trained model. The social governance professional dictionary is also a tool used to assist in understanding and processing natural language in the field of social governance. It usually contains definitions of a wide range of category concepts related to the field of social governance, and the definitions contain some basic concepts. The interpretable word embeddings are word embeddings with clear semantics in each dimension.
[0063] In one example, for each event text data, multiple original word embeddings contained in the event text data are extracted through the word embedding interpretable model, and the target wide category concept to which each original word embedding belongs is determined. Based on each original word embedding and its related synonyms and approximate semantic concepts in the target knowledge base, a wide category embedding corresponding to the target wide category concept to which each original word embedding belongs is generated. In the interpretable space with the wide category embedding as the base vector, each original word embedding is converted into the corresponding interpretable space by using the orthogonal rotation technique, and then the interpretable word embeddings can be obtained.
[0064] Step S106, perform attention weighting on the interpretable word embeddings contained in the event text data to obtain a document embedding corresponding to the event text data.
[0065] In one example, for each event text data, the attention weight corresponding to each interpretable word embedding can be determined according to the key vector and context vector corresponding to each interpretable word included in the event text data. By performing attention weighting on each interpretable word embedding included in the event text data according to the attention weight, the document embedding corresponding to the event text data can be obtained. Among them, the attention weight is used to measure the contribution of each interpretable word embedding to the final document embedding, and the key vector is obtained by performing a linear transformation on the interpretable word embedding.
[0066] Step S108: Determine the original class label corresponding to the document embedding.
[0067] In one example, multiple alternative clustering numbers can be set in advance. After clustering according to the alternative clustering numbers, the clustering effects of each alternative clustering number are evaluated based on the sum of the squares of the distances from all document embeddings to the centers of their respective clustering clusters, so as to screen out the target clustering data with the best clustering effect. Then, the document embeddings are clustered according to the target clustering number to determine the original class label corresponding to each document embedding.
[0068] Step S110: Use the document embedding and its corresponding original class label to train the event text classification model. The trained event text classification model is used to predict the class to which the new event text data belongs.
[0069] Among them, the event text classification model can select an improved TextCNN model. The improved TextCNN model includes a convolutional layer with dynamically adjustable parameters, a global max pooling layer, a Dropout layer, and a fully connected layer. In one example, the document embedding is used as the model input, and the improved TextCNN model is used to predict the class to which the document embedding belongs. The improved TextCNN model is trained based on the class to which the document embedding belongs and its corresponding original class label. After the model is trained, the document embedding corresponding to the new event text data is generated according to the foregoing steps, so as to use the trained improved TextCNN model to output the class to which the new event text data belongs for the document embedding corresponding to the new event text data.
[0070] The event text classification method for the social governance field provided by the embodiments of the present invention orthogonally rotates the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings with clear semantics in each dimension, and on this basis, generates document embeddings in an attention-weighted manner to capture the deep semantic information of the event text data, which helps to improve the accuracy of event text classification. Finally, the event text classification model is trained using the document embedding and its corresponding original class label, and the trained event text classification model can efficiently and accurately implement event text classification.
[0071] For ease of understanding, an embodiment of the present invention provides a specific implementation manner of an event text classification method for the field of social governance. Refer to Figure 2 the overall flowchart of an event text classification method for the field of social governance shown in Figure 2 which is explained in the model training stage and the model application stage. The model training stage includes the following steps: collecting the events reported by users from the social governance platform as the event text data set; after removing duplicate events, performing word segmentation and removing stop words; using the word embedding interpretable model to extract interpretable word embeddings; using attention weighting to generate document embeddings; using a clustering algorithm to obtain the clustering category of each event and writing it as a label into the event text data set; training the improved TextCNN model. The model application stage includes the following steps: receiving newly reported events; performing word segmentation and removing stop words; generating document embeddings; and outputting the category to which the document embedding belongs through the trained improved TextCNN model.
[0072] The embodiment of the present invention first explains the steps in the model training stage, and provides a specific implementation manner of an event text classification method for the field of social governance in the model training stage, including steps 1 to 4:
[0073] Step 1, interpretable word embeddings:
[0074] After cleaning and normalizing the event text data, an advanced WoBERT pre-trained language model is used to obtain the embeddings of each word in the event text data (i.e., the original word embeddings). However, in view of the professionalism and depth of the field of social governance, a social governance professional dictionary is further integrated to enhance the semantic information of the original word embeddings. The social governance professional dictionary contains definitions of a wide range of categories, and some basic concepts are included in the definitions. In addition, in order to improve the interpretability of the original word embeddings, an orthogonal rotation technique is also introduced. Refer to Figure 3 the schematic flowchart of a process for obtaining interpretable word embeddings shown in
[0075] Step 1a, the social governance platform obtains relevant event text data.
[0076] Step 1b, after data cleaning, use the WoBERT pre-trained model to obtain the original word embeddings.
[0077] Step 1c: Use the HowNet knowledge base to expand the broad category concepts in the social governance professional dictionary to obtain broad category embeddings that integrate semantic information. Specifically, it includes: determining the original word embeddings contained in the event text data and the target broad category concepts to which they belong in the broad category concepts contained in the social governance professional dictionary; combining the target knowledge base (such as the HowNet knowledge base), and generating broad category embeddings corresponding to the target broad category concepts based on the original word embeddings contained in the target broad category concepts.
[0078] Taking the definition of "community governance" as an example, community governance: "As the basic unit of social governance, community governance focuses on the needs and rights protection of grass-roots people, and advocates the improvement of residents' self-governance and self-service capabilities."
[0079] The definition of social governance also includes other concepts such as "grass-roots people" and "residents". Since the number of concepts contained in these classification definitions is limited, this inevitably leads to a rough estimate of their classification significance. Therefore, the HowNet knowledge base is borrowed here to expand this concept set. HowNet is a common sense knowledge base that takes the concepts represented by Chinese and English words as the description objects and reveals the relationships between concepts and the attributes of concepts. For example, the concept of "people" mentioned above can obtain its synonymous concepts such as "masses", "common people", and "people" in the HowNet knowledge base. Based on the concepts in the broad category concepts and the synonyms of the concepts in the external knowledge base, broad category embeddings are obtained.
[0080] After determining the target broad category concepts to which each original word embedding contained in the event text data belongs, the broad category embeddings corresponding to the target broad category concepts can be generated according to the following formula:
[0081]
[0082] where, L i is the broad category embedding corresponding to the i-th target broad category concept, J is the number of original word embeddings contained in the i-th target broad category concept, V ij is the j-th original word embedding in the i-th target broad category concept, Neig(v ij ) is the synonym of V ij in the target knowledge base, V ijk is the k-th approximate semantic concept of V ij in the target knowledge base, K ij is the number of Neig(v ij ), and α is a constant, for example, set to 0.2 according to experience, which is used to control the contribution intensity of the original word embedding and its approximate semantic concepts in the broad category embedding. The broad category embeddings rich in deep semantic information can be obtained by the above method.
[0083] Step 1d, an interpretable space with the broad category embeddings as the basis vectors.
[0084] Step 1e, transform the original word embeddings into the interpretable space through an orthogonal rotation matrix to obtain interpretable word embeddings. Specifically, it includes: through the pre-learned orthogonal rotation matrix, transform the original word embeddings contained in the event text data into the broad category embeddings corresponding to the target broad category concept to which the original word embeddings belong, so as to obtain the interpretable word embeddings contained in the event text data.
[0085] The embodiment of the present invention provides a specific implementation manner for learning the orthogonal rotation matrix. During the process of learning the orthogonal rotation matrix, it is desired that each dimension among the basis vectors is independent of each other, so the identity matrix is used as the learning target of the interpretable space, and thus the following constraints should be satisfied:
[0086]
[0087] where M represents the transformation matrix to be learned, L represents the matrix of the broad category embeddings, and I represents the identity matrix. At the same time, in order to ensure that more original information can be retained after the embeddings are rotated, an orthogonality constraint is imposed on the orthogonal rotation matrix:
[0088]
[0089] At the same time, orthogonal rotation can eliminate the redundancy between dimensions, making each dimension more clearly represent a specific semantic feature. Finally, according to the learned orthogonal rotation matrix, the original word embeddings are transformed into the interpretable space. In this interpretable space, each dimension of the word embeddings represents the semantics of a broad category.
[0090] Step 2, document clustering:
[0091] After obtaining the interpretable word embeddings, obtain the document embeddings through the attention weighting mechanism, and then use the K-means algorithm to cluster the event text data to initially obtain the categories of each event. Refer to Figure 4 A document clustering process diagram as shown. After tokenizing the event text annual dataset and obtaining the interpretable word embeddings using the pre-trained interpretable word embeddings, perform Steps 2a to 2d:
[0092] Step 2a, perform attention weighting on the interpretable word embeddings to obtain the document embeddings.
[0093] In natural language processing tasks, representing a document embedding as a weighted average of interpretable word embeddings is a common approach. However, the importance of interpretable word embeddings for document semantics varies, so directly taking the average may not fully capture the actual semantics of the document. To better represent the semantics of the document, embodiments of the present invention introduce an attention mechanism.
[0094] (1) Linearly transform the interpretable word embeddings contained in the event text data through a weight matrix to convert the interpretable word embeddings into key vectors.
[0095] In one example, the key vector K u is obtained by linearly transforming the current word embedding, and the process of the linear transformation is as follows:
[0096] K u = We e e u ;
[0097] where W e is a weight matrix used to transform the embedding vector to the dimension of the key vector; e u is the u-th interpretable word embedding in the document embedding.
[0098] (2) Based on the context vectors and key vectors corresponding to the interpretable word embeddings contained in the event text data, determine the attention weights corresponding to the interpretable word embeddings contained in the event text data.
[0099] In one example, the attention weight a u is used to measure the contribution of each interpretable word embedding to the final document embedding, and the attention weight a u can be determined according to the following formula:
[0100]
[0101] where n is the total number of interpretable word embeddings contained in the document embedding, e u , e v are the u-th and v-th interpretable word embeddings in the document embedding, K u , K v are the key vectors of the u-th and v-th interpretable word embeddings in the document embedding, and q is the context vector (i.e., query vector) of the interpretable word embedding.
[0102] (3) Use the attention weights to perform attention weighting on the interpretable word embeddings contained in the event text data to obtain the document embedding corresponding to the event text data.
[0103] In one example, use the interpretable word embedding e i , and perform attention weighting on it to obtain the document embedding h:
[0104]
[0105] Among them, h (d) is the document embedding corresponding to the d-th event text data, n is the total number of interpretable word embeddings included in the document embedding, a u is the attention weight of the u-th interpretable word embedding in the document embedding, is the u-th interpretable word embedding in the document embedding corresponding to the d-th event text data.
[0106] Combining the formulas (1) to (3) above, the following expression of the document embedding can be obtained:
[0107]
[0108] Among them, h (d) is the document embedding corresponding to the d-th event text data, n is the total number of interpretable word embeddings included in the document embedding, is the weight of the u-th interpretable word embedding in the document embedding, q is the context vector of the interpretable word embedding, W e is the weight matrix, e u and e v are the u-th and v-th interpretable word embeddings in the document embedding, W e e u and W e e v are the key vectors of the u-th and v-th interpretable word embeddings in the document embedding.
[0109] Step 2b: Use the elbow method to obtain the target number of clusters for all events.
[0110] (I) Cluster the document embeddings according to multiple alternative numbers of clusters respectively to obtain the clusters corresponding to each alternative number of clusters. In one example, K-means clustering can be used to cluster the document embeddings according to each alternative number of clusters respectively to obtain the clusters corresponding to each alternative number of clusters.
[0111] (II) Perform the following operations on the clusters corresponding to each alternative number of clusters: Take the sum of the squares of the distances between all document embeddings and the centers of their respective clusters as the clustering effect evaluation index corresponding to this alternative number of clusters.
[0112] Among them, the clustering effect evaluation index can adopt the Sum of Squared Errors (SSE). In one example, the elbow method is used to determine the target number of clusters for the text dataset. The elbow method is a graphical method for determining the number of data clusters. It identifies an obvious inflection point by calculating the loss function under different numbers of clusters. This inflection point indicates that the increase in the number of clusters no longer significantly reduces the loss function value, and thus is regarded as the optimal number of clusters. The elbow method calculates the Sum of Squared Errors (SSE) for each alternative number of clusters, that is, the sum of the squares of the distances from all documents to the center of their respective clusters:
[0113]
[0114] where k represents the density of clustering (i.e., the number of clusters), C i represents all document embeddings in the i-th cluster, x represents a document embedding in this cluster, and μ i is the mean of all document embeddings in all C i . The elbow method calculates the SSE corresponding to each k for different alternative numbers of clusters k (such as from 1 to a relatively large value, such as 10 or 15) using the clustering algorithm.
[0115] (III) Determine the target number of clusters from each alternative number of clusters based on the clustering effect evaluation index.
[0116] In one example, although a lower SSE value usually indicates a better clustering effect, excessive pursuit of a lower SSE value may lead to overfitting. Therefore, the point where the SSE value decreases and flattens out should be selected as the optimal target number of clusters to avoid the overfitting problem caused by excessive refinement.
[0117] Step 2c, use K-means clustering to obtain the original class labels for each event.
[0118] In one example, use the K-means algorithm to cluster the document embeddings according to the target number of clusters to determine the target cluster to which the document embedding belongs, and use the class of the target cluster to which the document embedding belongs as the original class label corresponding to the document embedding. It should be noted that here it is necessary to manually determine the name of each original class label according to experience.
[0119] Step 2d, write the clustering categories into the original event text dataset to obtain an event text dataset with original class labels.
[0120] Step 3, train the TextCNN classification model:
[0121] Combined with the characteristics of the reported event data: Domain-specific terms: The event text contains a large number of domain-specific terms and proper nouns, which are crucial for classification; Uneven text lengths: The length of the event description may vary from short sentences to detailed reports; High information density: The event text is usually information-dense, containing rich context and details.
[0122] In the embodiment of the present invention, the TextCNN model is selected as the event text classification model. It is a convolutional neural network (CNN) model specifically designed for text data, aiming to extract text features through word embedding, convolutional layers, pooling layers, and fully connected layers. See Figure 5 The structure diagram of an improved TextCNN model shown, including convolutional layers, pooling layers, Dropout layers, fully connected layers, and output layers. The embodiment of the present invention has made the following improvements to the TextCNN model architecture: (1) Dynamic convolution kernel adjustment: Introduce a convolution kernel generation network, which can adopt a generative adversarial network to dynamically adjust the size and weight of the convolution kernel according to the actual content of the input document embedding, so as to better adapt to the characteristics of the document embedding. First, extract the word frequency distribution and context information of the input document embedding, and then map the extracted features to the parameter space of the convolution kernel through a small neural network. During the training process, this network will adjust the convolution kernel size and weight in real time according to the data features of the current batch, enabling the model to flexibly adapt to different data characteristics. (2) Use global max pooling instead of local max pooling in the pooling layer to summarize the features of the entire document embedding. Global max pooling pools the entire feature map and selects the maximum value in each channel as a representative, thereby capturing the most significant features in the entire feature map. This can not only summarize the information of the entire feature map but also more comprehensively retain the important features in the document embedding. (3) To prevent model overfitting, the Dropout technique is introduced in the fully connected layer. Set the dropout ratio to 0.6, that is, randomly discard 60% of the neurons in each iteration of training, and the remaining 40% of the neurons will continue to participate in the calculation. It can effectively reduce the over-dependence of the model on the training data and enhance the generalization ability of the model.
[0123] Based on the above improved TextCNN model, the embodiment of the present invention provides a specific implementation method for training the improved TextCNN model, including steps 3a to step 3e:
[0124] Step 3a, through the convolutional kernel generation network in the event text classification model, extract the word frequency distribution information and context information corresponding to the document embedding, and adjust the parameters of the convolutional layer in the event text classification model based on the word frequency distribution information and context information. In one example, a generative adversarial network can be used as the convolutional kernel generation network to adjust the parameters of the convolutional layer based on the word frequency distribution information and context information. The adjusted parameters include the size and weight of the convolutional kernel.
[0125] Step 3b, perform a convolution operation on the document embedding through the adjusted convolutional layer in the event text classification model.
[0126] Step 3c, through the global max pooling layer in the event text classification model, perform global max pooling on the feature information output by the convolutional layer. In one example, the feature information output by the convolutional layer is input to the global max pooling layer, and pooling is performed on each channel of the feature information, and the maximum value is selected from each channel as the feature information output by the global max pooling layer.
[0127] Step 3d, through the fully connected layer in the event text classification model, determine the category to which the document embedding belongs based on the feature information output by the global max pooling layer.
[0128] Step 3e, train the event text classification model based on the category to which the document embedding belongs and its corresponding original category label, and randomly discard neurons in the event text classification model at a preset ratio during the training process until the training stop condition is met. Among them, the training stop condition can be reaching a preset number of iterations or loss convergence.
[0129] Through the above improvements, when this model processes the event texts reported by the social governance platform, it can effectively improve the classification accuracy and adapt to the diversity and complexity of event text data. The trained event text classification model can automatically classify newly reported events and automatically assign the events to the corresponding processing personnel, thereby reducing the workload of manual discrimination and improving work efficiency.
[0130] In summary, in the embodiments of the present invention, first, in combination with the social governance professional dictionary and the orthogonal rotation technology, an interpretable word embedding vector of the text is obtained. Then, attention weighting is performed on the word embedding to obtain the document embedding. Next, clustering is performed on the document embedding to obtain the original category label. Finally, the text embedding and its corresponding original category label are input into the improved TextCNN classification model for training. The trained model can classify newly arrived event data. The embodiments of the present invention have at least the following characteristics:
[0131] 1. In the embodiments of the present invention, in combination with the social governance professional dictionary and the orthogonal rotation technology, interpretable word embeddings with clear semantics in each dimension are obtained, thereby improving the credibility and accuracy of classification.
[0132] 2. The embodiments of the present invention combine word embedding technology and attention mechanism to generate document embeddings, so as to capture the deep semantic information of the document embeddings and improve the accuracy of classification.
[0133] 3. The embodiments of the present invention improve the architecture of the TextCNN classification model in combination with the data characteristics of the reported events, so as to improve the accuracy of classification.
[0134] Based on the foregoing embodiments, the embodiments of the present invention provide an event text classification device for the field of social governance. Refer to Figure 6 the structural schematic diagram of an event text classification device for the field of social governance shown in
[0135] A data acquisition module 602, configured to acquire an event text data set in the field of social governance, where the event text data set includes multiple event text data;
[0136] An interpretable word embedding generation module 604, configured to extract the original word embeddings included in the event text data through a word embedding interpretable model, and perform orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain the interpretable word embeddings included in the event text data;
[0137] A document embedding generation module 606, configured to perform attention weighting on the interpretable word embeddings included in the event text data to obtain the document embeddings corresponding to the event text data;
[0138] A label determination module 608, configured to determine the original category label corresponding to the document embedding;
[0139] A model training module 610, configured to train an event text classification model by using the document embedding and its corresponding original category label, and the trained event text classification model is used to predict the category to which the new event text data belongs.
[0140] The event text classification device provided by the embodiments of the present invention performs orthogonal rotation on the original word embeddings in combination with a social governance professional dictionary to obtain interpretable word embeddings with clear semantics in each dimension, and on this basis, adopts an attention weighting method to generate document embeddings to capture the deep semantic information of the event text data, which helps to improve the accuracy of event text classification. Finally, the event text classification model is trained by using the document embedding and its corresponding original category label, and the trained event text classification model can efficiently and accurately implement event text classification.
[0141] In one implementation, the interpretable word embedding generation module 604 is specifically configured to:
[0142] Determine the target broad category concept to which the original word embeddings contained in the event text data belong among the broad category concepts included in the social governance professional dictionary;
[0143] Combine the target knowledge base, and based on the original word embeddings included in the target broad category concept, generate broad category embeddings corresponding to the target broad category concept;
[0144] Through the pre-learned orthogonal rotation matrix, transform the original word embeddings contained in the event text data into the broad category embeddings corresponding to the target broad category concept to which the original word embeddings belong, so as to obtain interpretable word embeddings contained in the event text data.
[0145] In one implementation manner, the interpretable word embedding generation module 604 is specifically configured to:
[0146] Generate broad category embeddings corresponding to the target broad category concept according to the following formula:
[0147]
[0148] where L i is the broad category embedding corresponding to the i-th target broad category concept, J is the number of original word embeddings included in the i-th target broad category concept, V ij is the j-th original word embedding in the i-th target broad category concept, Neigh(v ij ) is the synonym of V ij in the target knowledge base, V ijk is the k-th approximate semantic concept of V ij in the target knowledge base, K ij is the number of Neigh(v ij ), and α is a constant used to control the contribution intensity of the original word embedding and its approximate semantic concept in the broad category embedding.
[0149] In one implementation manner, the document embedding generation module 606 is specifically configured to:
[0150] Perform a linear transformation on the interpretable word embeddings contained in the event text data through a weight matrix to convert the interpretable word embeddings into key vectors;
[0151] Based on the context vector and key vector corresponding to the interpretable word embeddings contained in the event text data, determine the attention weights corresponding to the interpretable word embeddings contained in the event text data;
[0152] Use the attention weights to perform attention weighting on the interpretable word embeddings contained in the event text data to obtain the document embedding corresponding to the event text data.
[0153] In one implementation manner, the expression of the document embedding is:
[0154]
[0155] Among them, h (d) is the document embedding corresponding to the d-th event text data, n is the total number of interpretable word embeddings included in the document embedding, is the weight of the u-th interpretable word embedding in the document embedding, q is the context vector of the interpretable word embedding, W e is the weight matrix, e u and e v are the u-th and v-th interpretable word embeddings in the document embedding, W e e u and W e e v are the key vectors of the u-th and v-th interpretable word embeddings in the document embedding.
[0156] In one implementation, the label determination module 608 is specifically configured to:
[0157] Cluster the document embeddings respectively according to multiple alternative cluster numbers to obtain cluster clusters corresponding to each alternative cluster number;
[0158] For the cluster clusters corresponding to each alternative cluster number, perform the following operations: Take the sum of the squares of the distances between all document embeddings and the centers of their respective cluster clusters as the clustering effect evaluation index corresponding to this alternative cluster number;
[0159] Determine the target cluster number from each alternative cluster number based on the clustering effect evaluation index;
[0160] Cluster the document embeddings according to the target cluster number to determine the target cluster cluster to which the document embedding belongs, and use the category of the target cluster cluster to which the document embedding belongs as the original category label corresponding to the document embedding.
[0161] In one implementation, the model training module 610 is specifically configured to:
[0162] Extract the word frequency distribution information and context information corresponding to the document embedding through the convolutional kernel generation network in the event text classification model, and adjust the parameters of the convolutional layer in the event text classification model based on the word frequency distribution information and context information;
[0163] Perform a convolution operation on the document embedding through the adjusted convolutional layer in the event text classification model;
[0164] Perform global max pooling on the feature information output by the convolutional layer through the global max pooling layer in the event text classification model;
[0165] Through the fully connected layer in the event text classification model, determine the category to which the document embedding belongs based on the feature information output by the global max pooling layer;
[0166] Based on the category to which the document embedding belongs and its corresponding original category label, train the event text classification model, and randomly discard neurons in the event text classification model at a preset ratio during the training process until the training stop condition is met.
[0167] The device provided in the embodiments of the present invention has the same implementation principle and the same technical effects as those in the foregoing method embodiments. For the sake of brief description, for the parts not mentioned in the device embodiments, reference may be made to the corresponding content in the foregoing method embodiments.
[0168] The embodiments of the present invention provide an electronic device. Specifically, the electronic device includes a processor and a storage device; a computer program is stored on the storage device, and when the computer program is run by the processor, it executes the method according to any one of the foregoing implementation manners.
[0169] Figure 7 FIG. 13 is a schematic structural diagram of an electronic device provided in an embodiment of the present invention. The electronic device 100 includes: a processor 70, a memory 71, a bus 72, and a communication interface 73. The processor 70, the communication interface 73, and the memory 71 are connected through the bus 72; the processor 70 is used to execute an executable module stored in the memory 71, such as a computer program.
[0170] Among them, the memory 71 may include a high-speed random access memory (RAM, Random Access Memory), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 73 (which may be wired or wireless), a communication connection is established between the system network element and at least one other network element, and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used.
[0171] The bus 72 may be an ISA bus, a PCI bus, an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 7 only a bidirectional arrow is used in FIG. 13, but it does not mean that there is only one bus or one type of bus.
[0172] Among them, the memory 71 is used to store a program. After receiving an execution instruction, the processor 70 executes the program. The method executed by the device defined by the flow process disclosed in any one of the foregoing embodiments of the present invention can be applied to the processor 70 or implemented by the processor 70.
[0173] The processor 70 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 70 or the instructions in the form of software. The above-mentioned processor 70 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP for short), an application specific integrated circuit (ASIC for short), a field-programmable gate array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed and completed by a hardware decoding processor, or executed and completed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 71, and the processor 70 reads the information in the memory 71 and combines its hardware to complete the steps of the above method.
[0174] The computer program product of the readable storage medium provided by the embodiments of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the method described in the foregoing method embodiments. For the specific implementation, reference can be made to the foregoing method embodiments and will not be elaborated herein.
[0175] If the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.
[0176] Finally, it should be noted that the above-mentioned embodiments are only specific implementation manners of the present invention, used to illustrate the technical solutions of the present invention, rather than limiting it. The protection scope of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present invention can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or perform equivalent replacements on some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims
1. A method for classifying event texts in the field of social governance, characterized in that: include: Acquire an event text dataset in the field of social governance, where the event text dataset includes multiple event text data; Extract the original word embedding contained in the event text data through the word embedding interpretable model, and perform orthogonal rotation on the original word embedding in combination with the social governance professional dictionary to obtain the interpretable word embedding contained in the event text data; Performing attention weighting on the interpretable word embedding contained in the event text data to obtain a document embedding corresponding to the event text data; Determine the original category label corresponding to the document embedding; The event text classification model is trained using the document embedding and the corresponding original category label, and the trained event text classification model is used to predict the category to which the new event text data belongs.
2. The event text classification method for the social governance field according to claim 1 is characterized in that: The original word embedding is orthogonally rotated in combination with the professional dictionary of social governance to obtain the interpretable word embedding contained in the event text data, including: Determine the target broad category concept to which the original word embedding contained in the event text data belongs among the broad category concepts contained in the social governance professional dictionary; In combination with a target knowledge base, based on the original word embeddings contained in the target broad category concept, a broad category embedding corresponding to the target broad category concept is generated; The original word embedding contained in the event text data is converted into the broad category embedding corresponding to the target broad category concept to which the original word embedding belongs through the orthogonal rotation matrix obtained by pre-learning, so as to obtain the interpretable word embedding contained in the event text data.
3. The event text classification method for the social governance field according to claim 2 is characterized in that: In combination with the target knowledge base, based on the original word embeddings contained in the target broad category concept, a broad category embedding corresponding to the target broad category concept is generated, including: Generate the broad category embedding corresponding to the target broad category concept according to the following formula: Among them, L i is the broad category embedding corresponding to the i-th target broad category concept, J is the number of original word embeddings contained in the i-th target broad category concept, V ij is the jth original word embedding in the i-th target broad category concept, Neig(v ij ) is V ij The synonyms in the target knowledge base, V ijk V ij The kth approximate semantic concept in the target knowledge base, K ij For Neig(v ij ), α is a constant used to control the contribution strength of the original word embedding and its approximate semantic concepts in the broad category embedding.
4. The event text classification method for the social governance field according to claim 1 is characterized in that: Attention weighting is performed on the interpretable word embedding contained in the event text data to obtain a document embedding corresponding to the event text data, including: Performing a linear transformation on the interpretable word embeddings contained in the event text data through a weight matrix to convert the interpretable word embeddings into key vectors; Determine, based on the context vector corresponding to the interpretable word embedding contained in the event text data and the key vector, an attention weight corresponding to the interpretable word embedding contained in the event text data; The attention weights are used to perform attention weighting on the interpretable word embeddings contained in the event text data to obtain document embeddings corresponding to the event text data.
5. The event text classification method for the social governance field according to claim 1 or 4 is characterized in that: The document embedding expression is: Among them, h (d) is the document embedding corresponding to the dth event text data, n is the total number of interpretable word embeddings contained in the document embedding, is the weight of the u-th interpretable word embedding in the document embedding, q is the context vector of the interpretable word embedding, W e is the weight matrix, e u 、e v is the uth and vth interpretable word embedding in the document embedding, W e e u , W e e v is the key vector of the uth and vth interpretable word embedding in the document embedding.
6. The event text classification method for the social governance field according to claim 1 is characterized in that: Determining the original category label corresponding to the document embedding includes: According to multiple candidate clustering numbers, clustering processing is performed on the document embeddings respectively to obtain cluster clusters corresponding to each candidate clustering number; For each cluster corresponding to the candidate cluster number, the following operation is performed: the sum of the squares of the distances between all the document embeddings and the centers of the clusters to which they belong is used as a clustering effect evaluation index corresponding to the candidate cluster number; Determining a target number of clusters from each of the candidate numbers of clusters based on the clustering effect evaluation index; The document embedding is clustered according to the target cluster number to determine the target cluster to which the document embedding belongs, and the category of the target cluster to which the document embedding belongs is used as the original category label corresponding to the document embedding.
7. The event text classification method for the social governance field according to claim 1 is characterized in that: Using the document embedding and the corresponding original category label, training an event text classification model includes: Extract word frequency distribution information and context information corresponding to the document embedding by generating a network through a convolution kernel in the event text classification model, and adjust the parameters of the convolution layer in the event text classification model based on the word frequency distribution information and the context information; Performing a convolution operation on the document embedding through the adjusted convolution layer in the event text classification model; Performing global maximum pooling processing on the feature information output by the convolution layer through the global maximum pooling layer in the event text classification model; Determine the category to which the document embedding belongs based on the feature information output by the global maximum pooling layer through the fully connected layer in the event text classification model; Based on the category to which the document embedding belongs and the corresponding original category label, the event text classification model is trained, and during the training process, neurons in the event text classification model are randomly discarded according to a preset ratio until a training stop condition is met.
8. An event text classification device for the field of social governance, characterized in that: include: A data acquisition module is used to acquire an event text data set in the field of social governance, wherein the event text data set includes multiple event text data; An interpretable word embedding generation module is used to extract the original word embedding contained in the event text data through a word embedding interpretable model, and perform orthogonal rotation on the original word embedding in combination with a professional dictionary of social governance to obtain an interpretable word embedding contained in the event text data; A document embedding generation module, configured to perform attention weighting on the interpretable word embeddings contained in the event text data to obtain a document embedding corresponding to the event text data; A label determination module, used to determine the original category label corresponding to the document embedding; The model training module is used to train the event text classification model using the document embedding and its corresponding original category label. The trained event text classification model is used to predict the category to which the new event text data belongs.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.