A method for automatic annotation and classification of environmental complaint and report texts

Abstract

The present invention provides an automatic annotation and classification method for environmental complaint and report texts, which belongs to the field of text feature extraction and text classification. The present invention extracts feature trigger words corresponding to different fields from a large amount of unlabeled data, and constructs a corresponding field feature dictionary; the dictionary and text similarity calculation can be used to automatically annotate texts in different fields, convert unlabeled data into labeled data, and use it to train a neural network to obtain a suitable classification model. Compared with the prior art, the present invention reduces the errors caused by manually annotated data, saves manpower and time; avoids the problems of low accuracy, complex model, high computing power requirements, etc. brought by unsupervised learning methods; the constructed classification model has strong generalization ability, and can be continuously upgraded and improved after the model is deployed online.

Description

Technical Field

[0001] The present invention relates to the field of text feature extraction and text classification, and in particular to a method for automatic labeling and classification of environmental complaint and report texts. Background Art

[0002] There are many types of supervision and reporting related to the environment. Classifying complaints and reports, quickly and accurately locating the causes of complaints, and then handing them over to the relevant competent departments will improve the efficiency of event handling to a certain extent. However, based on the data currently obtained, the vast majority of complaint data are unlabeled. Combined with the characteristics of the complaint and report texts being highly colloquial and unclear in expression, directly building a classifier faces many difficulties; and manual labeling is expensive, time-consuming and labor-intensive, and is easily affected by the subjective influence of the labeler. Therefore, the present invention proposes to combine automatic labeling with a deep learning model to achieve the task of automatic classification of complaint and report texts.

[0003] The essence of complaint and report classification is text classification. In the field of text classification, the current related work is mainly divided into two methods: classification methods based on knowledge engineering and classification methods based on machine learning. Methods based on knowledge engineering require manual judgment of the characteristics of each classification with the assistance of domain experts and the provision of recognition rules. This method is time-consuming and labor-intensive, and is highly dependent on the level of experts. With the continuous development of science and technology, many scholars have proposed the use of machine learning methods to achieve text classification, and have realized the automatic classification function of complaint and report texts in specific fields, saving manpower and material resources to a certain extent.

[0004] At present, in the field of text classification research, there are not many studies on the classification of complaint and report texts, and they are only concentrated in a few specific fields. There is no precedent for special research on environmental complaint and report texts, and the automatic classification of environmental complaint and report texts still faces the following problems:

[0005] 1) Problems in different fields and industries vary. For example, complaints in the telecommunications industry often include a large number of related professional terms such as calls, mobile phone signals, and network quality; while environmental complaints are more about descriptions of air pollution, noise pollution, etc. Therefore, the universality of the classification model established based on a certain field is poor.

[0006] 2) The complaint texts have problems such as lengthy sentences, unclear expressions, and arbitrary grammar. For example, the complaint texts about the water environment are likely to include words such as "sewage" and "pollution", which cannot reflect the specific causes of water pollution.

[0007] 3) In the field of water environment, combined with the data obtained from relevant departments, only a small part of the data is labeled with pollution source categories, and the classification mode is relatively extensive, with only four categories: "industrial wastewater", "mine wastewater", "domestic wastewater", and "others". This classification system cannot quickly and accurately locate the problem according to its classification, which brings certain difficulties to the relevant staff who deal with pollution incidents.

[0008] In summary, the vast majority of environmental complaint and report data are unlabeled, which brings great difficulties to the text classification task. The traditional manual labeling method is expensive, time-consuming and labor-intensive, and is easily affected by the subjective influence of the labeler. Directly using deep learning networks to build classifiers will face the problems of complex models and huge computational workload. Therefore, it is of great significance to invent a method for automatic labeling and classification of environmental complaint and report texts.

[0009] The colloquial and unstructured nature of complaint and report texts means that supervised learning methods that rely solely on a small number of labeled samples often do not have strong generalization capabilities. Choosing unsupervised learning that does not rely on labels requires the problem of building a complex deep neural network. Moreover, these unlabeled data come from different fields, and these different fields have their own unique ways of expressing knowledge. For machine learning models, to solve the above problems, a lot of computing power and time are needed to learn the unique features of the field, and the performance of the trained model cannot be guaranteed. Summary of the invention

[0010] In view of the above problems, the present invention provides a method for automatically labeling and classifying environmental complaint and report texts, which can solve the problem that environmental complaint and report data labels are missing and cannot be automatically classified according to pollution source types. The method specifically includes the following steps:

[0011] S1: Randomly sample from massive unlabeled data to form a sample set of a certain size;

[0012] S2: Preprocess the sample data, including Chinese word segmentation, removing stop words, and filtering out short texts.

[0013] S3: Use the Single-Pass algorithm to cluster texts with similar descriptions to obtain various topic clusters;

[0014] S4: Calculate the information entropy of each word in each cluster, extract the words with large information content and can more clearly reflect the main problems of the complaint, and construct a domain key dictionary;

[0015] S5: Through similarity calculation, synonyms are extracted from the Wikipedia corpus, the domain key dictionary is expanded, a large number of unregistered words are further included, and the coverage of the domain key dictionary is improved;

[0016] S6: Use domain key dictionaries as the basis for automatic classification to annotate large amounts of unlabeled data;

[0017] S7: Use the deep learning network TextCNN to extract features from the data, perform supervised learning, and build a classification model. Compared with the prior art, the present invention constructs a special dictionary for complaints and reports, which can realize the function of labeling a large amount of unlabeled data, and creates sample data for supervised learning. Compared with direct unsupervised learning, this method reduces the complexity of the deep learning network model and the amount of computational complexity for extracting features. The present invention can extract feature trigger words corresponding to different fields from a large amount of unlabeled data, and construct a corresponding field feature dictionary; the dictionary and text similarity calculation can be used to automatically annotate texts in different fields, convert unlabeled data into labeled data, and use it to train the neural network to obtain a suitable classification model. The beneficial effects of the present invention are:

[0018] (1) Reduce the errors caused by manual data annotation, saving manpower and time;

[0019] (2) It avoids the problems of low accuracy, complex models, and high computing power requirements brought about by unsupervised learning;

[0020] (3) The constructed classification model has strong generalization ability and can be continuously upgraded and improved after the model is deployed online.

[0021] The present invention takes water environment complaints and reports as an example. Therefore, in the example, in the above method, BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the process of the automatic labeling and classification method proposed in the present invention

[0023] Figure 2 This is the structure diagram of the TextCNN network model DETAILED DESCRIPTION

[0024] The following will be combined with the accompanying drawings of the present invention, taking the water environment complaint and reporting data as an example, to fully describe the technical route included in the present invention. The embodiments described herein are only part of the embodiments of the present invention. Therefore, all other embodiments obtained by other researchers in this field without creative work are within the scope of protection of the present invention.

[0025] See also Figure 1 , which is a flow chart of the automatic labeling and classification method proposed by the present invention, comprising the following steps:

[0026] Step S1: Randomly sample from massive unlabeled data to form a sample set of a certain size;

[0027] Step S2: Preprocess the sample data, including Chinese word segmentation and stop word removal.

[0028] Step S3: Use the Single-Pass algorithm to cluster texts with similar descriptions to obtain various topic clusters;

[0029] Step S4: extract words that can clearly reflect the main issues of the complaint and construct a domain key dictionary;

[0030] Step S5: extract synonyms from the Wikipedia corpus through similarity calculation, expand the domain key dictionary, further include a large number of unregistered words, and improve the coverage of the domain key dictionary;

[0031] Step S6: Using the domain key dictionary as the basis for automatic classification to label a large amount of unlabeled data;

[0032] Step S7: Extract features from the data through the deep learning network TextCNN, perform supervised learning, and build a classification model.

[0033] In the present invention, step S2 pre-processing includes:

[0034] Step S201 uses Jieba, a third-party Python library, to perform Chinese word segmentation on the original corpus;

[0035] Step S202 introduces a Chinese stop word list to remove some meaningless function words in the corpus to improve work efficiency;

[0036] Step S203 filters out sentences with a length of less than 5 words after steps S201 and S202, effectively removing short and meaningless complaint report texts to avoid affecting the accuracy of subsequent models.

[0037] Step S3 is a clustering process, the purpose of which is to classify the unlabeled complaint report data according to its internal structure, attributes, and information. In an embodiment of the present invention, the data is divided into different clusters according to the type of pollution source in the water environment complaint report text, so as to facilitate subsequent feature extraction from different clusters. The process mainly includes the following steps:

[0038] Step S301: From the input text sequence D=D(d1, d2, ..., d n ) select the first text d1 as the first topic cluster c1;

[0039] Step S302, select the second text d2, calculate the similarity between d2 and all the texts in c1 according to the following formula and find the average to get Sim avg (d2, c1);

[0040]

[0041] In the above formula, w 1i With w 2i are vectors representing documents d2 and c1 respectively;

[0042] Step S303: If Sim avg If (d2, c1) is greater than the similarity threshold Th, d2 is classified into topic cluster c1 and the process goes to step S305; otherwise, the process goes to step S304;

[0043] Step S304, create a new topic cluster c2 based on d2, and jump to step S306;

[0044] Step S305, take d3 and repeat step S302

[0045] Step S306, take d3, calculate the similarity between d3 and all the current topic clusters, that is, all the texts in c1 and c2, and find the average, and get Sim avg (d3,c1),Sim avg (d3, c2)

[0046] Step S307, if MAX(Sim avg (d3,c1),Sim avg (d3, c2))≥Th, d3 is included in the topic cluster with the maximum similarity value, otherwise a new topic cluster is created with d3;

[0047] Step S308, waiting for the next document to enter.

[0048] Steps S4 and S5 are respectively to construct and expand the domain feature dictionary. In the present invention, the function of the domain feature dictionary is to realize the automatic annotation function.

[0049] Step S401: Perform word frequency statistics on the topic clusters obtained in step S3 to obtain the frequency of all words in the i-th cluster, denoted as c i =c(p1, p2, ..., p n ), where p j (j=1, 2, ..., n) represents the frequency of occurrence of the jth word in the cluster, i.e., the TF value;

[0050] Step S402, calculate c according to formula (2) i The IDF value of all words in , denoted by ci =c(q1,q2,...,q n ), where q j (j=1, 2, ..., n) represents the IDF value of the jth word in the cluster;

[0051]

[0052] Where N represents the total number of texts in the corpus, and N(x) represents the total number of texts containing word x.

[0053] Step S403, combining the word frequency and TF-IDF value of each word, screen out a batch of domain feature words with high word frequency and clear features, and use these words as seed words to build a cell word library.

[0054] These domain feature words are obtained through clustering and word frequency statistics of complaint and report text sets. Due to the small size of the data set itself, the seed word library contains fewer domain feature words and cannot fully describe the domain attributes. Therefore, on the basis of the seed word library established using the complaint and report text set, the word vector model trained with the Wikipedia corpus is introduced to expand synonyms, which can make the word library richer and more comprehensive.

[0055] Furthermore, in step S5, the domain feature words in the seed word library obtained in step S4 are traversed, and the similarity value between the domain feature words in the external word library and the seed word library is calculated according to formula (3). Words above the threshold are filled into the dictionary to further enrich the dictionary.

[0056]

[0057] Among them, S(D1, D2) represents the text similarity between any two documents D1 and D2, w 1i With w 2i are vectors representing documents D1 and D2 respectively.

[0058] The domain-related words in the domain dictionary obtained in step S4 are used as the cell dictionary, and all existing seed words in the dictionary are traversed to obtain words with similarity to the seed words higher than the threshold value to fill in the domain feature dictionary, so that the domain dictionary is further enriched. At this point, the domain feature dictionary is constructed.

[0059] Furthermore, the purpose of step S6 is to annotate the unlabeled data through the domain feature dictionary. The detailed process of annotation is as follows:

[0060] Step S601, set a candidate list R, an unlabeled text sequence D = D (d1, d2, ..., d n), similarity threshold Th, domain dictionary Dict = {K1: V1, K2: V2, ..., K9: V9}, where {K i :V i , i∈1,2,...,9} represents the i-th group of pollution source types and their corresponding trigger phrase sets;

[0061] Step S602, select text d1, calculate d1 and trigger phrase V i7 The similarity values ​​of all trigger words in Sim max(1，i) , represents the maximum similarity between the first document and the i-th group of trigger words, which represents the similarity between d1 and V i degree of association;

[0062] Step S603: If Sim max(1，i) ≥Th, then V i Corresponding pollution source category K i Add it to the alternative list R, otherwise it means that the text is not very relevant to the current pollution source category.

[0063] Step S604, changing the value of i and repeating step S601 until the calculation is completed;

[0064] Step S605: if the candidate list R is still empty after traversing the i value, it means that the text cannot be annotated with limited information and needs manual processing.

[0065] Step S606: If the candidate list is not empty, select the largest Sim max(1，i) value, its corresponding K i Represents the pollution type that is most obviously associated with d1;

[0066] Step S607, mark d1 as K i , complete this round of automatic labeling and proceed to the next round of cycle.

[0067] Step S608: The final output is a label sequence K corresponding to the text sequence order = {d1: K1, d2: K2, ...d n :K n}, where d i K i (i=1, 2, ..., n) represents the i-th text d i The corresponding label name K i .

[0068] Furthermore, the purpose of step S7 is to train the neural network using the labeled text data automatically annotated by step S6. The present invention uses TextCNN as a neural network to further extract high-dimensional features from the complaint report text, and constructs a TextCNN model consisting of three one-dimensional convolutional layers with different convolution kernel sizes and three maximum pooling layers, which can improve the accuracy and generalization ability of classification.

[0069] The main network structure of the deep neural network TextCNN constructed in the present invention is as shown in the attached figure. Figure 2 As shown, it includes input layer, convolution layer, pooling layer and output layer.

[0070] Step S701, the first layer of the neural network model is the input layer. The input layer input is an n*k matrix, where n is the number of words in the complaint report text after preprocessing, and k is the word vector dimension corresponding to each word. In order to make the vector length consistent, the vectorized text needs to be padded (Padding) so that the length of each sentence is n, and sentences that are too long are truncated. Each complaint report text can be expressed as:

[0071]

[0072] in, represents the k-dimensional word embedding of the i-th word in the sentence, The operator represents concatenating each word vector. For sentences with a vocabulary size less than n, k-dimensional zero vectors are used for padding; for sentences with a vocabulary size greater than n, they are truncated so that each sentence can be represented as an n*k matrix.

[0073] In step S702, the second layer of the neural network model is a convolutional layer. In a complaint report text, the correlation between adjacent words is always high, so the features in the sentence can be extracted through one-dimensional convolution. Figure 2 As shown in the figure, this paper designs three convolution kernels of different sizes, namely 3×k, 4×k, and 5×k. The width of the convolution kernel is fixed to the word vector dimension k. Convolution kernels of different heights can help the model extract features in contexts of different lengths.

[0074] In step S703, the third pooling layer of the neural network model aims to reduce the feature dimension and compress the data of the vector obtained by convolution, so as to reduce overfitting and improve the fault tolerance of the model. Figure 2 As shown in the figure, this paper adopts the maximum pooling operation, that is, the largest feature is selected from the feature vector generated after the convolution operation, and then the features are concatenated to achieve the goal of pooling sentences of different lengths into fixed-length vector representations.

[0075] Step S704, the last fully connected softmax layer of the neural network model can output the probability of the expected classification category.

[0076] Step S705 uses the data annotated by the automatic annotation method of the present invention as input to implement supervised learning, thereby improving various performance indicators of the model.

[0077] In summary, the present invention proposes to extract coarse-grained feature words from large quantities of unlabeled data through unsupervised clustering, and then expand the feature words through semantic similarity calculation with the help of the Wikipedia corpus, construct a domain feature dictionary, and use similarity calculation to achieve automatic labeling of unlabeled data; the neural network model trained using the automatically labeled data has better versatility and higher accuracy.

[0078] The solution for automatic labeling and classification of complaints and reports presented by the present invention helps relevant departments save manpower and material resources, and quickly and accurately locate the problem, so as to arrange personnel in a targeted manner to properly handle it.

Claims

1. A method for automatically labeling and classifying environmental complaint and report texts, characterized by: The specific steps include: S1: Randomly sample from massive unlabeled data to form a sample set of a certain size; S2: Preprocess the sample data, including Chinese word segmentation, stop word removal, and filtering out short texts; S3: Use the Single-Pass algorithm to cluster texts with similar descriptions to obtain various topic clusters; S4: Calculate the information entropy of each word in each cluster, extract the words with large information content and reflecting the main problems of the complaint, and construct a domain key dictionary; S5: Through similarity calculation, synonyms are extracted from the Wikipedia corpus, the domain key dictionary is expanded, a large number of unregistered words are further included, and the coverage of the domain key dictionary is improved; S6: Use domain key dictionaries as the basis for automatic classification to annotate large amounts of unlabeled data; S7: Extract features from data through the deep learning network TextCNN, perform supervised learning, and build a classification model; The steps S3, S4 and S5 are together the process of constructing a domain feature dictionary; step S3 uses the Single-Pass algorithm to cluster a small amount of data, extract coarse-grained feature words, and construct a seed word library; the specific steps are: Step S301: From the input text sequence D=D(d1, d2, ..., d n ) select the first text d1 as the first topic cluster c1; Step S302, select the second text d2, calculate the similarity between d2 and all the texts in c1 according to the following formula and find the average to get Sim avg (d2, c1); In the above formula, w 1i With w 2i are vectors representing documents d2 and c1 respectively; Step S303: If Sim avg If (d2, c1) is greater than the similarity threshold Th, d2 is classified into topic cluster c1 and the process goes to step S305; otherwise, the process goes to step S304; Step S304, create a new topic cluster c2 based on d2, and jump to step S306; Step S305, take d3 and repeat step S302; Step S306, take d3, calculate the similarity between d3 and all the current topic clusters, that is, all the texts in c1 and c2, and find the average, and get Sim avg (d3,c1),Sim avg (d3, c2); Step S307, if MAX(Sim avg (d3,c1),Sim avg (d3, c2))≥Th, d3 is included in the topic cluster with the maximum similarity value, otherwise a new topic cluster is created with d3; Step S308, waiting for the next document to enter; Steps S4 and S5 are respectively to construct and expand the domain feature dictionary, and the function of the domain feature dictionary is to realize the automatic annotation function; Step S401: Perform word frequency statistics on the topic clusters obtained in step S3 to obtain the frequency of all words in the i-th cluster, denoted as c i =c(p1, p2, ..., p n ), where p j (j=1,2,...,n) represents the frequency of occurrence of the jth word in the cluster, i.e., the TF value; Step S402, calculate c according to formula (2) i The IDF value of all words in , denoted by c i =c(q1,q2,...,q n ), where q j (j=1, 2, ..., n) represents the IDF value of the jth word in the cluster; Where N represents the total number of texts in the corpus, and N(x) represents the total number of texts containing word x; Step S403, combining the word frequency and TF-IDF value of each word, screen out a batch of domain feature words with high word frequency and clear features, and use these words as seed words to build a cell word library.

2. The method for automatically labeling and classifying environmental complaint and reporting texts according to claim 1 is characterized by: The pre-processing process of step S2 includes: S201 uses Jieba, a third-party Python library, to perform Chinese word segmentation on the original corpus; S202 introduces a Chinese stop word list to remove some meaningless function words in the corpus and improve work efficiency; S203 filters out sentences whose length is less than 5 words after steps S201 and S202.

3. According to the method for automatically labeling and classifying environmental complaint and report texts as claimed in claim 1, when expanding the seed word library, it is characterized by: In step S5, all existing seed words in the seed word library established in step S4 are traversed, and words with high similarity to the seed words in the Wikipedia corpus are obtained and filled into the domain feature dictionary.

4. A method for automatically labeling and classifying environmental complaint and report texts according to claim 1, characterized in that: The step 6 annotates the unlabeled data through the domain feature dictionary. The detailed process of the annotation is as follows: Step S601, set a candidate list R, an unlabeled text sequence D = D (d1, d2, ..., d n ), similarity threshold Th, domain dictionary Dict = {K1: V1, K2: V2, ..., K9: V9}, where {K i :V i , i∈1,2,...,9} represents the i-th group of pollution source types and their corresponding trigger phrase sets; Step S602, select text d1, calculate d1 and trigger phrase V i The similarity values ​​of all trigger words in Sim max(1，i) , represents the maximum similarity between the first document and the i-th group of trigger words, which represents the similarity between d1 and V i degree of association; Step S603: If Sim max(1，i) ≥Th, then V i Corresponding pollution source category K i Add it to the alternative list R, otherwise it means that the text is not very relevant to the current pollution source category; Step S604, changing the value of i and repeating step S601 until the calculation is completed; Step S605: if after traversing the i value, the candidate list R is still empty, it means that the text cannot be annotated with limited information and needs manual processing; Step S606: If the candidate list is not empty, select the largest Sim max(1，i) value, its corresponding K i Represents the pollution type that is most obviously associated with d1; Step S607, mark d1 as K i , complete this round of automatic labeling and proceed to the next round of cycle; Step S608: The final output is a label sequence K corresponding to the text sequence order = {d1: K1, d2: K2, ...d n :K n }, where d i K i (i=1,2,...,n) represents the i-th text d i The corresponding label name K i .

5. The method for automatically labeling and classifying environmental complaint and reporting texts according to claim 1 is characterized by: The step S7 uses the labeled text data automatically annotated in step S6 to train the neural network, and uses TextCNN as a neural network to further extract high-dimensional features in the complaint report text to construct a TextCNN model consisting of three one-dimensional convolutional layers with different convolution kernel sizes and three maximum pooling layers. The specific steps are as follows: Step S701, the first layer of the neural network model is the input layer; the input layer input is an n*k matrix, where n is the number of words in the complaint text after preprocessing, and k is the word vector dimension corresponding to each word; Padding is performed on the vectorized text so that the length of each sentence is n. Sentences that are too long are truncated. Each complaint report text can be expressed as: in, represents the k-dimensional word embedding of the i-th word in the sentence, The operator means to concatenate each word vector; for sentences with a vocabulary size less than n, k-dimensional zero vectors are used for padding; for sentences with a vocabulary size greater than n, they are truncated so that each sentence is represented as an n*k matrix; In step S702, the second layer of the neural network model is a convolution layer. In a complaint report text, the correlation between adjacent words is always high, so one-dimensional convolution is used to extract features in the sentence. There are three different sizes of convolution kernels, namely 3×k, 4×k, and 5×k. The width of the convolution kernel is fixed to the word vector dimension k. Convolution kernels of different heights help the model extract features in contexts of different lengths. In step S703, a maximum pooling operation is used, that is, the largest feature is selected from the feature vector generated after the convolution operation, and then the features are concatenated to achieve the goal of pooling sentences of different lengths into fixed-length vector representations; Step S704, the last fully connected softmax layer of the neural network model outputs the probability of the expected classification category; Step S705 uses the data annotated by the automatic annotation method as input to implement supervised learning, thereby improving various performance indicators of the model.

Images