An intelligent classification method for customer electricity safety based on improved TF-IDF

Abstract

This invention provides an intelligent classification method for customer electricity safety based on an improved TF-IDF framework, comprising the following steps: ① Preprocessing: Classifying the text content in inspection work orders into multiple categories and preprocessing it into a text dataset of potential danger content; ② Weight Calculation: Calculating the weight of each word using an improved TF-IDF method; ③ Matrixization; ④ Table Construction; ⑤ Generating a representation vector; ⑥ Classification; and ⑦ Verification. This invention proposes an improved TF-IDF framework for text classification and combines it with word2vec word vectors, preserving semantics while reflecting word importance. An attention mechanism is introduced between the input layer and the convolutional layer. This attention mechanism is used to process the text vectors to generate an attention matrix, resulting in excellent generalization and achieving better classification results.

Description

Technical Field

[0001] The present invention relates to an intelligent classification method for customer electricity safety based on improved TF-IDF. Background Art

[0002] Currently, the rise of deep learning in text classification tasks has solved the problems of high labor costs, poor applicability, and low classification accuracy of traditional machine learning classification methods, but it still has shortcomings. The traditional TF-IDF algorithm does not pay attention to feature items with high category discrimination. If a feature item is evenly distributed in a certain category of documents, it means that this feature item is more representative of the theme of the documents in the category; if a feature item is only distributed in one document of a certain category, it means that this feature item is not universal and cannot represent the theme of the documents in this category. However, the traditional TF-IDF algorithm ignores this point and does not consider the position of the feature item. The position where the feature item first appears and the position where the feature item last appears in a document also contain a lot of effective information. By extracting information related to the position of the first and last feature items, the range of occurrence of the feature item in the document can be reflected, and the importance of the feature item can be more accurately expressed.

[0003] A typical example is a single-cell related technology data analysis method based on natural language processing disclosed in the Chinese invention patent with application number CN202110200765.X. It does not take into account the position factors of the feature items and it is difficult to express the importance of the feature items accurately enough. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent classification method for customer electricity safety based on improved TF-IDF. The intelligent classification method for customer electricity safety based on improved TF-IDF proposes an improved TF-IDF for text classification and combines it with word2vec word vectors, which can not only retain semantics but also reflect the importance of words.

[0005] The present invention is achieved through the following technical solutions.

[0006] The present invention provides a customer electricity safety intelligent classification method based on improved TF-IDF, comprising the following steps:

[0007] ① Preprocessing: The text content in the inspection work order is divided into multiple categories and preprocessed into a hidden danger content text dataset;

[0008] ② Calculate weight: Calculate the weight of each word by improving the TF-IDF method;

[0009] ③Matrixization: Use the word2vec model to convert the original text into a vector matrix;

[0010] ④ Table creation: Create a vocabulary-index mapping table based on the vector matrix, and combine it with the trained word vector file to form a .npz file;

[0011] ⑤Generate representation vector: Represent each word in the training dataset with a word vector and concatenate them to generate a text representation vector;

[0012] ⑥ Classification: Using the text representation vector as input, a convolutional neural network classification model is used to perform text classification.

[0013] ⑦Verification: Use accuracy and F1 value as evaluation indicators on the verification set to verify the effectiveness of the model.

[0014] In the step ①, each category is treated as a whole and then subjected to unstructured processing and then preprocessing.

[0015] In the step ①, the pretreatment includes:

[0016] 1) Word segmentation: Perform word segmentation on each category of hidden dangers in the inspection work order. After word segmentation, the long text content is divided into word-level units;

[0017] 2) Stop word removal: Further process the word segmentation results of the hidden danger content in the inspection work order, remove the low-importance words, and filter out the keywords that are highly important to the text classification results;

[0018] 3) Dataset division: The preprocessed dataset is divided into training set, validation set and test set in a ratio of 7:2:1.

[0019] The improved TF-IDF method is TF-IDF-ICP, which introduces the intra-class factor InterC and the word distance factor DS on the basis of TF-IDF. The calculation formula is:

[0020] TF-IDF-ICP=TF i *IDF i *inteC i *DS i

[0021] Where TF is the average term frequency and IDF is the inverse document frequency.

[0022] The specific calculation steps of TF-IDF-ICP are:

[0023] 1) Calculate the intra-class factor. First, count the number of times feature item i in a certain class j appears in document d. Then calculate the standard deviation of feature item i in class j. Finally, take the inverse of the standard deviation to express the uniformity of the distribution of feature items within the class.

[0024] 2) Calculate the word distance factor, count the number of feature items in the text and find the difference between the position numbers of the last and first appearance of the feature word, and then calculate the ratio of this difference to the number of feature items;

[0025] 3) Calculate the average word frequency TF of the feature item in the document, that is, count the total number of times feature item i appears in the documents contained in category j;

[0026] 4) Calculate the inverse document frequency (IDF) by dividing the total number of documents by the number of documents containing the term, and then taking the logarithm of the quotient;

[0027] 5) Multiply the intra-class factor, word distance factor, word frequency, and inverse word frequency values calculated above to obtain the feature item weight TF-IDF-ICP.

[0028] After step ③, the following steps are also included:

[0029] Combine: The weight of each word obtained in step ② and the vector matrix obtained in step ③ are combined to form a word embedding.

[0030] The convolutional neural network classification model introduces an attention mechanism both before and after the convolution layer.

[0031] After step ⑦, the following steps are also included:

[0032] Post-processing: Based on the verification results, evaluate whether the model has achieved the expected model effect. If so, perform subsequent model predictions. If not, fine-tune the model by adjusting the indicator system and model parameters.

[0033] The model prediction is specifically as follows: using the training model results on the test set to perform text type classification prediction on the test set data to obtain the final customer electricity safety intelligent classification result set.

[0034] The adjustment index system and adjustment model parameters are:

[0035] Adjust the indicator system: Based on the existing indicators, calculate derived variables through the experience of business experts or statistical methods to enrich the indicator system, provide more variables that have a significant impact on the target variable, and improve the accuracy of the model;

[0036] Adjust model parameters: By adjusting the numerical combination of model parameters, the optimal number of model parameters and parameter values are selected according to the existing indicator system and data size.

[0037] The beneficial effects of the present invention are as follows: by proposing an improved TF-IDF for text classification and combining it with word2vec word vectors, it can not only retain semantics but also reflect the importance of words. An attention mechanism is introduced between the input layer and the convolutional layer. The attention mechanism is used to process the text vectors to obtain an attention matrix, which has good generalization ability and can achieve better classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic flow diagram of at least one embodiment of the present invention;

[0039] Figure 2 is a flowchart of at least one embodiment of calculating feature item weights in the present invention;

[0040] Figure 3 It is a schematic diagram of the model structure of the convolutional neural network classification model in the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described below, but the scope of protection claimed is not limited to the description.

[0042] Example 1

[0043] like Figures 1 to 3 The customer electricity safety intelligent classification method based on improved TF-IDF is shown, including the following steps:

[0044] ① Preprocessing: The text content in the inspection work order is divided into multiple categories and preprocessed into a hidden danger content text dataset;

[0045] ② Calculate weight: Calculate the weight of each word by improving the TF-IDF method;

[0046] ③Matrixization: Use the word2vec model to convert the original text into a vector matrix;

[0047] ④ Table creation: Create a vocabulary-index mapping table based on the vector matrix, and combine it with the trained word vector file to form a .npz file;

[0048] ⑤Generate representation vector: Represent each word in the training dataset with a word vector and concatenate them to generate a text representation vector;

[0049] ⑥ Classification: Using the text representation vector as input, a convolutional neural network classification model is used to perform text classification.

[0050] ⑦Verification: Use accuracy and F1 value as evaluation indicators on the verification set to verify the effectiveness of the model.

[0051] Example 2

[0052] Based on Example 1, in step ①, each category is treated as a whole and subjected to unstructured processing, and then preprocessing is performed.

[0053] Example 3

[0054] Based on Example 1, in step ①, the pretreatment includes:

[0055] 1) Word segmentation: Perform word segmentation on each category of hidden dangers in the inspection work order. After word segmentation, the long text content is divided into word-level units;

[0056] 2) Stop word removal: Further process the word segmentation results of the hidden danger content in the inspection work order, remove the low-importance words, and filter out the keywords that are highly important to the text classification results;

[0057] 3) Dataset division: The preprocessed dataset is divided into training set, validation set and test set with a ratio of 7:2:1.

[0058] Example 4

[0059] Based on Example 1, the improved TF-IDF method is TF-IDF-ICP, which introduces the intra-class factor InterC and the word distance factor DS on the basis of TF-IDF. The calculation formula is:

[0060] TF-IDF-ICP=TF i *IDF i *inteC i *DS i

[0061] Where TF is the average term frequency and IDF is the inverse document frequency.

[0062] Example 5

[0063] Based on Example 4, the specific calculation steps of the TF-IDF-ICP are:

[0064] 1) Calculate the intra-class factor. First, count the number of times feature item i in a certain class j appears in document d. Then calculate the standard deviation of feature item i in class j. Finally, take the inverse of the standard deviation to express the uniformity of the distribution of feature items within the class.

[0065] 2) Calculate the word distance factor, count the number of feature items in the text and find the difference between the position numbers of the last and first appearance of the feature word, and then calculate the ratio of this difference to the number of feature items;

[0066] 3) Calculate the average word frequency TF of the feature item in the document, that is, count the total number of times feature item i appears in the documents contained in category j;

[0067] 4) Calculate the inverse document frequency (IDF) by dividing the total number of documents by the number of documents containing the term, and then taking the logarithm of the quotient;

[0068] 5) Multiply the intra-class factor, word distance factor, word frequency, and inverse word frequency values calculated above to obtain the feature item weight TF-IDF-ICP.

[0069] Example 6

[0070] Based on Example 1, after step ③, the following steps are further included:

[0071] Combine: The weight of each word obtained in step ② and the vector matrix obtained in step ③ are combined to form a word embedding.

[0072] Example 7

[0073] Based on Example 1, the convolutional neural network classification model introduces an attention mechanism both before and after the convolution layer.

[0074] Example 8

[0075] Based on Example 1, after step ⑦, the following steps are further included:

[0076] Post-processing: Based on the verification results, evaluate whether the model has achieved the expected model effect. If so, perform subsequent model predictions. If not, fine-tune the model by adjusting the indicator system and model parameters.

[0077] Example 9

[0078] Based on Example 8, the model prediction is specifically as follows: using the training model results on the test set to perform text type classification prediction on the test set data to obtain the final customer electricity safety intelligent classification result set.

[0079] Example 10

[0080] Based on Example 8, the adjustment index system and adjustment model parameters are:

[0081] Adjust the indicator system: Based on the existing indicators, calculate derived variables through the experience of business experts or statistical methods to enrich the indicator system, provide more variables that have a significant impact on the target variable, and improve the accuracy of the model;

[0082] Adjust model parameters: By adjusting the numerical combination of model parameters, the optimal number of model parameters and parameter values are selected according to the existing indicator system and data size.

[0083] Example 11

[0084] Based on the above embodiment, the following steps are specifically included:

[0085] Step 1: Classify the text content in the inspection work order into several categories, treat each category as a whole for unstructured processing, and preprocess the hidden danger content text dataset, such as word segmentation and stop word removal.

[0086] 1) Word segmentation: Perform word segmentation on each category of hidden dangers in the inspection work order. After word segmentation, the long text content is divided into word-level units;

[0087] 2) Stop word removal: Further process the word segmentation results of the hidden danger content in the inspection work order, remove the low-importance words, and filter out the keywords that are highly important to the text classification results;

[0088] 3) Divide the preprocessed dataset into a training set, a validation set, and a test set with a ratio of 7:2:1;

[0089] Step 2: Calculate the weight of each word using the improved TF-IDF method TF-IDF-ICP. This method introduces the intra-class factor InterC and the word distance factor DS on the basis of traditional TF-IDF to improve the defects of the traditional method. The calculation method is as follows:

[0090] TF-IDF-ICP=TF i *IDF i *inteC i *DS i

[0091] 1) Calculate the intra-class factor. First, count the number of times feature item i in a certain class j appears in document d. Then calculate the standard deviation of feature item i in class j. Finally, take the inverse of the standard deviation to express the uniformity of the distribution of feature items within the class.

[0092] 2) Calculate the word distance factor, count the number of feature items in the text and find the difference between the position numbers of the last and first appearance of the feature word, and then calculate the ratio of this difference to the number of feature items;

[0093] 3) Calculate the average word frequency TF of the feature item in the document, that is, count the total number of times feature item i appears in the documents contained in category j;

[0094] 4) Calculate the inverse document frequency (IDF) by dividing the total number of documents by the number of documents containing the term, and then taking the logarithm of the quotient;

[0095] 5) Multiply the above-calculated intra-class factor, word distance factor, word frequency, and inverse word frequency values to obtain the feature item weight TF-IDF-ICP;

[0096] Step 3: Use the word2vec model to convert the original text into a vector matrix;

[0097] Step 4: The weight of each word calculated based on the improved TF-IDF algorithm TF-IDF-ICP is combined with the word vector converted by the word2vec model to form a word embedding, which can preserve the semantics and reflect the importance of the word;

[0098] Step 5: After the word vector model training is completed, it exists in the form of a dictionary. The key is the word and the value is the word vector of the word. In order to facilitate the query of the model word vector, it is necessary to create a vocabulary-index mapping table, and then combine it with the trained word vector file to form a .npz file;

[0099] Step 6: Use the word embedding layer formed by the improved TF-IDF algorithm and word2vec model training, then represent each word in the training dataset with a word vector, and concatenate them to generate a text representation vector. The concatenated text representation vector is used as the input of the convolutional neural network classification model based on the attention mechanism;

[0100] Step 7: An attention mechanism is introduced between the input layer and the convolutional layer. The attention matrix obtained by processing the text vector using the attention mechanism increases the weight of important words in the text, thereby increasing their influence on text classification, while reducing the weight of unimportant words, thereby reducing their influence on text classification;

[0101] 1) Initialize vectors q, w, b;

[0102] 2) Calculate v, where the scoring function is the additive model v = tanh(wx + b);

[0103] 3) Calculate the similarity matching score, exps = vq;

[0104] 4) Normalize to obtain the weight coefficient, the calculation formula is a = exps / sum (exps);

[0105] 5) The weighted sum of the weight coefficient and the input sequence is used to obtain the attention value;

[0106] Step 8: Input the text vector after the attention layer into the convolution layer. After the convolution layer, the attention mechanism is introduced again, and then the pooling operation is performed. The pooling operation is a special convolution operation that further reduces the complexity of the model. The feature vectors obtained after convolution pooling using convolution kernels of different sizes are spliced to extract local features, and finally the text classification result is obtained through the softmax layer;

[0107] Step 7: Use accuracy (acc) and F1 value as evaluation indicators on the validation set to verify the effectiveness of the model. The formula is as follows:

[0108]

[0109]

[0110] Where n is the number of categories, acc i is the accuracy of each category, F1 i is the F1 value of each category;

[0111] Step 8: Based on the validation results of the validation set, evaluate whether the model has achieved the expected model effect. If it has, proceed to the next step of model prediction. If not, tune the model by adjusting the indicator system and model parameters.

[0112] 1) Adjust the indicator system: Based on the existing indicators, calculate derived variables through the experience of business experts or statistical methods to enrich the indicator system, provide more variables that have a significant impact on the target variable, and improve the accuracy of the model;

[0113] 2) Adjust model parameters: By adjusting the numerical combination of model parameters, the optimal number of model parameters and parameter values are selected according to the existing indicator system and data size;

[0114] Step 9: Use the training model results on the test set to perform text type classification prediction on the test set data.

[0115] Therefore, the present invention targets the unstructured text data including specific inspection contents and results generated during the execution of electricity inspection, combines deep learning algorithms, studies the electricity inspection work order text parsing technology, relies on natural language processing technology, improves the TF-IDF algorithm to calculate the weight of each word and combines it with the word2vec word vector, which can not only retain the semantics but also reflect the importance of the word, and splices to generate a text representation vector. The spliced text representation vector is used as the input of the convolutional neural network text classification model based on the attention mechanism. The attention mechanism is introduced between the input layer and the convolution layer. The attention mechanism is used to process the text vector to obtain the attention matrix, thereby increasing the weight of important words in the text and thus increasing the influence of these words on text classification. At the same time, the weight of unimportant words can be reduced, reducing the influence of unimportant words on text classification, and realizing intelligent classification of customer electricity safety.

Claims

1. An intelligent classification method for customer electricity safety based on improved TF-IDF, characterized by: The steps include: ① Preprocessing: The text content in the inspection work order is divided into multiple categories and preprocessed into a hidden danger content text dataset; ② Calculate weight: Calculate the weight of each word by improving the TF-IDF method; ③Matrixization: Use the word2vec model to convert the original text into a vector matrix; After step ③, the following steps are also included: Combine: The weight of each word obtained in step ② and the vector matrix obtained in step ③ are combined to form a word embedding; ④ Table creation: Create a vocabulary-index mapping table based on the vector matrix and combine it with the trained word vector file to form a .npz file; ⑤Generate representation vector: Represent each word in the training dataset with a word vector and concatenate them to generate a text representation vector; ⑥ Classification: Using the text representation vector as input, a convolutional neural network classification model is used to perform text classification. ⑦ Verification: Use accuracy and F1 value as evaluation indicators on the validation set to verify the effectiveness of the model; The improved TF-IDF method is TF-IDF-ICP, which introduces the intra-class factor InterC and the word distance factor DS on the basis of TF-IDF. The calculation formula is: In the formula, TF is the average word frequency, IDF is the inverse document frequency; The specific calculation steps of TF-IDF-ICP are: 1) Calculate the intra-class factor. First, count the number of times feature item i in a certain category j appears in document d. Then calculate the standard deviation of feature item i in category j. Finally, take the inverse of the standard deviation to express the uniformity of the distribution of feature items within the category. 2) Calculate the word distance factor, count the number of feature items in the text, find the difference between the position numbers of the last and first appearances of the feature word, and then calculate the ratio of this difference to the number of feature items; 3) Calculate the average word frequency TF of the feature item in the document, that is, the total number of times the feature item i appears in the documents contained in category j; 4) Calculate the inverse document frequency (IDF) by dividing the total number of documents by the number of documents containing the term, and then taking the logarithm of the quotient. 5) Multiply the intra-class factor, word distance factor, word frequency, and inverse word frequency values calculated above to obtain the feature item weight TF-IDF-ICP.

2. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 1 is characterized by: In the step ①, each category is treated as a whole and then subjected to unstructured processing and then preprocessing.

3. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 1 is characterized by: In the step ①, the pretreatment includes: 1) Word segmentation: Perform word segmentation on each category of hidden dangers in the inspection work order. After word segmentation, the long text content is divided into word-level units. 2) Stop word removal: Further process the word segmentation results of the hidden danger content in the inspection work order, remove low-importance words, and filter out keywords that are highly important to the text classification results; 3) Dataset division: The preprocessed dataset is divided into training set, validation set, and test set in a ratio of 7:2:

1.

4. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 1 is characterized by: The convolutional neural network classification model introduces an attention mechanism both before and after the convolution layer.

5. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 1 is characterized by: After step ⑦, the following steps are also included: Post-processing: Based on the verification results, evaluate whether the model has achieved the expected model effect. If so, perform subsequent model predictions. If not, fine-tune the model by adjusting the indicator system and model parameters.

6. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 5 is characterized by: The model prediction is specifically as follows: using the training model results on the test set to perform text type classification prediction on the test set data to obtain the final customer electricity safety intelligent classification result set.

7. The customer electricity safety intelligent classification method based on improved TF-IDF according to claim 5, characterized in that: The adjustment index system and adjustment model parameters are: Adjust the indicator system: Based on the existing indicators, calculate derived variables through the experience of business experts or statistical methods to enrich the indicator system, provide more variables that have a significant impact on the target variable, and improve the accuracy of the model; Adjust model parameters: By adjusting the numerical combination of model parameters, the optimal number of model parameters and parameter values are selected according to the existing indicator system and data size.

Images