A Method and System for Classifying and Grading Power Grid Text Data

Abstract

The present invention relates to a method for hierarchical classification of power grid text data, comprising the following steps: collecting power grid text data, preprocessing and enhancing the text data, and constructing the increased text data into a text data training set; constructing a framework for a hierarchical classification model of power grid text data, and inputting the text data training set into the framework to obtain the probability distribution of variables in the training set; updating the parameters of the framework for the hierarchical classification model of power grid text data according to the probability distribution result of the variables and training again; obtaining a final framework for the hierarchical classification model of power grid text data after training for a preset number of rounds, and performing hierarchical classification on the power grid text data through this framework.

Description

Technical Field

[0001] The present invention relates to a method and system for classifying and grading power grid text data, belonging to the technical field of machine learning recognition and processing. Background Art

[0002] With the increasing dependence on information systems, the amount of data and the importance of data in power grid enterprises are growing day by day, and the risk of data leakage is also increasing. Although equipment manufacturers at home and abroad have developed data leakage prevention systems based on different user requirements and have also formulated preliminary data classification and grading systems and control strategies, there is a lack of data classification and grading standards and control strategy systems for power grid enterprises. Information security construction mainly considers the security of hardware and software. The existing data classification and grading methods in power grid enterprises mainly classify and grade data macroscopically at the company level. The business departments have a low awareness of data security, and the data control center has insufficient grasp of the sensitivity of data. The effect of data leakage prevention work is limited. The existing methods ignore important objects such as personnel, data, documents, services, and intangible assets. As a result, there are various deficiencies and problems in the process of implementing data security work in power grid enterprises. The main problems and deficiencies are as follows:

[0003] Lack of security control measures for the data life cycle. The amount of data in power grid enterprises is huge, and there is a lack of sorting, distinction, and protection between core sensitive data and general data. The life cycle of the flow of core sensitive data is not clear.

[0004] The formulation of the security strategy of the data leakage prevention system has no reference to the data classification and grading standards of power grid enterprises, resulting in the lack of objectivity and sufficiency in the formulation of the security strategy. The data leakage prevention system itself does not comprehensively investigate and count the data of the enterprise and identify the sensitive data that the enterprise needs to protect. The formulation of the security strategy of the data leakage prevention system depends on the built-in strategy of the system and the subjective awareness of technical personnel, lacking standardization.

[0005] Inadequate identification of information assets; due to the lack of criteria for judging data classification and grading in power grid enterprises and no special statistics on the amount of enterprise data, the data leakage prevention system cannot accurately identify whether data assets need to be protected and what protection measures should be taken when formulating security strategies.

[0006] With the rapid development of deep learning, this technology has brought new breakthroughs to text classification. Currently, common text classification algorithms are machine learning and deep learning methods. Machine learning methods achieve classification by manually selecting features and classifiers. Common classical classification algorithms include: Naive Bayes classifier, Support Vector Machine (SVM), K-Nearest Neighbor algorithm, etc. Although traditional machine learning classification algorithms have achieved certain classification effects, they still have certain limitations, such as: weak text feature expression ability, high labor cost, low efficiency, etc. With the rise of deep learning technology, deep learning methods have been proven to be superior to machine learning methods and have achieved important results in the fields of computer vision and natural language processing. Text classification methods based on deep learning avoid manual rule design, save labor costs, and can automatically extract text information using neural network structures, improving the efficiency and accuracy of text classification.

[0007] In recent years, in the field of natural language processing, deep learning technologies and neural network models have developed rapidly. Convolutional Neural Networks (CNN), Recurrent Neural Network (RNN), and Attention mechanism are the main text classification models in deep learning. CNN uses a multi-layer perceptron structure to capture significant features of text. The capture of significant features depends on the size of the convolutional kernel, while CNN ignores the dependencies between local information, which has an important impact on the accuracy of text classification. Compared with CNN, the output value of RNN depends on the input of the current time unit and the output of the previous time unit, and can capture text context information. However, RNN has problems of gradient vanishing and explosion during training. Therefore, researchers have proposed improved RNN models, including Long Short-Term Memory (LSTM) and Gating Recurrent Unit (GRU). LSTM is a special type of RNN, and GRU is an improved version of LSTM with a more optimized and simple structure. Both can overcome the problems of gradient vanishing and explosion by selectively saving information, but there are still problems of long-term dependencies. To address the long-term dependency problem, researchers have proposed neural network structures of Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gating Recurrent Unit (BiGRU). Among them, BiLSTM consists of two layers of LSTM neural networks, which can obtain context dependencies simultaneously from the forward and backward directions of the text sequence, better extract deep features of the text, and thus improve the text classification effect. Attention was initially applied to the field of computer vision and had a good improvement effect on image classification. Later, some researchers introduced it into the field of natural language processing to calculate the importance of different features in the sequence, distinguish significant features and insignificant features of the text, and thus improve the text classification effect.

[0008] However, the above models all use the traditional word vector model: Word2vec model, which cannot reflect the polysemy of words in different contexts. Summary of the Invention

[0009] To solve the problems existing in the above-mentioned prior art, the present invention proposes a method and system for hierarchical classification of power grid text data.

[0010] The technical solution of the present invention is as follows:

[0011] On the one hand, the present invention provides a method for grading and classifying power grid text data, comprising the following steps:

[0012] Collect power grid text data and preprocess the collected text data, perform data augmentation on the preprocessed text data, and construct a text data training set through the augmented text data;

[0013] Construct a power grid text data grading and classification model framework, which is composed of a BERT pre-trained language model, a bidirectional gated recurrent network model, a convolutional neural network model, and an output layer, and fine-tune the text data training set based on the pre-trained parameters of the BERT pre-trained language model;

[0014] Input the fine-tuned text data training set into the BERT pre-trained language model to obtain the semantic information of the text data training set and the probability distribution of variables in the text data training set. Input the semantic information of the text data training set into the bidirectional gated recurrent network model. The bidirectional gated recurrent network model obtains the global semantic features of the text data training set according to the input data. Then input the global semantic features of the text data training set into the convolutional neural network model to obtain the local semantic features of the text data training set. The local semantic features pass through the output layer to obtain the power grid text data grading and classification result;

[0015] The above power grid text data grading and classification model framework has a preset number of training rounds. In each round of training, the collected power grid text data will be preprocessed and data-augmented again, and the parameters of each model in the next round of training will be updated according to the probability distribution of variables in the text data training set output by the BERT pre-trained language model in the previous round of training. When the training rounds are over, a trained power grid text data grading and classification model framework is obtained, and the power grid text data is graded and classified through this framework.

[0016] As a preferred embodiment of the present invention, the text data preprocessing includes data cleaning, word segmentation, and stop word removal;

[0017] In the data cleaning step, a text data cleaning rule is constructed based on regular expressions, and redundant information is deleted and filtered through the text data cleaning rule. At the same time, it is necessary to avoid deleting important information and causing semantic loss;

[0018] The word segmentation step is to split continuous text data into multiple word sequences with semantic units according to specific rules;

[0019] In the stop word removal step, a stop word list is constructed, and useless words in the text data are filtered through character matching based on the stop word list.

[0020] As a preferred embodiment of the present invention, the data augmentation is used to transform and expand the original text to increase the diversity of training data. Specifically:

[0021] The preprocessed text data is enhanced by means of synonym replacement, random insertion, random deletion, random swapping, and text recombination methods.

[0022] As a preferred embodiment of the present invention, the BERT pre-trained language model is composed of a masked language model and a neighboring sentence prediction model;

[0023] The masked language model is used to randomly erase one or several words in any sentence in the dataset, and predict the erased words based on the remaining words in the sentence;

[0024] The neighboring sentence prediction model is used to predict whether two randomly selected sentences in the given dataset are neighboring sentences;

[0025] Through the above two models, the semantic information of the text data training set and the probability distribution of the variables in the text data training set are obtained.

[0026] As a preferred embodiment of the present invention, the bidirectional gated recurrent network model is composed of two independent gated recurrent units, namely a forward gated recurrent unit and a backward gated recurrent unit. At each moment, the two units learn the input data in two opposite directions. At time t, the output value of the bidirectional gated recurrent network model is jointly determined by the forward gated recurrent unit and the backward gated recurrent unit. The calculation formula is as follows:

[0027]

[0028] Where: and respectively represent the forward and backward output states at time t; w t and v t represent the relevant weights; b t represents the bias value; h t represents the output value of the bidirectional gated recurrent network model; G represents the gated recurrent unit.

[0029] As a preferred embodiment of the present invention, the convolutional neural network model is composed of a convolutional layer and a pooling layer;

[0030] The convolutional layer is composed of multiple convolutional kernels. The convolutional kernels use the sliding window method to extract the semantic features of the input data, and different local semantic features are extracted by setting convolutional kernels of different sizes;

[0031] The pooling layer downsamples the output result of the convolutional layer to prevent overfitting, and reduces the vector dimension of the finally output local semantic features by the max pooling method while retaining the context relationship.

[0032] As a preferred embodiment of the present invention, the output layer uses the Dropout method to reduce the dimension of the data input to the convolutional neural network model through a fully connected layer to prevent overfitting, and finally uses the Softmax function as the output layer classifier to obtain the final text data classification result.

[0033] On the other hand, the present invention also provides a power grid text data hierarchical classification system, including a text data preprocessing module, a text data hierarchical classification module, and a classification result output module;

[0034] The text data preprocessing module is used to collect power grid text data and preprocess the collected text data, perform data augmentation on the preprocessed text data, and construct a text data training set through the augmented text data;

[0035] The text data hierarchical classification module includes a BERT pre-trained language model unit, a bidirectional gated recurrent network model unit, a convolutional neural network model unit, and an output unit. Fine-tune the text data training set based on the pre-trained parameters of the BERT pre-trained language model unit, input the fine-tuned text data training set into the BERT pre-trained language model unit to obtain the semantic information of the text data training set and the probability distribution of variables in the text data training set, input the semantic information of the text data training set into the bidirectional gated recurrent network model unit, and the bidirectional gated recurrent network model unit obtains the global semantic features of the text data training set according to the input data, and then input the global semantic features of the text data training set into the convolutional neural network model unit to obtain the local semantic features of the text data training set. The local semantic features pass through the output unit to obtain the power grid text data hierarchical classification result;

[0036] The above-mentioned power grid text data hierarchical classification module presets a plurality of training rounds. In each round of training, the collected power grid text data will be preprocessed and data-augmented again, and the parameters of each model unit in the next round of training will be updated according to the probability distribution of variables in the text data training set output by the BERT pre-trained language model unit in the previous round of training. When the training round ends, the trained power grid text data hierarchical classification module is obtained, and the power grid text data is hierarchically classified through the module rack and the final result is output through the classification result output module.

[0037] On yet another aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method according to any embodiment of the present invention.

[0038] On yet another aspect, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method according to any embodiment of the present invention.

[0039] The present invention has the following beneficial effects:

[0040] 1. The present invention uses a pre-trained language model for feature representation learning, which can capture the global semantic information and local semantic features of text data; through model fine-tuning and prediction, it can efficiently and accurately classify power grid text data; this classification method can improve the accuracy and efficiency of classification, and provide reliable decision-making support for power system management and maintenance.

[0041] 2. The present invention collects text data from multiple power grid-related data sources, and through preprocessing and feature processing, converts it into feature vectors that can be processed by the model. Such data fusion can comprehensively utilize the information of different data sources, improve the comprehensiveness and integrity of classification, and through integrating the features of multi-source data, can better understand and describe the background and features of power grid data, and improve the accuracy and robustness of classification.

[0042] 3. The present invention is based on a pre-trained model and a fine-tuning mechanism, and has good scalability and adaptability. Through pre-training on a large-scale dataset, the model can learn rich semantic information and patterns; then, through fine-tuning on the downstream dataset, the model can better adapt to the power grid text data in a specific domain; this scalability and adaptability enable the method to be applied to power grid systems of different scales and types, and has a certain degree of generality and flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the present invention;

[0044] Figure 2 is a framework diagram of the hierarchical classification model for power grid text data of the present invention;

[0045] Figure 3 is a diagram of the BERT pre-trained language model. DETAILED DESCRIPTION OF THE INVENTION

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] It should be understood that the step numbers used in the text are only for convenience of description and do not limit the execution order of the steps.

[0048] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0049] The terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0050] The term "and / or" refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] Example 1:

[0052] In 2018, pre-trained language models began to emerge, achieving major breakthroughs in the field of natural language processing. The ELMO (Embedding from Language Models) model, GPT (Generative Pre-Trained Transformer) model, and BERT (Bidirectional Encoder Representation from Transformers) model have emerged successively. Among them, the BERT model, a bidirectional encoder based on Transformer, has shown the best performance, refreshing the best performance records in 11 directions in the NLP (Natural Language Processing) field. Yao Ni et al. proposed a research on online review text sentiment classification based on BERT and BiGRU. Experiments have shown that the BERT-BiGRU model has better classification effects compared with the BiGRU model based on Word2vec.

[0053] See Figure 1 , a method for hierarchical classification of power grid text data, comprising the following steps:

[0054] Collect power grid text data and preprocess the collected text data. Perform data augmentation on the preprocessed text data, and construct a text data training set through the augmented text data;

[0055] See Figure 2 , construct a framework for hierarchical classification of power grid text data. The framework is composed of a BERT pre-trained language model, a bidirectional gated recurrent network model, a convolutional neural network model, and an output layer. Fine-tune the text data training set based on the pre-trained parameters of the BERT pre-trained language model;

[0056] The fine-tuned text data training set is input into the BERT pre-trained language model to obtain the semantic information of the text data training set and the probability distribution of variables in the text data training set. The semantic information of the text data training set is input into the bidirectional gated recurrent network model. The bidirectional gated recurrent network model obtains the global semantic features of the text data training set according to the input data. Then, the global semantic features of the text data training set are input into the convolutional neural network model to obtain the local semantic features of the text data training set. The local semantic features pass through the output layer to obtain the hierarchical classification results of the power grid text data;

[0057] The above-mentioned hierarchical classification model framework for power grid text data presets the number of training rounds. In each round of training, the collected power grid text data will be preprocessed and data-augmented again, and the parameters of each model in the next round of training will be updated according to the probability distribution of variables in the text data training set output by the BERT pre-trained language model in the previous round of training. When the training rounds are completed, the trained hierarchical classification model framework for power grid text data is obtained, and the power grid text data is hierarchically classified through this framework.

[0058] In this embodiment, the power grid text data includes text data such as power equipment status reports, fault reports, maintenance records, and name fields.

[0059] As a preferred implementation manner of this embodiment, the text data preprocessing includes data cleaning, word segmentation, and stop word removal;

[0060] Data cleaning: When processing text data samples, information irrelevant or useless to the theme often appears, such as spaces, numbers, HTML tags, and punctuation marks. In text classification experiments, it is crucial to retain the main text information, and data cleaning plays an important role in achieving this goal. By removing unnecessary data, data cleaning helps reduce the impact of noisy data, eliminate redundancy, and improve the overall quality of the text, thereby enabling better model training, testing, and optimization.

[0061] Regular expressions are a commonly used and effective method for text data cleaning, including a series of steps such as constructing cleaning rules suitable for specific requirements, using regular expressions to match rules, deleting redundant information that has been filtered, and reconstructing valid characters to generate new samples. However, great care must be taken when constructing cleaning rules to avoid deleting important information, which may lead to semantic loss and affect subsequent data analysis or model training. When performing data cleaning, it is crucial to retain as much of the original text information as possible. When processing text data samples, information that is irrelevant or useless to the topic often appears, such as spaces, numbers, HTML tags, and punctuation marks. In text classification experiments, retaining the main text information is crucial, and data cleaning plays an important role in achieving this goal. By removing unnecessary data, data cleaning helps reduce the impact of noisy data, eliminate redundancy, and improve the overall quality of the text, thereby enabling better model training, testing, and optimization.

[0062] Word segmentation: Word segmentation is a process of splitting continuous language text into a sequence of words with semantic units according to certain specific rules. Usually, splitting the words or Chinese characters in a sentence or paragraph of a text into words with semantic information according to a fixed rule is only the first step in natural language processing. Chinese word segmentation is a relatively difficult task. Due to the particularity of the Chinese language, there are often multiple possibilities for the division of a character or word. For example, "python programming introduction" can be divided into "python / programming introduction" or "python programming / introduction" and other situations. Therefore, it is necessary to use computer algorithms and natural language processing techniques to achieve this.

[0063] Removing stop words: Generally, the original text will contain some words that have no actual meaning. For example, "also", "then", "is" in Chinese, and "all", "are", "the", "is" in English. These words usually do not have a clear semantic meaning, but they appear frequently in the text. To save storage space and prevent them from interfering with the text expression and feature extraction process. We generally choose to delete these words, and such words are called stop words.

[0064] Removing stop words requires us to first construct a stop word list, and then use methods such as character matching to filter out the useless words in the text. Filtering out meaningless words through regular expressions or other methods can better capture the main features and information of the text, thereby effectively improving the accuracy of the text classification model.

[0065] As a preferred implementation manner of this embodiment, the data augmentation aims to increase the diversity of training data and improve the generalization ability of the model by performing various transformations and expansions on the original text;

[0066] Data augmentation methods can introduce some variations and noises to make the generated new samples semantically consistent while having certain differences. Common data augmentation methods in the natural language field include: Synonym Replacement: Using a dictionary or thesaurus to replace some words in the text with their synonyms to increase the diversity of sentences. Random Insertion: Randomly select a position in the sentence and insert a randomly selected word or phrase to increase the length and diversity of the sentence. Random Deletion: Randomly delete words in the sentence with a certain probability to simulate the situation of noise and missing data. Random Swap: Randomly select two words in the sentence and swap their positions to increase the diversity of sentence structures. Text Shuffling: Randomly shuffle the order of words in the sentence while keeping the order within each word unchanged to generate new sentences.

[0067] These methods can be selected and combined according to the requirements of the task and the dataset to obtain richer and more diverse training data and improve the performance and robustness of the model.

[0068] As a preferred implementation manner of this embodiment, refer to Figure 3 , the BERT model is a pre-trained language representation model proposed by the Google team in 2018. This model mainly consists of two parts: the input layer and the Transformer encoding layer;

[0069] The goal of pre-training is to train text semantic representations containing rich semantic information on a large corpus, and then fine-tune the text semantic representations for specific NLP tasks. BERT designed two tasks: Masked Language Model (MLM) and Next Sentence Prediction (NSP), and pre-trained the model through these two tasks.

[0070] The task of MLM is to randomly remove one or several words from a given sentence and predict the removed word based on the remaining words in the sentence. This pre-training task of the BERT model is similar to the English cloze test questions done by middle school students. The role of MLM is to solve the problem of missing context in the language model. During the BERT training process, it is processed in two steps: In the first step, 15% of the words in the sentence are randomly covered. For example: I likeapple, I[MASK]apple. Here, like is covered; After determining the covered word in the second step, in 80% of the cases, the covered word is replaced with MASK, 10% of the cases, an arbitrary word is randomly selected to replace the covered word, and 10% of the cases, the original word remains unchanged. For example:

[0071] (1) In 80% of the cases, it is replaced with [MASK], I like apple -> I[MASK]apple.

[0072] (2) In 10% of the cases, the covered word is replaced with a random word, I like apple -> I eat apple.

[0073] (3) In 10% of the cases, it remains unchanged, I like apple -> I like apple.

[0074] The task of NSP is to given two sentences A and B in an article, predict whether B is the next sentence of A. During the training process, two consecutive sentences are randomly extracted from the text corpus, and 50% of them are trained with consecutive correct sentence pairs, and the other 50% are trained with incorrect sentence pairs. Combined with the MLM task, it is ensured that the model represents the text semantic information as comprehensively and accurately as possible.

[0075] As a preferred implementation manner of this embodiment, the bidirectional gated recurrent network model (BiGRU) is composed of two independent gated recurrent units (GRU), namely the forward GRU and the backward GRU. The GRUs in both directions are used to extract the global features of the text information, and learn the information in both directions of the text context at each moment to obtain more sufficient information. At time t, the output value of BiGRU is jointly determined by the forward GRU and the backward GRU, and they are independent of each other and do not share parameters. The calculation formula is:

[0076]

[0077] Where: and respectively represent the forward and backward output states at time t; w t and v t represent the relevant weights; b t represents the bias value; h trepresents the output value of the bidirectional gated recurrent network model; G represents the gated recurrent unit.

[0078] As a preferred implementation manner of this embodiment, the convolutional neural network model is composed of a convolutional layer and a pooling layer; the convolutional layer and the pooling layer are used to capture significant features of text information. The convolutional layer is the core part of the CNN and consists of multiple convolutional kernels. The convolutional kernels extract semantic features of the text by means of a sliding window, and the convolutional operation is an important step in feature extraction. More local feature information is extracted by setting different window widths. The role of the pooling layer is to downsample the result of the convolution, reduce the features, and prevent overfitting to a certain extent. Common pooling methods are max pooling and average pooling. The author adopts the max pooling method and inputs the feature vector output by the CNN layer into the max pooling layer, which can reduce the vector dimension of the final sentence, achieve the purpose of dimensionality reduction, and retain the context relationship to a certain extent.

[0079] As a preferred implementation manner of this embodiment, the output layer uses the Dropout method to reduce the dimension of the data input to the convolutional neural network model to prevent overfitting, conducts multi-classification experiments on the text, and finally uses the Softmax function as the output layer classifier to obtain the final text data classification result.

[0080] Embodiment Two:

[0081] A power grid text data hierarchical classification system includes a text data preprocessing module, a text data hierarchical classification module, and a classification result output module;

[0082] The text data preprocessing module is used to collect power grid text data, preprocess the collected text data, perform data augmentation on the preprocessed text data, and construct a text data training set with the augmented text data;

[0083] The text data hierarchical classification module includes a BERT pre-trained language model unit, a bidirectional gated recurrent network model unit, a convolutional neural network model unit, and an output unit. Fine-tune the text data training set based on the pre-trained parameters of the BERT pre-trained language model unit, input the fine-tuned text data training set into the BERT pre-trained language model unit to obtain the semantic information of the text data training set and the probability distribution of variables in the text data training set, input the semantic information of the text data training set into the bidirectional gated recurrent network model unit, the bidirectional gated recurrent network model unit obtains the global semantic features of the text data training set according to the input data, then input the global semantic features of the text data training set into the convolutional neural network model unit to obtain the local semantic features of the text data training set, and the local semantic features pass through the output unit to obtain the power grid text data hierarchical classification result;

[0084] The above power grid text data classification module presets multiple training rounds. In each round of training, the collected power grid text data is preprocessed and data-augmented again, and the parameters of each model unit in the next round of training are updated according to the probability distribution of the variables in the text data training set output by the BERT pre-trained language model unit in the previous round of training. When the training rounds are completed, the trained power grid text data classification module is obtained, and the power grid text data is classified through the module framework and the final result is output through the classification result output module.

[0085] This embodiment is used to implement the functions in Embodiment 1, which will not be elaborated here.

[0086] Embodiment 3:

[0087] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method described in any embodiment of the present invention.

[0088] Embodiment 4:

[0089] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method described in any embodiment of the present invention.

[0090] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0091] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0092] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be repeated here.

[0093] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (hereinafter referred to as ROM), random access memories (hereinafter referred to as RAM), magnetic disks, or optical discs that can store program codes.

[0094] The above are only the embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for classifying and grading power grid text data, characterized in that, It includes the following steps: Collect power grid text data, preprocess the collected text data, perform data augmentation on the preprocessed text data, and construct a text data training set with the augmented text data; The preprocessing of the text data includes data cleaning, word segmentation, and stop word removal; In the data cleaning step, text data cleaning rules are constructed based on regular expressions, and redundant information is deleted and filtered through the text data cleaning rules. At the same time, it is necessary to avoid deleting important information and causing semantic loss; The word segmentation step is to split continuous text data into multiple word sequences with semantic units; In the stop word removal step, a stop word list is constructed, and useless words in the text data are filtered based on the stop word list through character matching; Construct a power grid text data hierarchical classification model framework. The framework consists of a BERT pre-trained language model, a bidirectional gated recurrent network model, a convolutional neural network model, and an output layer. Fine-tune the text data training set based on the pre-trained parameters of the BERT pre-trained language model; Input the fine-tuned text data training set into the BERT pre-trained language model to obtain the semantic information of the text data training set and the probability distribution of variables in the text data training set. Input the semantic information of the text data training set into the bidirectional gated recurrent network model. The bidirectional gated recurrent network model obtains the global semantic features of the text data training set according to the input data. Then input the global semantic features of the text data training set into the convolutional neural network model to obtain the local semantic features of the text data training set. The local semantic features pass through the output layer to obtain the hierarchical classification result of the power grid text data training; The BERT pre-trained language model consists of a masked language model and a next sentence prediction model; The masked language model is used to randomly erase one or several words in any sentence in the dataset, and predict the erased words according to the remaining words in the sentence; The next sentence prediction model is used to predict whether two randomly selected sentences in the given dataset are adjacent sentences; The semantic information of the text data training set and the probability distribution of variables in the text data training set are obtained through the above two models; The above power grid text data hierarchical classification model framework has a preset number of training rounds. In each round of training, the collected power grid text data will be preprocessed and data-augmented again, and the parameters of each model in the next round of training will be updated according to the probability distribution of variables in the text data training set output by the BERT pre-trained language model in the previous round of training. When the training rounds are over, a trained power grid text data hierarchical classification model framework is obtained, and the power grid text data is hierarchically classified through this framework.

2. The hierarchical classification method for power grid text data according to claim 1, characterized in that, The data augmentation is used to transform and expand the original text, increasing the diversity of training data. Specifically: Augment the preprocessed text data through methods such as synonym replacement, random insertion, random deletion, random swapping, and text recombination.

3. A method for hierarchical classification of power grid text data according to claim 1, characterized in that, The bidirectional gated recurrent network model is composed of two independent gated recurrent units, namely, a forward gated recurrent unit and a backward gated recurrent unit. At each moment, the two units learn the input data in both forward and backward directions. At time t, the output value of the bidirectional gated recurrent network model is jointly determined by the forward gated recurrent unit and the backward gated recurrent unit. The calculation formula is as follows: Wherein: and respectively represent the forward and reverse output states at time t; w t and v t represent the relevant weights; b t represents the bias value; h t represents the output value of the bidirectional gated recurrent network model; G represents the gated recurrent unit.

4. A method for classifying power grid text data by level, according to claim 1, characterized in that The convolutional neural network model is composed of a convolutional layer and a pooling layer; The convolutional layer is composed of multiple convolutional kernels. The convolutional kernels use the sliding window method to extract the semantic features of the input data, and different local semantic features are extracted by setting convolutional kernels of different sizes; The pooling layer downsamples the output result of the convolutional layer to prevent overfitting. The maximum pooling method is used to reduce the vector dimension of the finally output local semantic features and retain the context relationship.

5. A method for classifying and grading power grid text data according to claim 1, characterized in that The output layer uses the Dropout method to reduce the dimension of the data input to the convolutional neural network model through a fully connected layer to prevent overfitting. Finally, the Softmax function is used as the output layer classifier to obtain the final text data classification result.

6. A power grid text data grading and classification system for implementing a power grid text data grading and classification method according to any one of claims 1-5, characterized in that, It includes a text data preprocessing module, a text data hierarchical classification module, and a classification result output module; The text data preprocessing module is used to collect power grid text data and preprocess the collected text data, perform data augmentation on the preprocessed text data, and construct a text data training set through the augmented text data; The text data hierarchical classification module includes a BERT pre-trained language model unit, a bidirectional gated recurrent network model unit, a convolutional neural network model unit, and an output unit. Based on the pre-trained parameters of the BERT pre-trained language model unit, the text data training set is fine-tuned. The fine-tuned text data training set is input into the BERT pre-trained language model unit to obtain the semantic information of the text data training set and the probability distribution of the variables in the text data training set. The semantic information of the text data training set is input into the bidirectional gated recurrent network model unit. The bidirectional gated recurrent network model unit obtains the global semantic features of the text data training set according to the input data, and then inputs the global semantic features of the text data training set into the convolutional neural network model unit to obtain the local semantic features of the text data training set. The local semantic features pass through the output unit to obtain the power grid text data hierarchical classification result; The above-mentioned text data hierarchical classification module presets multiple training rounds. In each round of training, the collected power grid text data will be preprocessed and data-augmented again, and the parameters of each model unit in the next round of training will be updated according to the probability distribution of the variables in the text data training set output by the BERT pre-trained language model unit in the previous round of training. When the training rounds are over, the trained power grid text data hierarchical classification module is obtained, and the power grid text data is hierarchically classified through the module frame and the final result is output through the classification result output module.

7. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and capable of running on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 5.

Images