Power Grid Project Feature Attribute Identification Method Based on Deep Learning
Through the deep learning-based grid project feature attribute recognition method, the BERT model, Bi-GRU and CRF model are used to solve the problem of grid project document management, efficient feature extraction and recognition are achieved, and project management efficiency and investment efficiency are improved.
Patent Information
- Application Number
- CN202111194442.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-10-13
AI Technical Summary
The existing technology is difficult to meet the high-quality lean management and control requirements of power grid project documents and the difficulty in extracting feature, especially in the management of the large number of power grid project documents and cumbersome content, it is difficult to achieve accurate demand positioning and intelligent solution screening.
The network project feature attribute recognition method based on deep learning is adopted, including establishing a project labeling system, text preprocessing, entity annotation, and using BERT model, Bi-GRU neural network and CRF model to establish a network learning model, and perform entity recognition and feature extraction.
Through the entity identification model, project management efficiency is improved, management costs are reduced, standardized management of power grid project documents and investment efficiency are achieved, and manpower and time resources are saved.
Smart Images

Figure CN113869055B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of named entity recognition and deep learning, and specifically relates to a method for identifying characteristic attributes of power grid projects based on deep learning. Background Art
[0002] In the management of power grid companies, the summarization and compilation of historical power grid project documents is of the highest importance, and of course the greatest difficulty. With the help of efficient document project feature extraction technology, each company can focus on precise control, quality improvement and efficiency improvement, and thus put forward high-quality and lean development requirements for comprehensive planning management, clarify the "profit points" and "loss points" of investment, accurately control the investment in the power grid, reduce unnecessary investment, and improve the effectiveness of capital investment. In the document feature extraction stage, it is necessary to achieve accurate demand positioning, intelligent solution screening, and accurate process control, so as to improve project quality and management efficiency. Summary of the invention
[0003] In view of the fact that the existing technology is difficult to meet the requirements of high-quality lean management and control of power grid project documents and the difficulty in feature extraction, the present invention proposes a method for identifying power grid project feature attributes based on deep learning. In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:
[0004] A method for identifying characteristic attributes of power grid projects based on deep learning, comprising the following steps:
[0005] S1, summarize the project business characteristics according to the categories of power grid project documents and project business management requirements, and establish a project label system based on the project business characteristics;
[0006] S2, classify power grid project documents based on the project tag system;
[0007] S3, performing text preprocessing on the classified power grid project documents;
[0008] S4, using open source text annotation tools to perform entity annotation on the pre-processed power grid project documents, and generate a BIO format dataset corresponding to the power grid project documents;
[0009] S5, using the BERT model, Bi-GRU neural network and CRF model to establish a network learning model, and input the BIO format data set into the network learning model for training to obtain an entity recognition model;
[0010] S6, use the entity recognition model to perform feature recognition on new power grid project documents.
[0011] In step S1, the project label system includes primary category labels and secondary entity labels. The primary category labels include power grid infrastructure, industrial infrastructure, small-scale power grid infrastructure, production technical transformation, industrial technical transformation, production auxiliary technical transformation, sporadic purchases, production overhaul, industrial overhaul, production auxiliary overhaul, power market marketing, power grid digitization, research and development, management consulting, education and training, and equity investment. The secondary entity labels include institutions, power grid informatization, and amounts.
[0012] In step S3, the text preprocessing method is: after converting the PDF file of the power grid project document into EXCEL format using a format conversion tool, auxiliary information including blank pages, directories and references is removed, and then the removed file is converted into a TXT file.
[0013] In step S4, the BIO format data set includes statements and labels corresponding to each line of statements, and the labels include general labels, separators and specific label categories. The general label is the BIO label, B represents the starting position of the entity, I represents the non-first character of the entity, O represents the non-entity, and the specific label category corresponds to the secondary entity label. O represents the organization, E represents the power grid informatization, and M represents the amount.
[0014] The step S5 comprises the following steps:
[0015] S5.1, dividing the BIO format data set obtained in step S4 into a training set, a validation set, and a test set;
[0016] S5.2, using the Bert model, Bi-GRU neural network and CRF model to establish a network learning model;
[0017] S5.3, setting the verification accuracy threshold and the prediction accuracy threshold;
[0018] S5.4, using the training set data to train the network learning model to obtain an entity recognition model;
[0019] S5.5, using the validation set data to evaluate the entity recognition model, if the recognition accuracy of the entity recognition model is greater than the validation accuracy, execute step S5.6, otherwise return to step S5.4;
[0020] S5.6, use the test set data to test the entity recognition model. If the accuracy of the entity recognition model is less than the prediction accuracy threshold, return to step S5.4 and adjust the hyperparameters of the network learning model, otherwise execute step S6.
[0021] In step S5.2, the network learning model includes a BERT model, a Bi-GRU neural network and a CRF model. The input end of the BERT model is connected to the output end of the open source text annotation tool, the output end of the BERT model is connected to the input end of the Bi-GRU neural network, and the output end of the Bi-GRU neural network is connected to the input end of the CRF model.
[0022] Beneficial effects of the present invention:
[0023] The present invention uses the BERT model to replace the original word embedding algorithm for semantic encoding, and uses the Bi-GRU and CRF models for decoding to realize entity recognition and result optimization, which solves the problem that the number of unstructured documents in the power grid project system is huge and the content is complicated and difficult to manage. The construction of the project label system is conducive to the standardized management of the company's projects and the clarification of project boundaries, and is conducive to improving project management and investment efficiency; the application based on natural language processing is conducive to improving the recognition technology rules, improving the company's business feature recognition ability, improving the company's business management efficiency and level, reducing the project management costs of related personnel, saving manpower and time resources, and facilitating the application and transformation of historical stock data. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0025] Figure 1 This is the YEDDA operation interface diagram.
[0026] Figure 2 This is an example diagram of a BIO format dataset.
[0027] Figure 3 Input diagram for the BERT model.
[0028] Figure 4 Schematic diagram of the GRU model.
[0029] Figure 5 This is a structural diagram of the model of the present invention.
[0030] Figure 6 This is a schematic diagram of the change in the loss value of the model of the present invention with the number of training rounds.
[0031] Figure 7 This is a schematic diagram showing how the recognition accuracy of the model of the present invention changes with the number of training rounds.
[0032] Figure 8This is a schematic diagram of the change of the F1 value of the model recognition of the present invention with the number of training rounds. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] Named Entity Recognition (NER) is a subtask in the field of natural language processing (NLP). It is usually interpreted as identifying entities that humans have recognized, known or defined through historical practice rules from a piece of unstructured text. It also represents the ability to discover new named entities in a wide range of texts based on the composition rules of existing entities. Entities are meaningful semantic units in texts. The process of identifying entities is divided into two stages: first, determining the boundary range of the entity, and then assigning the entity to its type.
[0035] The main technical methods for named entity recognition are currently divided into: rule-based and dictionary-based methods, statistical-based methods, hybrid methods, and neural network methods. With the development of deep learning, vector representations of natural language have emerged, enabling natural language to be processed using deep neural networks. BERT is a leader in deep learning in the field of text processing, and it is currently the best pre-trained language model. Among feature extraction technologies, the most common is the variant of the RNN (Recurrent Neural Network) model, such as GRU (Gate Recurrent Unit), which can extract longer-range context feature information of the text. Combining the ability of the Conditional Random Field (CRF) to model sequence models, using the Bi-directional Gated Recurrent Unit (Bi-GRU) to provide context information can achieve better recognition results.
[0036] A method for identifying characteristic attributes of power grid projects based on deep learning, comprising the following steps:
[0037] S1, summarize the project business characteristics according to the categories of power grid project documents and project business management requirements, and establish a project label system based on the project business characteristics;
[0038] The project label system includes primary category labels and secondary entity labels. The primary category labels include power grid infrastructure, industrial infrastructure, small power grid infrastructure, production technical transformation, industrial technical transformation, production auxiliary technical transformation, sporadic purchases, production overhaul, industrial overhaul, production auxiliary overhaul, power market marketing, power grid digitization, research and development, management consulting, education and training, and equity investment. The secondary entity labels include project business characteristics such as institutions, power grid informatization, and amounts. The primary category labels are established based on the business classification of each historical power grid project document, and the secondary entity labels are determined based on the company's project business and project business operation management needs. Relevant adjustments can be made according to actual business during specific operations. The establishment of the project label system facilitates the management of each historical power grid project document.
[0039] In addition, when actually managing power grid project documents, the project labeling system can be adjusted. For example, it can be based on project categories, project contents, project time periods, etc. Project categories include large-scale infrastructure, technical transformation, informatization, marketing, etc. Project contents include project names, main research contents of projects, project contracts, project feasibility reports, etc. Project time periods include three years, two years, one year, quarter, etc.
[0040] ,Classify the power grid project documents based on the project tag system;
[0041] Classifying power grid project documents is conducive to batch management of power grid project documents.
[0042] ,Perform text preprocessing on the classified power grid project documents;
[0043] The text preprocessing method is to use a format conversion tool to convert the PDF file into EXCEL format, then remove auxiliary information such as blank pages, directories, references, etc., and then convert the removed files into TXT files to form an initial corpus as input for an open source text annotation tool, while retaining table information.
[0044] ,like Figure 2 As shown, the pre-processed power grid project documents are annotated with the help of open source text annotation tools to generate a BIO format dataset corresponding to the power grid project documents;
[0045] In this embodiment, the open source text annotation tool adopts YEDDA, which is a desktop annotation tool developed using tkinter and runs in Python 2.7 environment. Its code has been made public on Github. YEDDA supports three annotation tasks: chunk, entity, and event. Text input supports two modes: writing and file importing. The entity mode selected in the present invention is the entity recognition annotation task. Figure 1As shown in the figure, after entering the operation interface, you first need to define the entity category to be annotated on the right side of the interface; then use the shortcut keys corresponding to each entity category to annotate the text on the left. The annotated words will be distinguished by different background colors and their labels. After the annotation is completed, click export to export the Ann format text. The Ann format is the standard export format specified by the tool. Sentences are separated by blank lines, and each line includes unannotated text, annotated text and its label information. Figure 2 As shown, three secondary entity labels of organization, power grid informationization and amount are labeled. The BIO format data set includes statements and labels corresponding to each line of statements. The labels include general labels, separators and specific categories of labels. The general label is the BIO label, B represents the starting position of the entity, I represents the non-first character of the entity, and O represents non-entity; the separator is "-"; in the specific category of the label, O represents the organization, E represents the power grid informationization, and M represents the amount.
[0046] , using the Bert model, Bi-GRU neural network and CRF model to establish a network learning model, the BIO format data set obtained in step S4 is input into the network learning model for training to obtain an entity recognition model, including the following steps:
[0047] S5.1, divide the preprocessed BIO format dataset into training set, validation set and test set;
[0048] In this embodiment, the data volume ratio of the training set, validation set and test set is 7:1:2.
[0049] , using the Bert model, Bi-GRU neural network and CRF model to establish a network learning model;
[0050] BERT stands for Bidirectional Encoder Representation from Transformers, which is a bidirectional pre-trained model that uses Transformer as a feature extractor. This model learns a large amount of prior language, syntax, word meaning and other information for downstream tasks through unsupervised training of a large amount of corpus in the early stage. Compared with ordinary Transformer models, BERT draws on the characteristics of excellent models such as bidirectional LSTM and GPT, and uses bidirectional Transformer extraction to enable the model to deepen its understanding of sentence order and context. At the same time, it uses NSP and Mask-LM mechanisms to further deepen the model's feature extraction capabilities.
[0051] The full name is Next Sentence Prediction, which means the next sentence prediction. This mechanism is added to allow the model to understand the connection between two sentences. Many important downstream tasks, such as question answering (QA) and natural language inference (NLI), are based on understanding the relationship between two sentences, performing language modeling and directly acquiring. In order to train the model to understand the sentence relationship, the present invention pre-trains the next sentence prediction task in BERT. Specifically, when selecting sentences A and B for each pre-training example, there is a 50% chance that B is the actual next game of A, marked as IsNext, and the remaining 50% chance that B comes from other corpora, marked as NotNext.
[0052] It is the current mainstream feature extraction model. It improves the problems of slow training and long-term dependency of RNN, which are criticized by many people. It uses the Attention mechanism to reduce the distance between any two positions in the sequence to a constant, and abandons the sequential structure to have good parallelism. Transformer is essentially an Encoder-Decoder structure. In Encoder, the first layer of structure is the Attention module. The Multi-head Self-Attention mechanism proposed in Transformer can be explained in two parts, namely the Self-Attention mechanism and the Multi-head mechanism.
[0053] In the module, each word has three different vectors, namely the Query vector, the Key vector and the Value vector, which are obtained by multiplying the embedding vector X by three different coefficient matrices, respectively. This module imagines the constituent elements as consisting of a series of (Key, Value) data pairs. At this time, given an element Query in the target, by calculating the similarity or correlation between the Query and each Key, the weight coefficient of the Value corresponding to each Key is obtained, and then the value is weighted and summed to obtain the final Attention value. So in essence, the Attention mechanism is to perform a weighted summation of the Value values of the elements in the Source, and the Query and Key are used to calculate the weight coefficients of the corresponding Value. Here, the Key and Value in the Source are combined into one, that is, the semantic code corresponding to each word in the input sentence. It can be rewritten as the following formula:
[0054] Among them, is the weight coefficient, which can refer to the correlation between any two elements. The larger the weight, the more important the information is, so that the model can focus more on the key points in the learning process. N represents the number of sequence elements, and represents the sequence. The formula of the weight coefficient is:
[0055] The overall process of Self-Attention is:
[0056] 1. The input word is converted into an embedding vector X;
[0057] 2. Get three vectors Q, K, and V according to the embedded vector X;
[0058] 3. Calculate each vector;
[0059] 4. Unification operation, that is, division;
[0060] 5. Pass the score into the softmax activation function to get the Attention weight coefficient;
[0061] 6. The Attention weight coefficient is weighted and summed with the V vector to obtain the input vector Attention value v;
[0062] Multi-head integrates h different self-attentions. Multi-head stands for multi-head, which means dividing the model into multiple heads to form multiple subspaces, allowing the model to focus on different aspects of information, apply the Attention mechanism separately, and finally splice the results obtained separately. Multi-head attention helps the model understand richer features / information. The overall process of this step is:
[0063] 1. Input the embedding vector X into h self-attention modules respectively to obtain h weighted feature matrices;
[0064] 2. Concatenate the h feature matrices and finally pass the feature matrix through the fully connected layer to obtain the output Z;
[0065] The Multi-head Self-Attention mechanism is a major innovation proposed by the Transformer model. In essence, attention is to select and find a small amount of important information from a large amount of information without any selection and focus on this important information. The weight calculation of the attention calculation mechanism is a calculation process with weighted weight summation. The weight summation coefficient multiplied by the average formula can obtain the result such as attention, where the value of each weight summation coefficient represents the quality importance of the given information and the maximum contribution to the result of the given task, and the value corresponds to the information.
[0066] Strictly speaking, a bidirectional language model is more effective than a unidirectional model. It can better use context information to predict the target word, so that the model has a better feature extraction effect. In order to train a bidirectional Transformer, the present invention randomly shields the input training corpus. In the specific implementation, 15% of the words will be replaced with "[mask]" during the data processing process, and the model will be used to predict the shielded words to achieve the learning effect. In order to solve the problem of "[mask]" mismatch between pre-training and training, that is, if a word is shielded each time it is trained, the model will not be able to recognize the word during subsequent fine-tuning. Replacement is not always performed during the implementation process. When generating training data, on the basis of 15% of the words being selected, 80% of the words will be replaced, 10% of the words remain unchanged, and the remaining 10% of the words will be replaced with other words.
[0067] The model uses a bidirectional Transformer to make full use of the contextual information on the left and right sides, and its feature representation depends on the context on the left and right sides in all layers. This model combines the advantages of other models and abandons their disadvantages, and has achieved good results in many subsequent specific tasks of natural language processing. The BERT model requires a large number of parameters and powerful computing power to complete training. The present invention uses Google's open source BERT model to perform word embedding processing on documents.
[0068] Input the dataset into the BERT network model. For each word, its input representation can be composed of the sum of three parts of embedding. The three parts of embedding are all learned by the model. The sum of the three parts is the process of semantic feature fusion of the model. The visual representation of Embedding is as follows: Figure 3 As shown in the figure. Token Embeddings represents word vectors. When processing Chinese text information, word vectors can represent both a word and a character. In entity tag extraction, word vectors that are more in line with Chinese characteristics are used. The first word is the CLS mark, which can be used for subsequent classification tasks. When doing classification tasks with two sentences as input, Segment Embeddings is used to distinguish the two sentences. Position Embeddings is the position information learned through the model. After obtaining the word vector, the output of the BERT model is input into the Bi-GRU neural network to learn context features, and the output is the vector representation of the context of each word.
[0069] It is a neural network model that serializes sequential operations. Its variable length and serialization characteristics are naturally suitable for NLP tasks. Although RNN can theoretically learn long dependencies, in practice we prefer to use its variant GRU model. The GRU model aims to use the threshold mechanism to solve the problems of gradient vanishing and gradient exploding through memory units. This model is not only more lightweight, but has also been proven to capture long dependency features.
[0070] like Figure 4 As shown, the GRU model contains two threshold mechanisms, where the reset gate mechanism mainly realizes selective forgetting, and the update gate mechanism realizes selective retention of information. The formula is as follows:
[0071] Among them, is the sigmoid function, represents the Hadamard product operation, and are the output of this moment and the hidden layer state of the previous moment respectively, is the update gate state, indicating whether the previous information needs to be updated. is the reset gate state, indicating whether the previous information needs to be reset. is the hidden layer state after reset, is the hidden layer state after the new moment is updated. and represent the coefficient matrices for and in the calculation of the update gate state, and represent the coefficient matrices for and in the calculation of the reset gate state.
[0072] For a given sentence, containing n words, each word is represented as a d-dimensional vector, such as Figure 4 The GRU shown can calculate the previous representation of the sentence for each word t. Similarly, the following representation will also contain important information, which can be achieved by using a reverse GRU to read the same sequence. The forward and reverse GRUs are collectively called bidirectional GRUs (Bi-GRUs), which are different networks with different parameters. The word representation using this model is obtained by concatenating its context representation. Such a representation can effectively include the word representation in the context, which is very effective for multi-label applications.
[0073] In multi-classification problems, SoftMax classifier is often used for prediction, but SoftMax classifier does not take into account the dependency between labels in sequence labeling problems. The present invention adopts CRF model, which can consider the global information of label sequence and better predict labels.
[0074] The advantage of GRU is that it can learn the dependencies between input sequences through bidirectional feature extraction. During training, GRU can automatically extract the features of the observation sequence according to the goal, such as identifying entities, but its disadvantage is that it cannot learn the relationship between state sequences, such as output labels. In the named entity recognition task, there is a certain relationship between the labels. For example, the B-type label indicating the beginning of the entity will not be followed by another B-type label. Therefore, when GRU solves sequence labeling tasks such as NER, although it can save a lot of complicated feature engineering, it also has the disadvantage of not being able to learn the labeling context. On the contrary, the advantage of CRF is that it can model implicit states and learn the characteristics of state sequences, but its disadvantage is that it needs to manually extract sequence features. Therefore, adding another layer of CRF after GRU can obtain the advantages of both.
[0075] For the input sentence, the score matrix output by the Bi-GRU network is P. The size of P is n*k, where k is the number of different labels, corresponding to the score of the jth label of the i-th word in a sentence. For a series of predictions, where represents the score predicted as label i, the higher the score, the higher the probability that the model believes the result is the label. We define the score as:
[0076] Where A is a transition score matrix, representing the transition score from label i to label j, and are the start and end labels of a sentence, adding them to the set of possible labels, so A is a matrix of size k+2.
[0077] The function generates the probability of a sequence y for all possible label sequences:
[0078] During the training process, the log-probability of the correct label sequence is maximized:
[0079] where represents all possible token sequences for a sentence X. It is obvious from the above formula that the network is encouraged to generate a valid output token sequence. When decoding, the output sequence predicted to get the maximum score is:
[0080] like Figure 5As shown, the structure of the network learning model is a BERT-Bi-GRU-CRF cascade model, which includes a BERT model, a Bi-GRU neural network and a CRF model. The Bi-GRU neural network includes two GRU modules, namely a forward GRU network and a backward GRU network. The BERT model inputs the prepared BIO format data set and outputs the encoded text vector. The encoded text vector is input into the Bi-GRU neural network and outputs a further potential representation vector. The potential representation vector is then input into the CRF model and the final prediction result is output. This model uses the BERT model as a feature representation layer and adds it to the bidirectional GRU model. The bidirectional GRU model acts on a forward and backward GRU network for each training sequence, and these two are connected to the same output layer. Such a network structure can provide the output layer with complete context information for each sequence point, and the label information before and after the sequence is effectively considered through the CRF model.
[0081] , set the verification accuracy threshold and prediction accuracy threshold;
[0082] S5.4, using the training set data to train the network learning model to obtain an entity recognition model;
[0083] S5.5, using the validation set data to evaluate the entity recognition model, if the recognition accuracy of the entity recognition model is greater than the validation accuracy, execute step S5.6, otherwise return to step S5.4;
[0084] S5.6, test the entity recognition model using the test set data. If the accuracy of the entity recognition model is less than the prediction accuracy threshold, return to step S5.4 and adjust the hyperparameters of the network learning model. Otherwise, execute step S6;
[0085] S6, use the entity recognition model to perform feature recognition on new power grid project documents.
[0086] The specific process used in the present invention is: first download Google's official BERT-base (uncased) model, select and import the official Chinese pre-training parameter configuration file covering all words, and train according to the constructed training set, wherein the learning rate is set to, the optimizer selects Adam, the batch size is set to 128, and the number of GRU hidden units is set to 200. At the end of each round of training, the model will first be evaluated using the validation set, that is, the evaluation tool will output key data such as recognition accuracy and model loss value to observe the convergence of the model and check whether overfitting and other problems occur. Another function of the validation set is to assist in adjusting the hyperparameters. According to the performance of several sets of models with different hyperparameters on the validation set, it can be determined which set of hyperparameters corresponds to the model with better performance. After the validation set evaluation, the model will be tested on the test set to evaluate the model effect. The prediction accuracy threshold is set to 95%, that is, the training stops when the model's recognition accuracy for each label is above 95%. For the evaluation of the model on the validation set and test set, we chose conlleval, an open source sequence model performance evaluation tool that can be used for named entity recognition tasks. The tool automatically outputs information such as the overall recognition accuracy, recall rate, and F1 value of each label.
[0087] Figures 6 to 8 is the result of the simulation experiment, where Figure 6 The change of the overall loss value of the model is shown. It can be seen that the model is close to convergence after the 15th round. Figure 7 The changes in the model's prediction accuracy for power grid informatization E, organization O, and amount M are shown. It can be seen that after the 15th round of training, the accuracy fluctuates around 90%, 82%, and 86%, respectively, approaching convergence. Figure 8 The changes in the MF1 values of power grid informatization E, organization O, and amount are shown. It can be seen that after the 15th round of training, the F1 values fluctuate around 90, 82, and 88, respectively, which are close to convergence.
[0088] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for identifying characteristic attributes of power grid projects based on deep learning, characterized in that: The steps include: S1, summarize the project business characteristics according to the categories of power grid project documents and project business management requirements, and establish a project label system based on the project business characteristics; S2, classify power grid project documents based on the project tag system; The project labeling system includes primary category labels and secondary entity labels; S3, performing text preprocessing on the classified power grid project documents; S4, using open source text annotation tools to perform entity annotation on the pre-processed power grid project documents, and generate a BIO format dataset corresponding to the power grid project documents; S5, using the BERT model, Bi-GRU neural network and CRF model to establish a network learning model, and input the BIO format data set into the network learning model for training to obtain an entity recognition model; S6, using the entity recognition model to perform feature recognition on new power grid project documents; The BIO format data set includes statements and labels corresponding to each line of statements, wherein the labels include general labels, separators, and specific label categories. The general label is the BIO annotation, B indicates the starting position of the entity, I indicates the non-first character of the entity, O indicates non-entity, and the specific label category corresponds to the secondary entity label; The step S5 comprises the following steps: S5.1, dividing the BIO format data set obtained in step S4 into a training set, a validation set, and a test set; S5.2, using the Bert model, Bi-GRU neural network and CRF model to establish a network learning model; S5.3, setting the verification accuracy threshold and the prediction accuracy threshold; S5.4, using the training set data to train the network learning model to obtain an entity recognition model; S5.5, using the validation set data to evaluate the entity recognition model, if the recognition accuracy of the entity recognition model is greater than the validation accuracy, execute step S5.6, otherwise return to step S5.4; S5.6, test the entity recognition model using the test set data. If the accuracy of the entity recognition model is less than the prediction accuracy threshold, return to step S5.4 and adjust the hyperparameters of the network learning model. Otherwise, execute step S6; In step S5.2, the network learning model includes a BERT model, a Bi-GRU neural network and a CRF model, the input end of the BERT model is connected to the output end of the open source text annotation tool, the output end of the BERT model is connected to the input end of the Bi-GRU neural network, and the output end of the Bi-GRU neural network is connected to the input end of the CRF model; the Bi-GRU neural network includes two GRU modules, namely a forward GRU network and a backward GRU network, the input end of the forward GRU network and the input end of the backward GRU network are both connected to the output end of the BERT model, and the output end of the forward GRU network and the output end of the backward GRU network are both connected to the input end of the CRF model.
2. The method for identifying characteristic attributes of power grid projects based on deep learning according to claim 1 is characterized in that: In step S1, the first-level category tags include power grid infrastructure, industrial infrastructure, small-scale power grid infrastructure, production technical transformation, industrial technical transformation, production auxiliary technical transformation, sporadic purchases, production overhaul, industrial overhaul, production auxiliary overhaul, power market marketing, power grid digitalization, research and development, management consulting, education and training, and equity investment, and the second-level entity tags include institutions, power grid informatization, and amounts.
3. The method for identifying characteristic attributes of power grid projects based on deep learning according to claim 1 is characterized in that: In step S3, the text preprocessing method is: after converting the PDF file of the power grid project document into EXCEL format using a format conversion tool, auxiliary information including blank pages, directories and references is removed, and then the removed file is converted into a TXT file.
