Substation flood prevention case text classification method, device and equipment and medium

Abstract

The invention relates to a transformer substation flood prevention case text classification method and device, equipment and a medium, belongs to the field of natural language processing text classification, and is used for carrying out multi-label classification on transformer substation flood prevention case texts to be predicted based on a completely trained improved BERT model to obtain disaster cause labels of the flood prevention case texts. The improved BERT model comprises an embedded layer, an encoder layer, a TextCNN (Convolutional Neural Network) layer and a Seq2Seq layer; the transformer substation flood prevention case text classification method comprises the steps of extracting a text vector of a to-be-predicted transformer substation flood prevention case text based on an embedded layer, and compressing the text vector by adopting a lightweight strategy; extracting global semantic features of the compressed text vector based on an encoder layer; performing feature extraction on the global semantic features based on a TextCNN layer to obtain feature vectors; the feature vector is coded and decoded based on the Seq2Seq layer, the disaster-causing cause label of the flood prevention case text is obtained, the overall classification performance is enhanced, and the accuracy of the single cause label is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing text classification, and in particular to a method, device, equipment and medium for classifying text of flood prevention cases of substations. Background Art

[0002] With the rapid development of smart grid construction, the power system is developing towards the direction of large capacity and high proportion of new energy and renewable energy utilization. Substations are key nodes in the power grid. The substation flood control case text records a wealth of flood control information and experience, which is of great significance for identifying the causes of disasters and conducting effective flood control work. These causes of disasters are generally obtained through manual text analysis. In the conventional substation disaster cause identification work, researchers identify the potential causes of disasters in the substation based on their own experience and knowledge reserves. It is inevitable that there will be some situations where the analysis is not objective or the consideration is not comprehensive enough, resulting in inaccurate analysis of the causes of disasters, which brings great difficulties to the flood control decision-making of the substation. In addition, the manual analysis process is time-consuming and is not conducive to the digitization of flood control knowledge. In the field of natural language processing, text multi-label classification is a core technology for realizing automatic analysis of disaster causes.

[0003] Text multi-label classification models can be divided into methods based on traditional machine learning, deep learning, and pre-training. Methods based on traditional machine learning can be efficiently applied to small-scale data sets, and text multi-label classification models based on deep learning can effectively extract multiple types of complex features in text. Pre-training-based multi-label text classification models combine the advantages of deep learning and transfer learning, and have strong text understanding capabilities. Its representative model is BERT. BERT introduces a bidirectional attention mechanism, which enables the model to simultaneously consider the forward and backward information of the text, thereby capturing the global contextual information of the text and having strong text understanding capabilities.

[0004] However, as the number of labels and sample space increase, traditional machine learning methods find it difficult to capture complex features in text, resulting in low coupling between causal labels and difficulty in achieving expected classification performance. Deep learning-based multi-label text classification models cannot fully consider the contextual relationship of the text, resulting in insufficient understanding of text information. The BERT algorithm, as a pre-trained model, has advantages in text comprehension capabilities, but it still has defects. BERT contains many hyperparameters and requires a lot of computing resources, resulting in long running times. Summary of the invention

[0005] In view of this, it is necessary to provide a substation flood prevention case text classification method, device, equipment and medium to solve the technical problems of large number of parameters, insufficient understanding of text information and low coupling between cause labels when performing multi-label classification of substation flood prevention case text.

[0006] In order to solve the above problems, the present invention provides a substation flood prevention case text classification method, which is used to perform multi-label classification on the substation flood prevention case text to be predicted based on a well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; The substation flood prevention case text classification method includes: Extracting a text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compressing the text vector using a lightweight strategy; Extracting global semantic features of the compressed text vector based on the encoder layer; Perform feature extraction on the global semantic feature based on the TextCNN layer to obtain a feature vector; The feature vector is encoded and decoded based on the Seq2Seq layer to obtain the disaster cause label of the flood prevention case text.

[0007] In a possible implementation, extracting the text vector of the substation flood prevention case text to be predicted based on the embedding layer includes: Get text data of substation flood prevention cases; Preprocessing the substation flood prevention case text data to obtain a first input text; Adding a CLS tag and a SEP tag to the first and last part of the first input text respectively, to obtain a second input text; Inputting the second input text into the embedding layer, segmenting the second input text based on the embedding layer to obtain a plurality of characters of the text; Each character of the text is converted to obtain multiple codes corresponding to the characters, and a text vector is obtained based on the multiple codes.

[0008] In a possible implementation, the lightweight strategy includes dimensionality reduction mapping and parameter sharing; and the use of the lightweight strategy to compress the text vector includes: After decomposing the text vector into latent mappings by using dimensionality reduction mapping, parameters are shared among multiple Transformer layers through parameter sharing to achieve compression of the text vector.

[0009] In a possible implementation, the encoder layer includes a plurality of encoders, and the encoder includes a self-attention mechanism and a feedforward neural network; and extracting the global semantic features of the compressed text vector based on the encoder layer includes: The compressed text vector is input into the encoder layer, the compressed text vector is processed by the self-attention mechanism of the encoder, and the processed data is sent to the feedforward neural network to obtain the output of the feedforward neural network, and the output of the feedforward neural network is input into the next encoder, wherein the attention weights between the tokens at different positions in the sequence of the text vector are calculated by the self-attention mechanism, and the feature representation of each character in the sequence is processed and refined by the feedforward neural network.

[0010] In a possible implementation, the TextCNN layer includes a convolution layer, a pooling layer, and a fully connected layer, and the convolution layer includes convolution kernels of different sizes; and extracting features from the global semantic features based on the TextCNN layer to obtain a feature vector includes: The global semantic features are input into the TextCNN layer, and after the local semantic features are captured by multiple convolution kernels of the convolution layer, the feature vector is obtained through the maximum pooling operation of the pooling layer and the processing and combination of the fully connected layer.

[0011] In a possible implementation, the Seq2Seq layer includes a BiLSTM encoder and an LSTM decoder; encoding and decoding the feature vector based on the Seq2Seq layer to obtain a disaster cause label of the flood control case text includes: Encoding the feature vector using the BiLSTM encoder to obtain an encoder hidden state; Obtaining an initial hidden state of the decoder, based on the initial hidden state of the decoder, the LSTM decoder calculates the correlation between the encoder hidden state and the decoder hidden state using an attention mechanism, and aligns the encoder hidden state and the decoder hidden state through an alignment model to obtain a decoder output; The decoder output is classified by a softmax function to obtain the disaster cause label of the flood prevention case text.

[0012] In one possible implementation, the calculation formula of the encoder hidden state is: , in, is the encoder hidden state, is the encoder hidden state at the previous moment, is the semantic encoding vector; The calculation formula of the decoder hidden state is: , in, is the decoder hidden state, is the decoder hidden state at the previous moment, is the context vector; The calculation formula output by the decoder is: , in, is the decoder output, is the bias term, is the weight matrix.

[0013] On the other hand, the present invention also provides a substation flood prevention case text classification device, which is used to perform multi-label classification on the substation flood prevention case text to be predicted based on a well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; The substation flood prevention case text classification device comprises: A text vector extraction unit, used to extract the text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compress the text vector using a lightweight strategy; A semantic feature extraction unit, used for extracting global semantic features of the compressed text vector based on the encoder layer; A feature vector extraction unit, used for extracting features of the global semantic features based on the TextCNN layer to obtain a feature vector; The disaster cause label obtaining unit is used to encode and decode the feature vector based on the Seq2Seq layer to obtain the disaster cause label of the flood control case text.

[0014] In another aspect, the present invention further provides an electronic device, comprising: a processor and a memory; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the substation flood prevention case text classification method described above are implemented.

[0015] On the other hand, the present invention also provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the substation flood prevention case text classification method as described above.

[0016] The beneficial effects of the present invention are as follows: the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer, the text vector of the substation flood prevention case text to be predicted is extracted based on the embedding layer, and the text vector is compressed by a lightweight strategy, the parameter amount is reduced through the lightweight strategy, the calculation amount of the BERT model is simplified, and the training speed of the BERT model is accelerated; the global semantic features of the compressed text vector are extracted based on the encoder layer, the global semantic features are extracted based on the TextCNN layer to obtain the feature vector, after the global semantic features are extracted through the encoder layer, the feature extraction is performed through TextCNN to enhance the key local features, so that the BERT model can more fully understand the text information, the feature vector is encoded and decoded based on the Seq2Seq layer to obtain the disaster cause label of the flood prevention case text, the cause feedback attention mechanism decoding strategy based on Seq2Seq is used to process the correlation between different labels of the text, and the coupling between the cause labels in the BERT model is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of the structure of an embodiment of the improved BERT model provided by the present invention; Figure 2 A flow chart of an embodiment of a method for classifying texts of flood prevention cases of substations provided by the present invention; Figure 3 A schematic diagram of the structure of the BERT model of the substation flood prevention case text classification method provided by the present invention; Figure 4 A schematic diagram of three coding representations of the text classification method for substation flood prevention cases provided by the present invention; Figure 5 A schematic diagram of a lightweight strategy for the substation flood prevention case text classification method provided by the present invention; Figure 6 A schematic diagram of the self-attention mechanism of the substation flood prevention case text classification method provided by the present invention; Figure 7 A schematic diagram of the structure of an embodiment of a text classification device for flood prevention cases in substations provided by the present invention; Figure 8 A schematic structural diagram of an embodiment of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0019] The present invention discloses a method, device, equipment and medium for classifying text of flood prevention cases of substations, which can be used in a computer. The method, equipment or computer-readable storage medium involved in the present invention can be integrated with the above-mentioned equipment or can be relatively independent.

[0020] A specific embodiment of the present invention discloses a method for classifying text of substation flood prevention cases, which can be executed by a computer, and specifically by one or more processors of the computer. Figure 1 As shown, the improved BERT model 100 includes an embedding layer 110, an encoder layer 120, a TextCNN layer 130, and a Seq2Seq layer 140. Based on the well-trained improved BERT model, multi-label classification is performed on the text of the substation flood prevention case to be predicted, and the disaster cause label of the flood prevention case text is obtained; Figure 2 As shown in the figure, the text classification method of substation flood prevention case includes: S201, extracting a text vector of the substation flood prevention case text to be predicted based on the embedding layer 110, and compressing the text vector using a lightweight strategy; S202, extracting global semantic features of the compressed text vector based on the encoder layer 120; S203, extracting the global semantic features based on the TextCNN layer 130 to obtain a feature vector; S204, encoding and decoding the feature vector based on the Seq2Seq layer 140 to obtain the disaster cause label of the flood control case text.

[0021] Among them, the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer. The TextCNN layer includes a convolutional layer, a pooling layer and a fully connected layer. The output layer of the BERT model is replaced with a Seq2Seq layer. The Seq2Seq layer includes a BiLSTM encoder and an LSTM decoder.

[0022] Compared with the prior art, the substation flood prevention case text classification method provided in this embodiment includes an improved BERT model including an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer. The text vector of the substation flood prevention case text to be predicted is extracted based on the embedding layer, and the text vector is compressed using a lightweight strategy. The lightweight strategy reduces the number of parameters, simplifies the calculation amount of the BERT model, and speeds up the training of the BERT model. The global semantic features of the compressed text vector are extracted based on the encoder layer; the global semantic features are extracted based on the TextCNN layer to obtain feature vectors, and the key local features are enhanced through TextCNN so that the BERT model can more fully understand the text information. The feature vectors are encoded and decoded based on the Seq2Seq layer to obtain the disaster cause labels of the flood prevention case texts. The cause feedback attention mechanism decoding strategy based on Seq2Seq is used to process the correlation between different text labels, thereby improving the coupling between cause labels in the BERT model.

[0023] In some embodiments, in step S201, a text vector of a substation flood prevention case text to be predicted is extracted based on an embedding layer. First, the substation flood prevention case text data is obtained, and the substation flood prevention case text data is preprocessed to obtain a first input text. The preprocessing process is as follows: the substation flood prevention data text is segmented by a combination of jieba word segmentation and a custom dictionary, and stop words are removed by using a stop word dictionary on the basis of the substation flood prevention data text segmentation. Special symbols without practical meaning are removed, so that the text preprocessing result is significantly improved, and the reliable progress of subsequent tasks is ensured, which helps to improve the accuracy and efficiency of subsequent text structuring. Secondly, after obtaining the first input text, a CLS tag and a SEP tag are added to the head and tail of the first input text respectively through a BERT model to obtain a second input text. The CLS tag represents the characteristics of the entire input text sequence, and the SEP tag represents the end position of the input text. The BERT model includes an embedding layer, an encoder layer, and an output layer. For a structural diagram of the BERT model, please refer to Figure 3 ,like Figure 3 As shown in Figure 2, the substation flood prevention case text is converted into a vector representation for machine understanding and learning through the embedding layer. , i.e., text vectors, and then the Transformer encoder layer (denoted as TrmE) processes these vectors to extract semantic features, and then the output layer maps these features to the output vector , that is, the disaster cause label; finally, after obtaining the second input text, the second input text is input into the embedding layer, the second input text is segmented based on the embedding layer, multiple characters of the text are obtained, each character of the text is converted, multiple codes corresponding to the character are obtained, and a text vector is obtained based on the multiple codes, that is, in the embedding layer, the text is segmented into single characters, and each character is converted into three code representations, the three codes include token embeddings, segment embeddings, and position embeddings. For a schematic diagram of the three code representations, please refer to Figure 4 ,like Figure 4 As shown in the figure, token embedding is the encoding representation of the character itself, which is used to distinguish different characters; position embedding is the encoding representation of the fragment where the character is located, which is used to distinguish different sentences; word embedding is the encoding representation of the position of the character in the sentence, and the calculation formula for the encoding representation of the position of the character in the sentence is: , , in, The position index is The position encoding corresponding to the word, is the frequency corresponding to each position, is the word vector dimension.

[0024] The parameters of the output layer of the BERT model usually remain unchanged. The model size is mainly determined by the parameters of the embedding layer and the encoding layer. The larger the parameters, the longer the model runs. Therefore, a lightweight strategy is used to compress the text vector to reduce the parameter size of the BERT model. The lightweight strategy includes dimensionality reduction mapping and parameter sharing. The embedding layer of the traditional BERT model directly maps the vector to the hidden layer, making the dimension of the embedded vector and the number of parameters in the hidden layer The same calculation formula is: , By the dimension of the embedding vector With all vocabulary The product of determines the number of parameters of the embedding layer , and its calculation formula is: , The number of its parameters increases as encoder layers are stacked, and generally has large values, while all vocabulary is a large constant. Due to the large vocabulary in NLP, its value is of large magnitude, which all leads to a large In fact, the embedding layer uses a static mapping, which does not consider context information, and can reduce the dimension of the embedding vector to prevent parameter redundancy; for a schematic diagram of the lightweight strategy, see Figure 5 , a lightweight strategy is used to compress the text vector. The process is: after decomposing the text vector into latent mappings by using dimensionality reduction mapping, parameters are shared between multiple Transformer layers through parameter sharing to achieve text vector compression. The specific process is: decomposing the vector mapping into latent mappings, Mapping an input vector into a smaller embedding space of size H, the mapping from arrive Reduced ,because , ,therefore, , The parameters of each layer in the traditional BERT are independent, resulting in a significant increase in the number of parameters N in the Transformer encoder layer. Parameter sharing reduces the model size by sharing parameters between different Transformer layers, which is equivalent to learning only the parameters of the first layer and reusing the parameters of this layer in all Transformer encoders. The Transformer encoder layer uses a 12-layer Transforme encoder. Through parameter sharing, the number of parameters N is reduced to 1 / 12 of the original number of parameters. The lightweight strategy reduces the huge demand for computing resources and improves the efficiency of the BERT model.

[0025] In some embodiments, in step S202, the global semantic features of the compressed text vector are extracted based on the encoder layer, wherein the encoder layer includes multiple encoders, the encoder includes a self-attention mechanism and a feedforward neural network, the compressed text vector is input to the encoder layer, the compressed text vector is processed by the self-attention mechanism of the encoder, and the processed data is sent to the feedforward neural network to obtain the output of the feedforward neural network, and the output of the feedforward neural network is input to the next encoder, wherein the attention weights between the tokens at different positions in the sequence of the text vector are calculated by the self-attention mechanism, and the feature representation of each character in the sequence is processed and refined by the feedforward neural network. For the schematic diagram of the self-attention mechanism, please refer to Figure 6 ,like Figure 6 As shown, the text vector are assembled into a matrix , as the input of the self-attention mechanism, and then the vector Multiply by the weight matrix , and , respectively obtain the key vector , query vector Sum value vector , and its calculation formula is: , , , in, and Multiplying the points together gives the self-attention score, which determines how much attention is paid to other parts of the input sentence when encoding a certain word; then, Softmax is applied to calculate and The product of , where the resulting value indicates the relevance of each word to the word at the current position, and finally, the resulting value is multiplied by , and obtain the value of the self-attention mechanism. The output of the self-attention mechanism is: , in, is the penalty factor to ensure and The product of is not too large. In order to capture multiple features of the input text in parallel, the encoder layer of BERT uses a multi-head attention mechanism, which is an extension of the self-attention mechanism and is defined as: , in, To each After the matrix is ​​connected, it is connected to the matrix Multiply, For the The result output by the attention mechanism is calculated as follows: , in, For the The output of the attention mechanism is , and is the weight matrix.

[0026] In some embodiments, in step S203, since the flood prevention case text has many local semantic features, BERT is improved, that is, a separate feature extraction network TextCNN is superimposed on BERT to enhance these key local features, further extract the characteristics of the substation flood prevention case text, and obtain the global semantic features of the text through a multi-layer Transformer encoder. The global semantic features are input into the TextCNN layer, and feature extraction is performed on the global semantic features based on the TextCNN layer to obtain a feature vector. The TextCNN layer includes a convolutional layer, a pooling layer, and a fully connected layer. The global semantic features are input into the TextCNN layer. In the tCNN layer, after the local semantic features are captured by multiple convolution kernels of the convolution layer, the feature vector is obtained through the maximum pooling operation of the pooling layer and the processing and combination of the fully connected layer. That is, various convolution kernels of different sizes are used in the convolution layer of the TextCNN layer to capture local semantic features at different levels. The local semantic features are aggregated in the pooling layer and further processed and combined by the fully connected layer. Finally, the output layer receives the combined feature vector; the encoder converts the input text into a semantic vector, and the convolution layer extracts feature information of different granularities from the semantic vector. By setting convolution kernels of different sizes, a richer feature representation in the text can be obtained. The convolution result is: , in, is the output of the convolutional layer elements, is the convolution kernel weight, The input matrix is Go to A window composed of rows, is the number of words in the window, b is the bias term, is the activation function; The pooling layer is used to reduce the dimensionality of the convolutional layer output and retain the most important information. The maximum pooling operation is used, and its calculation formula is: , in, is the output of the pooling layer elements.

[0027] Finally, the outputs of the pooling layer are concatenated into a vector, which is further processed by the fully connected layer to achieve the corresponding downstream tasks.

[0028] In some embodiments, in step S204, since the causes of disasters are strongly coupled and affect the classification task, the output layer of BERT is replaced by a Seq2Seq layer with a cause feedback attention mechanism, wherein the Seq2Seq layer includes a BiLSTM encoder and an LSTM decoder, and the feature vector is encoded and decoded based on the Seq2Seq layer to obtain the disaster cause label of the flood control case text, and the feature vector is encoded using the BiLSTM encoder to obtain the encoder hidden state, and the initial hidden state of the decoder is obtained. Based on the initial hidden state of the decoder, the LSTM decoder calculates the correlation between the encoder hidden state and the decoder hidden state using the attention mechanism, and aligns the encoder hidden state and the decoder hidden state through the alignment model to obtain the decoder output, and classifies the decoder output through the softmax function to obtain the disaster cause label of the flood control case text. The specific process is: the encoder performs a In the example, BiLSTM is used to receive the semantic encoding vector of the input text. , and update its hidden state , the initial hidden state of the decoder LSTM is the final hidden state of the encoder BiLSTM, at each time step of the decoder The attention mechanism is used to calculate the encoder hidden state and decoder hidden state The relevance of and The calculation formulas are: , , in, is the output of the decoder at the previous time step, is the context vector, and the calculation formula of the context vector is: , in, is the attention weight, and the attention weight calculation formula is: , in, The learnable alignment model is used to calculate the alignment of the encoder and decoder hidden states. Finally, Seq2Seq ensures that the decoder generates the correct output sequence by dynamically focusing on different fragments of the input sequence during the decoding process. The decoder outputs The calculation formula is: , in, is the weight matrix, are bias terms, all of which are learnable parameters; Although both BERT and Seq2Seq require attention mechanisms, BERT focuses on relating input characters to the encoded sequence, while Seq2Seq instructs the decoder to align input characters and output generation, effectively capturing complex couplings.

[0029] In some embodiments, in order to verify the effectiveness of the improved BERT model, the experimental results before and after the introduction of different improvement strategies are compared, and the effectiveness is further verified by ablation experiments. For the experimental parameter settings, please refer to Table 1. Table 1

[0030] The performance of the model is evaluated using evaluation indicators, including accuracy, precision, recall and F1-measure (F1). The higher the value of the evaluation indicator, the better the performance of the model. The calculation formula is: , , , , in, is the accuracy, is the proportion of positive samples in the sample, that is, accuracy, is the proportion of negative samples in the sample, that is, the recall rate, To capture both precision and recall, For a real example, For a false positive example, For a true counterexample, A false counterexample.

[0031] Verification of the effectiveness of the local semantic feature enhancement strategy based on TextCNN. After implementing the local semantic feature enhancement strategy based on TextCNN, its model is called the BERT-TextCNN model. Experiments are conducted using BERT and BERT-TextCNN respectively. The changes in the accuracy of each label before and after the introduction of the local semantic feature enhancement strategy are shown in Table 2. Table 2

[0032] As shown in Table 2, the classification accuracy of various disaster causes is compared. After adopting the local semantic feature enhancement strategy, the accuracy of the model for "insufficient drainage capacity of drainage channel" and "no hidden dangers" slightly decreased by 1.07% and 0.53%, respectively. On the contrary, the accuracy of the other eight types of disaster causes has improved. It is worth noting that the accuracy of "insufficient water blocking capacity of cable channel" and "elevation does not meet the requirements" has increased by 3.73%, and the accuracy of "low level of flood control intelligence" has increased by 4.79%, which is a significant improvement. In general, the local semantic feature enhancement strategy based on TextCNN effectively enhances the model's ability to identify various disaster causes. The impact of its local semantic feature enhancement strategy on performance is shown in Table 3. Table 3

[0033] Table 3 shows the overall performance changes before and after the implementation of the strategy. Initially, the precision (P), recall (R), and F1 score of BERT were 81.94%, 76.74%, and 79.25%, respectively. After the introduction of the local semantic enhancement strategy, these indicators increased by 3.5%, 1.71%, and 2.55%, respectively, indicating that the strategy enhances the extraction of semantic features of substation flood control cases, thereby improving the overall classification performance of the model.

[0034] The effectiveness of the Seq2Seq-based causal feedback attention mechanism decoding strategy is verified. After implementing the Seq2Seq-based causal feedback attention mechanism decoding strategy, the BERT and BERT-Seq2Seq models are used for experimental comparison. The accuracy changes of each label before and after the introduction of the causal feedback attention mechanism decoding strategy are shown in Table 4. Table 4

[0035] Table 4 shows the classification results of various disaster causes. Compared with the BERT model, the BERT-Seq2Seq model shows higher accuracy in classifying eight disaster causes. Specifically, the accuracy of the "no hidden danger" category increased by 10.11%, while the accuracy of "insufficient water-blocking capacity of the wall" increased by 7.45%, which is the highest improvement. Although the accuracy of "insufficient water-blocking capacity of the building" decreased by 1.07%, the decrease was small. These results show that compared with the original BERT model, the Seq2Seq structure and its integrated attention mechanism for sequence prediction effectively consider the dependencies between different disaster causes, resulting in more accurate classification results. The impact of its cause feedback attention mechanism decoding strategy on performance is shown in Table 5. Table 5

[0036] As shown in Table 5, Table 5 shows the overall performance changes before and after the implementation of the Seq2Seq-based causal feedback attention mechanism decoding strategy. Initially, BERT's P, R, and F1 scores were 81.94%, 76.74%, and 79.25%, respectively. After the introduction of the strategy, these indicators increased by 6.47%, 8.7%, and 7.65%, respectively. This improvement shows that the causal feedback attention mechanism decoding strategy solves the coupling problem between causal labels, thereby improving the overall classification performance of the model and verifying the effectiveness of the Seq2Seq-based method.

[0037] The effectiveness of the lightweight strategy based on dimensionality reduction mapping and parameter sharing is verified. The impact of lightweight improvement on running time is shown in Table 6. Table 6

[0038] As can be seen from Table 6, the lightweight strategy reduces the training time of the BERT model by 38.07%, and brings nearly 1.6 times acceleration to the original BERT model. The results show that the parameter sharing technology in the lightweight strategy reduces the number of parameters, and the dimension reduction mapping reduces the dimension of the embedding vector, thereby reducing the computational complexity of the model. At the same time, the accuracy of the model decreases by 1.2%, indicating that lightweighting has little damage to accuracy and balances the speed and accuracy of the model.

[0039] In order to verify whether the combination of the improved semantic information extraction strategy and the attention mechanism decoding strategy can improve the multi-label classification performance of BERT, ablation experiments were conducted. For the experimental results, please refer to Table 7. Table 7

[0040] As shown in Table 7, both the local semantic feature enhancement strategy based on TextCNN and the causal feedback attention mechanism decoding strategy based on Seq2Seq enhance the P, R and F1 indicators of the model. It is worth noting that the Seq2Seq strategy shows more obvious improvements, indicating that the Seq2Seq strategy has a stronger ability to extract semantic information while taking into account the coupling between causal labels, which enhances the model performance. When the two strategies are introduced at the same time, compared with the original BERT model, P, R and F1 are improved by 6.75%, 10.41% and 8.67%, respectively. This dual-strategy approach (denoted as the BERT-TextCNN-Seq2Seq model) significantly enhances the model effectiveness, exceeding the use of a single strategy alone. Integrating the lightweight strategy into the BERT-TextCNN-Seq2Seq model reduces the training time by 30.68%, which is equivalent to almost 1.4 times acceleration, while the accuracy loss is minimal. The changes in the classification accuracy of each category before and after the introduction of the three improved strategies are shown in Table 8. Table 8

[0041] Table 8 shows that compared with the BERT model, the BERT-TextCNN-Seq2Seq-Lightweight model improves the classification accuracy of all disaster causes. The accuracy of "insufficient building water resistance" and "improper drainage equipment configuration" has increased slightly by 1.06%, while the accuracy of "no danger" has increased by 10.11%. These results show that applying all strategies simultaneously not only enhances the overall classification performance, but also improves the accuracy of individual cause labels.

[0042] In order to better implement the substation flood prevention case text classification method in the embodiment of the present invention, based on the substation flood prevention case text classification method, correspondingly, Figure 7 As shown, the embodiment of the present invention further provides a substation flood prevention case text classification device, which is used to perform multi-label classification on the substation flood prevention case text to be predicted based on the well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; the substation flood prevention case text classification device 700 includes: A text vector extraction unit 701 is used to extract the text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compress the text vector using a lightweight strategy; A semantic feature extraction unit 702, used to extract global semantic features of the compressed text vector based on the encoder layer; A feature vector extraction unit 703 is used to extract the global semantic features based on the TextCNN layer to obtain a feature vector; The disaster cause label obtaining unit 704 is used to encode and decode the feature vector based on the Seq2Seq layer to obtain the disaster cause label of the flood control case text.

[0043] like Figure 8 As shown, the present invention also provides an electronic device 800 , which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palmtop computer, a server, etc. The electronic device 800 includes a processor 801 , a memory 802 , and a display 803 . Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0044] In some embodiments, the memory 802 may be an internal storage unit of the electronic device 800, such as a hard disk or memory of the electronic device 800. In other embodiments, the memory 802 may also be an external storage device of the electronic device 800, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 800. Further, the memory 802 may also include both an internal storage unit of the electronic device 800 and an external storage device. The memory 802 is used to store application software and various types of data installed in the electronic device 800, such as program codes installed in the electronic device 800. The memory 802 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a text classification program for substation flood prevention cases is stored on the memory 802, and the text classification program for substation flood prevention cases can be executed by the processor 801, thereby realizing the text classification method for substation flood prevention cases in various embodiments of the present invention.

[0045] In some embodiments, the processor 801 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 802, such as a text classification method for substation flood prevention cases.

[0046] In some embodiments, the display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 803 is used to display identification information of the substation flood prevention case text classification program and to display a visual user interface. The components 801-803 of the electronic device 800 communicate with each other through a system bus.

[0047] In some embodiments, when the processor 801 executes the substation flood prevention case text classification program in the memory 802, the various steps in the substation flood prevention case text classification method described in the above embodiments are implemented. Since the substation flood prevention case text classification method has been described in detail above, it will not be repeated here.

[0048] Accordingly, the present invention also provides a computer-readable storage medium, which is used to store computer-readable programs or instructions. When the program or instructions are executed by a processor, the steps or functions in the substation flood prevention case text classification method provided in the above-mentioned method embodiments can be implemented.

[0049] In summary, the substation flood prevention case text classification method, device, equipment and medium provided by the present invention are used to perform multi-label classification on the substation flood prevention case text to be predicted based on the well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; the substation flood prevention case text classification method includes: extracting the text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compressing the text vector using a lightweight strategy; extracting the global semantic features of the compressed text vector based on the encoder layer; extracting features of the global semantic features based on the TextCNN layer to obtain a feature vector; encoding and decoding the feature vector based on the Seq2Seq layer to obtain the disaster cause label of the flood prevention case text, thereby enhancing the overall classification performance and improving the accuracy of a single cause label.

[0050] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0051] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A text classification method for substation flood prevention cases, characterized in that: Used to perform multi-label classification on the text of the substation flood prevention case to be predicted based on the well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; The substation flood prevention case text classification method includes: Extracting a text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compressing the text vector using a lightweight strategy; Extracting global semantic features of the compressed text vector based on the encoder layer; Perform feature extraction on the global semantic feature based on the TextCNN layer to obtain a feature vector; The feature vector is encoded and decoded based on the Seq2Seq layer to obtain the disaster cause label of the flood prevention case text.

2. The substation flood prevention case text classification method according to claim 1 is characterized in that: The text vector of the substation flood prevention case text to be predicted is extracted based on the embedding layer, including: Get text data of substation flood prevention cases; Preprocessing the substation flood prevention case text data to obtain a first input text; Adding a CLS tag and a SEP tag to the first and last part of the first input text respectively, to obtain a second input text; Inputting the second input text into the embedding layer, segmenting the second input text based on the embedding layer to obtain a plurality of characters of the text; Each character of the text is converted to obtain a plurality of codes corresponding to the characters, and a text vector is obtained based on the plurality of codes.

3. The substation flood prevention case text classification method according to claim 2 is characterized in that: The lightweight strategy includes dimensionality reduction mapping and parameter sharing; the lightweight strategy is used to compress the text vector, including: After decomposing the text vector into latent mappings by using dimensionality reduction mapping, parameters are shared among multiple Transformer layers through parameter sharing to achieve compression of the text vector.

4. The substation flood prevention case text classification method according to claim 3 is characterized in that: The encoder layer includes a plurality of encoders, and the encoder includes a self-attention mechanism and a feedforward neural network; the global semantic features of the compressed text vector extracted based on the encoder layer include: The compressed text vector is input into the encoder layer, the compressed text vector is processed by the self-attention mechanism of the encoder, and the processed data is sent to the feedforward neural network to obtain the output of the feedforward neural network, and the output of the feedforward neural network is input into the next encoder, wherein the attention weights between the tokens at different positions in the sequence of the text vector are calculated by the self-attention mechanism, and the feature representation of each character in the sequence is processed and refined by the feedforward neural network.

5. The substation flood prevention case text classification method according to claim 4 is characterized in that: The TextCNN layer includes a convolution layer, a pooling layer and a fully connected layer, and the convolution layer includes convolution kernels of different sizes; the feature extraction of the global semantic feature based on the TextCNN layer to obtain a feature vector includes: The global semantic features are input into the TextCNN layer, and after the local semantic features are captured by multiple convolution kernels of the convolution layer, the feature vector is obtained through the maximum pooling operation of the pooling layer and the processing and combination of the fully connected layer.

6. The substation flood prevention case text classification method according to claim 5 is characterized in that: The Seq2Seq layer includes a BiLSTM encoder and an LSTM decoder; encoding and decoding the feature vector based on the Seq2Seq layer to obtain the disaster cause label of the flood prevention case text includes: Encoding the feature vector using the BiLSTM encoder to obtain an encoder hidden state; Obtaining an initial hidden state of the decoder, based on the initial hidden state of the decoder, the LSTM decoder calculates the correlation between the encoder hidden state and the decoder hidden state using an attention mechanism, and aligns the encoder hidden state and the decoder hidden state through an alignment model to obtain a decoder output; The decoder output is classified by a softmax function to obtain the disaster cause label of the flood prevention case text.

7. The substation flood prevention case text classification method according to claim 6 is characterized in that: The calculation formula of the encoder hidden state is: ， in, is the encoder hidden state, is the encoder hidden state at the previous moment, is the semantic encoding vector; The calculation formula of the decoder hidden state is: ， in, is the decoder hidden state, is the decoder hidden state at the previous moment, is the context vector; The calculation formula output by the decoder is: ， in, is the decoder output, is the bias term, is the weight matrix.

8. A substation flood prevention case text classification device, characterized in that: Used to perform multi-label classification on the text of the substation flood prevention case to be predicted based on the well-trained improved BERT model to obtain the disaster cause label of the flood prevention case text; the improved BERT model includes an embedding layer, an encoder layer, a TextCNN layer and a Seq2Seq layer; The substation flood prevention case text classification device comprises: A text vector extraction unit, used to extract the text vector of the substation flood prevention case text to be predicted based on the embedding layer, and compress the text vector using a lightweight strategy; A semantic feature extraction unit, used for extracting global semantic features of the compressed text vector based on the encoder layer; A feature vector extraction unit, used for extracting features of the global semantic features based on the TextCNN layer to obtain a feature vector; The disaster cause label obtaining unit is used to encode and decode the feature vector based on the Seq2Seq layer to obtain the disaster cause label of the flood control case text.

9. An electronic device, characterized in that: including memory and processor; The memory stores a computer-readable program executable by the processor; When the processor executes the computer-readable program, the steps in the substation flood prevention case text classification method as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the substation flood prevention case text classification method as described in any one of claims 1-7.

Images