Power grid communication equipment fault identification method and system based on multi-source heterogeneous data representation learning

Abstract

The invention discloses a power grid communication equipment fault identification method and system based on multi-source heterogeneous data characterization learning. The method comprises the following steps: firstly, collecting various complex multi-source heterogeneous data containing fault power grid communication equipment and preprocessing the complex multi-source heterogeneous data; extracting deep semantic features by using a pre-trained feature extraction network and obtaining a feature vector sequence; capturing time features of the feature vector sequence through a bidirectional long-short-term memory network, and obtaining complete sequence feature representation through a full connection layer; performing low-rank multi-modal fusion on the complete sequence feature representation in combination with cross-modal Transform to obtain three types of output splicing construction joint representation, and obtaining a fusion feature by using a self-attention mechanism; and finally, constructing a fault identification model by using the fusion features. Through innovative data preprocessing and data fusion technologies and in combination with an advanced deep learning model, fault-related characteristics can be automatically learned from multi-source heterogeneous data, and intelligent identification of the fault state of the power grid communication equipment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid communication equipment operation fault identification, and in particular to a power grid communication equipment fault identification method and system based on multi-source heterogeneous data characterization learning. Background Art

[0002] In smart grid communication systems, ensuring stable operation of equipment is crucial to maintaining grid reliability, and timely identification of faults is the key to achieving this goal. However, the data generated by grid communication equipment has obvious multi-source heterogeneity, which poses a huge challenge to traditional fault identification methods. Traditional methods rely on manual feature extraction and domain expert knowledge, which are difficult to adapt to the rapid growth and complexity of data, and can no longer meet the needs of modern power grids in terms of accuracy and efficiency.

[0003] With the development of big data and machine learning technology, representation learning can automatically extract data features so that machines can learn effective feature representations from raw data, thereby simplifying the subsequent analysis and decision-making process. However, applying representation learning to fault identification of power grid communication equipment requires solving the problem of fusion of multi-source heterogeneous data. Most existing multi-source heterogeneous data fusion methods use simple feature splicing or weighted average fusion strategies, ignoring the association between different data sources, and methods involving the use of inter-modal correlation interactions are still insufficient. For example, the use of low-rank decomposition tensor fusion for overall interaction between multiple data sources is prone to loss of original data information. The use of cross-modal Transformer to fuse multiple data sources involves more parameters and higher computational complexity, requiring more resources. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a power grid communication equipment fault identification method based on multi-source heterogeneous data characterization learning, which method uses a deep learning model to build a fault identification model.

[0005] In order to achieve the above object, the present invention provides the following technical solutions: The method for identifying faults in power grid communication equipment based on multi-source heterogeneous data characterization learning provided by the present invention comprises the following steps: S1, collects and preprocesses various complex multi-source heterogeneous data including faulty power grid communication equipment; S2, use the pre-trained feature extraction network to extract deep semantic features and obtain a feature vector sequence; S3, the feature vector sequence is passed through a bidirectional long short-term memory network to capture the temporal features and then passed through a fully connected layer to obtain the full sequence feature representation; S4, using low-rank multimodal fusion combined with cross-modal Transformer to obtain output concatenation to construct a joint representation for the full sequence feature representation, and using the fusion feature obtained by the self-attention mechanism; S5, constructing a fault identification model using the fusion features.

[0006] Furthermore, the various types of complex multi-source heterogeneous data including faulty power grid communication equipment in S1 include three types of source data, namely thermal image video, operation audio data and operation voltage and current text data, and the thermal image video, operation audio data and operation voltage and current text data of the same type of equipment collected in the same time period are one sample.

[0007] Further, the preprocessing in S1 includes any one or a combination of preprocessing the thermal image video, preprocessing the operation audio data, and preprocessing the operation voltage and current text data: The thermal image video is preprocessed by first cutting the thermal image video frame to obtain The thermal image sequence of the frame is processed, and then the thermal image sequence is subjected to mean filtering to reduce image noise, and finally the thermal image sequence is processed into an image sequence of preset resolution; The running audio data is preprocessed by first segmenting the running audio data to obtain The audio data is processed into a sequence of images with a preset resolution. The operating voltage and current text data is preprocessed by first extracting the sequence to obtain Each segment of voltage and current text data is subjected to denoising by discrete wavelet transform, each segment of text sequence is encoded into an image sequence using Gram angular field, and finally the voltage and current text data encoded image sequence is processed into an image sequence with a preset resolution.

[0008] Further, the feature vector sequence in S2 is obtained by the following steps: The pre-trained Mobile netV1 is used to extract the deep features of the three image sequences obtained after the pre-processing in the convolution layer; The feature map extracted by the convolution layer is flattened by reducing the dimension through the average pooling layer; The three feature vector sequences are encoded using positions to obtain feature vector sequences arranged in chronological order, wherein the feature vector sequences include a thermal image feature vector sequence, a spectrogram feature vector sequence, and a voltage and current text data encoding image feature vector sequence.

[0009] Furthermore, the full sequence feature representation in S3 is obtained by the following steps: Input each type of the feature vector sequence into a bidirectional long short-term memory network respectively; Use the forward and backward LSTM layers to process the forward and backward information of the sequence respectively; The last hidden layer output of the bidirectional long short-term memory network is mapped to the common sample label space through a fully connected layer to obtain the full sequence feature representation of each feature vector sequence.

[0010] Furthermore, the low-rank multimodal fusion in S4 is combined with the cross-modal Transformer to obtain a joint representation constructed by concatenating three outputs, which specifically includes the following steps: The three full-sequence feature representations are fused in pairs using low-rank multimodal fusion to obtain three low-dimensional outputs of the fused two data sources; The interaction modeling between two outputs is completed based on the cross-modal Transformer.

[0011] Furthermore, the fusion features in S4 include the common and complementary power grid communication equipment fault information of the thermal image video, the operation audio data and the operation voltage and current text data.

[0012] Furthermore, the fault identification model in S5 utilizes the fused features through a fully connected layer and applies a softmax function to output a predicted classification result.

[0013] The power grid communication equipment fault identification system based on multi-source heterogeneous data characterization learning provided by the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The above method is implemented when the processor executes the program.

[0014] The beneficial effects of the present invention are: The present invention provides a method for identifying faults in power grid communication equipment based on multi-source heterogeneous data representation learning. The method first collects various types of complex multi-source heterogeneous data containing faulty power grid communication equipment and performs preprocessing; uses a pre-trained feature extraction network to extract deep semantic features and obtain a feature vector sequence; then passes the feature vector sequence through a bidirectional long short-term memory network to capture temporal features and obtains a full-sequence feature representation through a fully connected layer; uses low-rank multimodal fusion combined with a cross-modal Transformer to obtain three output splicings to construct a joint representation for the full-sequence feature representation, and uses a self-attention mechanism to obtain fusion features; finally, uses the fusion features to construct a fault identification model. The present invention uses innovative data preprocessing and data fusion technologies, combined with advanced deep learning models, to automatically learn fault-related features from multi-source heterogeneous data and achieve intelligent identification of fault states of power grid communication equipment.

[0015] When preprocessing complex multi-source heterogeneous data such as temperature, vibration audio, voltage and current of power grid equipment, the present invention uses thermal images obtained by thermal imagers, vibration audio signals to convert into spectrograms, and voltage and current text data to encode into images using Lagum angular field, thus achieving efficient data integration and feature enhancement. This preprocessing not only easily captures key features and reduces noise interference, but also reduces the computational burden of subsequent model processing through dimensionality reduction technology.

[0016] In the joint characterization stage, the present invention proposes a low-rank multimodal fusion combined with a cross-modal Transformer method, which models the three outputs of low-rank multimodal fusion for pairwise interaction to complete information interaction, so that the output fusion strengthens the characteristics of a single data source information while including the characteristics of the other two information sources. Finally, the outputs are spliced ​​together to form a joint characterization, which reduces the parameters of the model, reduces the risk of data loss caused by low-rank decomposition, and also speeds up the training speed of the model.

[0017] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to make the purpose, technical solution and beneficial effects of the present invention clearer, the present invention provides the following drawings for illustration: Figure 1 The present invention is a simplified schematic diagram of the process.

[0019] Figure 2 The feature fusion diagram is a detailed operation flow of the present invention.

[0020] Figure 3 This is the structural diagram of the fusion features.

[0021] Figure 4 Confusion matrices for fault identification for multi-source heterogeneous data and three types of single-source data.

[0022] Figure 5 This is a comparison chart of the recognition rate curve during the training process. DETAILED DESCRIPTION

[0023] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0024] refer to Figure 1 and Figure 2, Figure 1 It is a simplified schematic diagram of the process of the present invention, Figure 2 This is a detailed operation flow of the present invention. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning provided in this embodiment includes the following steps: S1, collects and preprocesses various complex multi-source heterogeneous data including faulty power grid communication equipment; It should be noted that the various types of faulty power grid communication equipment collected usually include switches, routers, communication power supply equipment, etc. This example mainly takes the communication power supply equipment as an example, and uses the temperature, vibration audio, and output voltage and current timing signal of the communication power supply equipment as the criteria for judging equipment failure.

[0025] It should be noted that the temperature of the communication power supply equipment is reflected by the thermal image video of the equipment collected by the thermal imager, the vibration audio is reflected by the operating audio data, and the output voltage and current timing signal is reflected by the operating voltage and current text data.

[0026] It should be noted that the preprocessing includes preprocessing the thermal image video, the operating audio data, and the operating voltage and current text data: The thermal image video is preprocessed by first cutting the thermal image video frame to obtain The thermal image sequence of the frame is processed, and then the thermal image sequence is filtered to reduce the image noise, and finally the thermal image sequence is processed to a preset resolution. An image sequence of Indicates the number of horizontal pixels, Indicates the number of vertical pixels. In this embodiment , the thermal image sequence resolution is processed as ; The running audio data is preprocessed by first segmenting the running audio data to obtain Each audio segment is processed by fast Fourier transform, and the frequency domain audio data is converted into a spectrogram sequence using the Mel frequency cepstrum coefficients. Finally, the spectrogram sequence is processed to a preset resolution. In this embodiment, , the spectrogram sequence resolution is processed as ; The operating voltage and current text data is preprocessed by first extracting the sequence to obtain Each voltage and current text data is processed by discrete wavelet transform for noise reduction, and each text sequence is encoded into an image sequence using Gram angular field. Finally, the voltage and current text data encoded image sequence is processed into a preset resolution. In this embodiment, , the voltage and current text data encoding image sequence resolution is processed as .

[0027] It should be noted that the thermal image video, operating audio data and operating voltage and current text data of the communication power supply equipment collected in the same time period are a sample. Each sample is preprocessed to obtain a data set containing pictures. The training set, test set and verification set are divided into training set, test set and verification set in the ratio of 7:2; 1. The fault picture of the communication power supply equipment in the sample is marked as 0, and the normal picture of the communication power supply equipment is marked as 1. The label of the sample can also be expanded to the type of power grid communication equipment plus the operating status type. In this embodiment, a total of 523 samples are collected, including 421 samples of normal operation of the communication power supply equipment and 102 samples of equipment failure. Each sample contains the thermal image video, operating audio data and operating voltage and current text data of the communication power supply equipment collected simultaneously within 15s, wherein the frame rate of the thermal image video is 30Hz, the sampling frequency of the operating audio data is 44.1kHz, and the sampling frequency of the operating voltage and current text data is 1kHz.

[0028] Multi-dimensional fault feature analysis can be achieved through three types of multi-source heterogeneous data: thermal image video, operating audio data, and operating voltage and current text of communication power supply equipment, which can improve the accuracy and comprehensiveness of fault detection.

[0029] S2, use the pre-trained feature extraction network to extract deep semantic features and obtain a feature vector sequence; It should be noted that in S2, the feature vector sequence is obtained through the following steps: The pre-trained Mobile netV1 is used to extract the deep features of the three image sequences obtained after the preprocessing in the convolution layer, and the deep features are 7×7×1024 feature maps; The feature map extracted by the convolutional layer is reduced in dimension and flattened into a 1×1024 vector through the average pooling layer; Using position encoding of the three feature vector sequences, a feature vector sequence arranged in time order is obtained, including a thermal image feature vector sequence of , the spectrogram feature vector sequence is , the voltage and current text data encoding image feature vector sequence is ;in, Indicates the frame number of the thermal image video; Indicates the number of segments for running audio data segmentation; Indicates the sequence number of the running voltage and current text extraction.

[0030] S3, the feature vector sequence is passed through a bidirectional long short-term memory network to capture the temporal features and then passed through a fully connected layer to obtain the full sequence feature representation; It should be noted that the three full sequence feature representations are obtained through the following steps: Input each type of the feature vector sequence into a bidirectional long short-term memory network respectively; Use the forward and backward LSTM layers to process the forward and backward information of the sequence respectively; The last hidden layer output of the bidirectional long short-term memory network is mapped to the common sample label space through a fully connected layer to obtain the full sequence feature representation of each feature vector sequence.

[0031] S4, using low-rank multimodal fusion combined with cross-modal Transformer to obtain three output concatenations to construct a joint representation for the full sequence feature representation, and using the fusion feature obtained by the self-attention mechanism; Reference Figure 3 , Figure 3 This is the structural diagram of the fusion features.

[0032] It should be noted that low-rank multimodal fusion combined with cross-modal Transformer obtains three output concatenations to construct a joint representation, which specifically includes the following steps: The three full-sequence feature representations are fused in pairs using low-rank multimodal fusion to obtain three low-dimensional outputs of the fused two data sources; Based on the cross-modal Transformer, the interaction modeling between two outputs is completed, and a total of three conversions are performed. Among them, the interaction modeling is to transfer the information of one data source to another data source so that one data source has the information of another data source. The specific process is to use the features of the two data sources as the input of the encoder part of the cross-modal Transformer, and finally obtain the output after interaction modeling.

[0033] It should be noted that the steps of using low-rank multimodal fusion are to perform Hadamard product between the results of the inner product of the weight matrix after each low-rank decomposition of the full sequence feature representations and each other to obtain a low-dimensional output. The calculation formula is as follows: In the above formula, h represents the low-dimensional output obtained by fusing the full sequence feature representations pairwise. and Respectively represent two weight matrices after low-rank decomposition, and Represents two full sequence feature representations.

[0034] It should be noted that in this example, the low-rank decomposition used to decompose the weight matrix adopts singular value decomposition to obtain a low-rank approximation of the weight matrix.

[0035] It should be noted that the splicing formula is as follows: Where F is the joint representation vector after concatenation, , and They represent the three outputs obtained by using low-rank multimodal fusion combined with cross-modal Transformer to obtain the full sequence features of thermal images, the full sequence features of spectrograms, and the full sequence features of voltage and current text data encoded images.

[0036] It should be noted that the spliced ​​joint representation vector F is used as the input of the encoder part of the self-attention Transformer to obtain the fused features. The joint representation vector spliced ​​using the features of the three data sources can integrate the three parameter information of temperature, sound, and voltage and current, and the fused features obtained by fusing the three types of information can provide more comprehensive fault feature information and enhance the fault diagnosis capability.

[0037] It should be noted that the fusion features include the common and complementary power grid communication equipment fault information of thermal image video, operation audio data and operation voltage and current text data.

[0038] S5, constructing a fault recognition model using the fusion features; It should be noted that the fault identification model uses the fusion features through the fully connected layer and applies the softmax function to output the predicted classification results.

[0039] It should be noted that the fault identification model is mainly divided into feature extraction layer, average pooling layer, bidirectional long short-term memory network layer, full connection layer, joint representation layer, feature fusion layer, and output layer. The detailed functions of each layer are as follows: Figure 2 shown.

[0040] It should be noted that this example mainly classifies communication equipment power failure into two categories, and selects cross entropy CrossEntropy as the loss function of the model. The formula of the loss function is as follows: in, is the predicted value of the model, is the true label value.

[0041] refer to Figure 4 and Figure 5 , Figure 4 It is the confusion matrix of fault identification for multi-source heterogeneous data and three single-source data. Figure 5 It is a comparison chart of the recognition rate curves during the training process, which shows that the classification effect of fault identification using multi-source heterogeneous data is better than that of the other three single data sources. This method can automatically learn fault-related features from multi-source heterogeneous data, and integrate the fault features of multi-source heterogeneous data to realize intelligent identification of the fault status of power grid communication equipment.

[0042] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A power grid communication equipment fault identification method based on multi-source heterogeneous data characterization learning, characterized by: The following steps are involved: S1, collects and preprocesses various complex multi-source heterogeneous data including faulty power grid communication equipment; S2, use the pre-trained feature extraction network to extract deep semantic features and obtain a feature vector sequence; S3, the feature vector sequence is passed through a bidirectional long short-term memory network to capture the temporal features and then passed through a fully connected layer to obtain the full sequence feature representation; S4, using low-rank multimodal fusion combined with cross-modal Transformer to obtain output concatenation to construct a joint representation for the full sequence feature representation, and using the fusion feature obtained by the self-attention mechanism; S5, constructing a fault identification model using the fusion features.

2. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The various types of complex multi-source heterogeneous data containing faulty power grid communication equipment in S1 include thermal image video, operation audio data and operation voltage and current text data, a total of three source data, and the thermal image video, operation audio data and operation voltage and current text data of the same type of equipment collected in the same time period are one sample.

3. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The preprocessing in S1 includes any one or a combination of preprocessing the thermal image video, preprocessing the operation audio data, and preprocessing the operation voltage and current text data: The thermal image video is preprocessed by first cutting the thermal image video frame to obtain The thermal image sequence of the frame is processed, and then the thermal image sequence is subjected to mean filtering to reduce image noise, and finally the thermal image sequence is processed into an image sequence of preset resolution; The running audio data is preprocessed by first segmenting the running audio data to obtain The audio data is processed into a sequence of images with a preset resolution. The operating voltage and current text data is preprocessed by first extracting the sequence to obtain Each segment of voltage and current text data is subjected to denoising by discrete wavelet transform, each segment of text sequence is encoded into an image sequence using Gram angular field, and finally the voltage and current text data encoded image sequence is processed into an image sequence with a preset resolution.

4. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The feature vector sequence in S2 is obtained by the following steps: The pre-trained Mobile netV1 is used to extract the deep features of the three image sequences obtained after the pre-processing in the convolution layer; The feature map extracted by the convolution layer is flattened by reducing the dimension through the average pooling layer; The three feature vector sequences are encoded using positions to obtain feature vector sequences arranged in chronological order, wherein the feature vector sequences include a thermal image feature vector sequence, a spectrogram feature vector sequence, and a voltage and current text data encoding image feature vector sequence.

5. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The full sequence feature representation in S3 is obtained by the following steps: Input each type of the feature vector sequence into a bidirectional long short-term memory network respectively; Use the forward and backward LSTM layers to process the forward and backward information of the sequence respectively; The last hidden layer output of the bidirectional long short-term memory network is mapped to the common sample label space through a fully connected layer to obtain the full sequence feature representation of each feature vector sequence.

6. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The low-rank multimodal fusion in S4 is combined with the cross-modal Transformer to obtain a joint representation constructed by concatenating three outputs, which specifically includes the following steps: The three full-sequence feature representations are fused in pairs using low-rank multimodal fusion to obtain three low-dimensional outputs of the fused two data sources; The interaction modeling between two outputs is completed based on the cross-modal Transformer.

7. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The fusion features in S4 include common and complementary power grid communication equipment fault information among thermal image video, operation audio data and operation voltage and current text data.

8. The method for identifying power grid communication equipment faults based on multi-source heterogeneous data characterization learning according to claim 1 is characterized in that: The fault identification model in S5 uses the fused features through a fully connected layer and applies a softmax function to output a predicted classification result.

9. A power grid communication equipment fault identification system based on multi-source heterogeneous data characterization learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images