Deep learning transmission line fault reason identification method based on multi-modal information fusion

Abstract

The invention provides a deep learning power transmission line fault reason identification method based on multi-modal information fusion. The method comprises the following steps: acquiring recording data and external discrete information of a power transmission line; converting the recording data into a two-dimensional image by using a Grubm angle field, and extracting fault time-frequency features from the two-dimensional image by using a first model; performing keyword extraction and coding on the external discrete information to obtain coded information; performing feature extraction on the fault time-frequency features by using a first branch in a second model to obtain image representation, and performing feature extraction on the coding information by using a second branch in the second model to obtain discrete representation, the second model being a CNN-BiLSTM-based model; and fusing the image representation and the discrete representation through a second model, and obtaining a fault reason based on a fusion result. According to the method, the fault transient waveform features can be extracted, the nonlinear mapping relation between the internal and external information and the fault reasons can be constructed, and the fault reasons can be quickly and accurately identified.

Description

Technical Field

[0001] This invention relates to the field of power systems and their automation technology, and in particular to a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion. Background Technology

[0003] Transmission line faults are often caused by lightning strikes, wind deflection, wildfires, icing, pollution flashover, and foreign objects. Existing technologies have been developed to analyze the mechanisms from both internal and external factors and propose prevention and control measures. For example, lightning strike faults are identified using the median absolute deviation method and lightning location systems; wind deflection faults are detected by calculating conductor offset using insulator string deflection angles; and image recognition technology has also facilitated the identification of faults such as wildfires, bird wading, and icing. However, these technologies often focus on single faults and lack comprehensive identification systems. With the development of machine learning and artificial intelligence, algorithms such as support vector machines, backpropagation neural networks, and deep belief networks have been applied to the identification of transmission line fault causes. These methods rely on manual steps such as feature extraction and threshold setting, which can easily lead to error accumulation and affect classification results. To overcome these problems, relevant technologies have proposed corresponding solutions. For example, Gram angle fields are used to convert fault signals into two-dimensional images, and residual networks are introduced to improve recognition performance; an AlexNet classifier is built based on fault transient waveforms, achieving high accuracy on single-phase grounding fault datasets. However, the above methods do not effectively integrate multi-source data such as lightning location, meteorology, and waveform recording, and their versatility is still limited under complex working conditions. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion. This method is used to extract transient waveform features of faults and construct a nonlinear mapping relationship between internal and external information and fault causes, thereby achieving rapid and accurate fault cause identification.

[0005] To achieve the above objectives, the present invention adopts the following technical solution.

[0006] In a first aspect, the present invention provides a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion, comprising:

[0007] Acquire waveform data and external discrete information of transmission lines;

[0008] The recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model;

[0009] Keyword extraction and encoding are performed on the external discrete information to obtain encoded information;

[0010] The first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation, wherein the second model is a CNN-BiLSTM-based model.

[0011] The second model fuses the image representation with the discrete representation, and the cause of the fault is obtained based on the fusion result.

[0012] In some embodiments of this application, the first branch is a convolutional neural network branch, and the step of using the first branch in the second model to extract features from the time-frequency features of the fault to obtain an image representation includes:

[0013] Based on the fault time-frequency characteristics, the image features are obtained using the following formula:

[0014] ;

[0015] In the formula, It is the feature vector output by the l-th network layer in the first branch, and the dimension of the feature vector is... , These are the CNN features input to the l-th network layer. It is a bias parameter. It is a vectorization operator or a flattening operator, which will Mapped to , It is a non-linear activation function, ReLU.

[0016] The forward pass of the j-th feature map in the l-th layer of the first branch is:

[0017] ;

[0018] In the formula, This is the output of the j-th feature map in the l-th layer. The index of the input feature map. This is the set of feature maps used to compute the j-th feature vector in the l-th layer. It is the connection between the lth layer and the first layer. The input and the first Each output convolution kernel, It is the first Layer One input feature map, For the lth layer Bias terms of each feature map, For convolution, This is the activation function.

[0019] In some embodiments of this application, the second branch is a BiLSTM branch, and the step of using the second branch in the second model to extract features from the encoded information to obtain discrete representations includes:

[0020] Based on the encoded information, the discrete representation is obtained using the following formula:

[0021] , or ;

[0022] ;

[0023] ;

[0024] ;

[0025] ;

[0026] ;

[0027] ;

[0028] In the formula, Let be the hidden layer state vector at time t; and Forward and backward LSTMs at time [times] The hidden states capture past and future information respectively; This is the hidden state from the previous moment; This is the input gate at time t; For the forget gate at time t; This is the output gate at time t; Let be the cell state vector at time t; The cell state at the previous moment; Let be the candidate cell state at time t; Let be the input vector at time t; , , , These are the weight matrices input to each gate; , , , These are the bias terms for each gate; It is the sigmoid activation function; It is a hyperbolic activation function; For Hadamrad product.

[0029] In some embodiments of this application, the step of fusing the image representation with the discrete representation through the second model and obtaining the cause of the fault based on the fusion result includes:

[0030] In the same dimension, the image representation and the discrete representation are spatially aligned, and the aligned two representations are added and fused:

[0031] ;

[0032] ;

[0033] In the formula, The image representation of the first branch output. This is a discrete representation of the output of the second branch; , These are the aligned image representation and the discrete representation, respectively; , These are the alignment weight matrices for the image representation and the discrete representation, respectively; , These are the bias vectors for the image representation and the discrete representation, respectively; For fusion weighting coefficients; The fused feature vector; To calculate the weight matrix for the fusion weights; Calculate the bias term for the fusion weights.

[0034] In some embodiments of this application, during the training process, the second model calculates the logit value through the fully connected layer in the second model, and introduces learnable boundary parameters on the logit value to dynamically adjust the decision boundaries of each branch:

[0035] ;

[0036] In the formula, Adjusting the logit value after introducing boundary parameters; , These are the original logit outputs for the first and second branches, respectively. , The adaptive boundary parameters for the first and second branches are respectively learned automatically through training and used to adjust the decision boundary; then the optimization objective of the model is defined to minimize the cross-entropy between the predicted distribution and the true label, the model is trained, and the loss is calculated;

[0037] The loss function is:

[0038] ;

[0039] ;

[0040] In the formula, The total loss function value is N; N is the total number of training samples. For the first The true label of each sample; For the first The predicted probability of each sample; For the previous formula The logit value of each sample after boundary adjustment; Total number of fault categories It is the natural logarithm function; It is an exponential function.

[0041] In some embodiments of this application, the waveform data includes time-domain voltage, current, and zero-sequence current time series. Before converting the waveform data into a two-dimensional image using Gram angle field and extracting fault time-frequency features from the two-dimensional image using a first model, the method further includes:

[0042] The Karenbauer transform is used to perform phase-mode transformation on the time-domain voltage and current to obtain the fault point voltage and the current modulus flowing through the transition resistor, and the transition resistor value is obtained based on the fault point voltage and the current modulus.

[0043] The zero-sequence current time series is decomposed using discrete Fourier transform and converted into harmonic components.

[0044] Secondly, the present invention also provides a deep learning-based transmission line fault cause identification device based on multimodal information fusion, comprising:

[0045] The acquisition module is used to acquire waveform recording data of transmission lines and external discrete information;

[0046] The first feature extraction module is used to convert the recorded waveform data into a two-dimensional image using the Gram angle field, and to extract fault time-frequency features from the two-dimensional image using the first model.

[0047] The second feature extraction module is used to extract and encode keywords from the external discrete information to obtain encoded information.

[0048] The representation processing module is used to extract features from the fault time-frequency features using the first branch in the second model to obtain image representation, and to extract features from the encoded information using the second branch in the second model to obtain discrete representation, wherein the second model is a CNN-BiLSTM based model.

[0049] The fusion module is used to fuse the image representation with the discrete representation through the second model, and obtain the cause of the fault based on the fusion result.

[0050] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0051] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0052] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0053] The beneficial effects of this invention are as follows: The deep learning-based transmission line fault cause identification method based on multimodal information fusion provided by this invention first selects fault waveform data containing three-phase voltage and three-phase current, and external discrete information containing meteorological descriptions, time information, and reclosing operation status as data input. The fault waveform data is preprocessed using Gram angle field to obtain a two-dimensional image, and the external discrete information is numerically encoded and serialized in a fixed order. Fault feature representation is constructed from both two-dimensional image and tabular features. A CNN-BiLSTM dual-branch model is constructed: the CNN branch extracts spatial features of the two-dimensional image, and the BiLSTM branch performs temporal modeling on the encoded sequence of external discrete information. After feature layer fusion, fault cause identification and classification are completed. To enhance inter-class separability and generalization ability, adaptive boundary parameters are introduced and jointly optimized with cross-entropy. Validation results based on actual fault data show that this method can effectively identify the causes of transmission line faults.

[0054] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion, provided in an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of the structure of the first model provided in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the structure of the second model provided in an embodiment of the present invention;

[0059] Figure 4 A schematic diagram illustrating the training of the second model provided in an embodiment of the present invention;

[0060] Figure 5 This is a schematic diagram of the prediction results provided in an embodiment of the present invention. Detailed Implementation

[0061] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0062] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or couplings. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0063] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0064] To facilitate understanding of the embodiments of the present invention, the following will provide further explanation and description with reference to the accompanying drawings and several specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0065] Example 1

[0066] like Figure 1 As shown, a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion includes the following steps:

[0067] S101 acquires waveform recording data of the transmission line and external discrete information.

[0068] The waveform data includes three-phase voltage and three-phase current data. Waveform data from measurement points M and N are collected in real-time within a certain time window using protection devices at both ends of the line. M is the upstream measurement point of the transmission line, and N is the downstream measurement point. Additionally, transmission line parameters, including the total line length, distances from the fault point to points M and N, and the line's resistance and reactance values ​​under positive and zero-sequence conditions, are acquired. These parameters are used to calculate the transition resistance and then input into the CNN-BiLSTM model as one of its features.

[0069] External discrete information includes common weather descriptions, time information, reclosing operation status, etc.

[0070] S102, the recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model.

[0071] The first model is based on a convolutional neural network.

[0072] In some examples, Gram angle field is used to preprocess the three-phase voltage and current time series in the waveform data and convert them into two-dimensional images. The diagonal of the two-dimensional image obtained after the waveform data is transformed by Gram angle field reflects the increase of the time series, and the color change of the image can reflect the change of waveform amplitude and be sent into the first model, thereby deeply exploring the hidden features in the fault waveform time series.

[0073] For a certain current time series , For the first The process of generating a Gram angle field from point sampled values ​​is as follows:

[0074] (1) Data normalization: The time series is scaled to the interval [-1,1] by the following formula.

[0075] ;

[0076] in, The maximum value in the current-time series Y; The minimum value in the current-time series Y; These are the normalized values. The original current time series is normalized to... .

[0077] (2) Polar coordinate transformation: The normalized current-time series is transformed using the following formula. The values ​​and timestamps are mapped to angle and radius, respectively:

[0078] ;

[0079] in, The angle is in polar coordinates, and its range is [0, ... ]; N represents a constant that divides the interval [-1, 1] into equal intervals; For timestamps, It is the polar radius in polar coordinates.

[0080] (3) Gram angle field: The Gram angle field is generated by the following formula, which is the inner product of the sum of angles and the inner product of their differences. Difference from Gram's Point :

[0081] ;

[0082] .

[0083] (4) Image fusion: using a weighted average method to fuse images. and The data is fused and converted into a two-dimensional image. After Gram angle field transformation, the diagonal of the two-dimensional image of the fault record reflects the increase of the time series, and the color changes of the image can reflect the changes in the waveform amplitude.

[0084] To extract fault features from the Gram corner field image, a first model is used to extract fault time-frequency features from the two-dimensional image, such as... Figure 2 As shown, the first model includes convolutional layers, pooling layers, and fully connected layers.

[0085] S103, extract and encode keywords from the external discrete information to obtain encoded information. In some examples, common meteorological description keywords are extracted and encoded in conjunction with the line patrol report. For time information, the date and time period corresponding to the fault occurrence time are normalized respectively, and the reclosing operation is encoded.

[0086] S104, the first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation. The second model is a CNN-BiLSTM based model, and its structure is as follows: Figure 3 As shown.

[0087] In some embodiments of this application, the second branch is a BiLSTM branch, and the step of using the second branch in the second model to extract features from the encoded information to obtain discrete representations includes:

[0088] Based on the encoded information, the discrete representation is obtained using the following formula:

[0089] , or ;

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] ;

[0095] ;

[0096] In the formula, Let be the hidden layer state vector at time t; and Forward / backward LSTM at time [times] The hidden states capture past and future information respectively; This refers to the hidden state of the previous time step. This is the input gate at time t; For the forget gate at time t; This is the output gate at time t; Let be the cell state vector at time t; The cell state at the previous moment; Let be the candidate cell state at time t; Let be the input vector at time t; , , , Let be the weight matrix input to each gate; , , , These are the bias terms for each gate; It is the sigmoid activation function; It is a hyperbolic activation function; For Hadamrad product.

[0097] The CNN branch processes spatial information (two-dimensional image of Gram angle field) and outputs image representation. BiLSTM branches process time-series information (external discrete information sequences) and output discrete representations. These two branches run in parallel and independently, processing data from different modalities respectively, and then fusing the information in the subsequent steps.

[0098] S105, the image representation and the discrete representation are fused through the second model, and the cause of the fault is obtained based on the fusion result.

[0099] In some embodiments of this application, the step of fusing the image representation with the discrete representation through the second model and obtaining the cause of the fault based on the fusion result includes:

[0100] The output features of CNN and BiLSTM are mapped to a unified feature space through a linear transformation, giving them the same dimension.

[0101] ;

[0102] In the formula, The image representation output by the CNN branch is called the feature output by the CNN. Discrete representation of the output of the BiLSTM branch; , For the aligned image and discrete representation; , The alignment weight matrix for the image and discrete representation; , is the bias vector for the image and discrete representation.

[0103] The two aligned representations are then added and merged:

[0104] ;

[0105] In the formula, For fusion weighting coefficients (range 0-1); The fused feature vector; To calculate the weight matrix for the fusion weights; Calculate the bias term for the fusion weights.

[0106] In some embodiments of this application, the second model, during training:

[0107] The input is passed to the classification layer, and the logit value is obtained through the fully connected layer. Learnable boundary parameters are introduced into the logit output to dynamically adjust the decision boundary of each branch and enhance the separability between different fault categories.

[0108] ;

[0109] In the formula, Adjusting the logit value after introducing boundary parameters; , These are the original logit outputs for the CNN and BiLSTM branches, respectively. , The adaptive boundary parameters for the CNN and BiLSTM branches are learned automatically during training to adjust the decision boundary. Then, the optimization objective of the model is defined to minimize the cross-entropy between the predicted distribution and the true label. The model is then trained and the loss is calculated.

[0110] ;

[0111] in:

[0112] ;

[0113] In the formula, The total loss function value is N; N is the total number of training samples. For the first The true labels of each sample (one-hot encoded); For the first The predicted probability of each sample (after softmax); For the previous formula The logit value of each sample after boundary adjustment; This represents the total number of fault categories (e.g., lightning strike, wind deflection, wildfire, icing, pollution flashover, foreign object, etc.). It is the natural logarithm function; It is an exponential function.

[0114] During training, fault samples are preprocessed and used for offline training before being applied in the testing phase. The process flow is as follows: Figure 4 As shown, each fault case contains two types of features: one is external discrete information, which is numerically encoded to form a feature vector; the other is fault waveform data, which is converted into a 224×224×3 RGB two-dimensional feature map based on the Gram angle field method. Both inference and training adopt a CNN-BiLSTM dual-branch structure: the two-dimensional feature map is input into the CNN branch to extract spatial / local features, and the encoded vector of the external discrete information is serialized in a fixed order and input into the BiLSTM branch for modeling. The features from the two branches are linearly aligned, added, and fused, and the fault cause determination is output by the classification head. After model training is completed, the input samples are preprocessed and forward inferenced according to the same process during the testing phase to output the fault identification result.

[0115] During the training phase, the data is divided into training and validation sets. The network parameters and boundary hyperparameters are jointly optimized by cross-entropy loss (with adaptive boundary correction that can be superimposed) and the Adam optimizer, and the optimal model is selected based on the performance of the validation set. During the testing phase, the trained model is directly called to complete the fault cause identification.

[0116] In some embodiments of this application, the waveform data includes time-domain voltage, current, and zero-sequence current time series. Before converting the waveform data into a two-dimensional image using Gram angle field and extracting fault time-frequency features from the two-dimensional image using a first model, the method further includes:

[0117] The Karenbauer transform is used to perform a phase-mode transformation on the time-domain voltage and current to obtain the fault point voltage and the current modulus flowing through the transition resistor, and the transition resistor value is obtained based on the fault point voltage and the current modulus.

[0118] In some examples, the Karenbauer transform is used to perform phase-mode transformation on the acquired time-domain voltage and current, combining fault type and fault phase information. The voltage at the fault point and the current modulus flowing through the transition resistor are calculated to obtain the instantaneous value of the transition resistor. The average value within one cycle after the fault is then taken as the final transition resistor value. Its expression is as follows:

[0119] ;

[0120] In the formula, , This represents the number of sampling points; for The instantaneous value of the transition resistance at time t.

[0121] The zero-sequence current time series is decomposed using discrete Fourier transform and converted into harmonic components.

[0122] In some examples, based on the acquired fault waveform data, the discrete Fourier transform is used to decompose the zero-sequence current time series within two cycles after the fault occurs, transforming it into harmonic components, as shown below:

[0123] ;

[0124] in, This represents the zero-sequence current sample value. The range of values ​​for is N is the total number of sampling points in one period; This represents the harmonic order, and its value range is... Therefore, the DC component of the zero-sequence current , fundamental component With third harmonic components It can be calculated using the following formula:

[0125] .

[0126] DC component of zero-sequence current , fundamental component With third harmonic components It will be used as one of the features input into the CNN-BiLSTM model.

[0127] In some embodiments of this application, the test data comes from 7,157 fault cases from a provincial branch of the State Grid Corporation of China from 2021 to 2024. The causes of the faults are categorized into seven types based on the textual records of the patrol reports: lightning strikes, high-resistivity foreign objects, low-resistivity foreign objects, pollution flashover, wind deflection, wildfires, and icing. The fault samples are divided into training, validation, and test sets in a 4:2:1 ratio.

[0128] Learning rate and batch size were chosen as the optimal hyperparameters. When the model had been trained for 500 epochs, the fault identification model achieved a classification accuracy of 97.58% on the test set, and its confusion matrix was as follows. Figure 5 As shown.

[0129] Depend on Figure 5 It can be seen that the proposed method has a high accuracy rate in identifying faults such as lightning strikes and wind deflection, and has a strong adaptability to imbalance problems caused by a small number of samples, such as icing and wildfires. The CNN-BiLSTM method relies on CNN to capture local spatial features from the Gram corner field map, while BiLSTM models the time dependence of the fault process and fuses the serialized representation of external discrete information, which can complete the fault cause identification task with higher accuracy, further verifying the effectiveness of multimodal information fusion.

[0130] The deep learning-based transmission line fault cause identification method based on multimodal information fusion provided in this invention first selects fault waveform data containing three-phase voltage and three-phase current, and external discrete information containing meteorological descriptions, time information, and reclosing operation status as data input. The fault waveform data is preprocessed using Gram angle field to obtain a two-dimensional image, and the external discrete information is numerically encoded and serialized in a fixed order. Fault feature representation is constructed from both two-dimensional image and tabular features. A CNN-BiLSTM dual-branch model is constructed: the CNN branch extracts spatial features from the two-dimensional image, and the BiLSTM branch performs temporal modeling on the encoded sequence of the external discrete information. After feature layer fusion, fault cause identification and classification are completed. To enhance inter-class separability and generalization ability, adaptive boundary parameters are introduced and jointly optimized with cross-entropy. Validation results based on actual fault data show that this method can effectively identify the causes of transmission line faults.

[0131] Example 2

[0132] Based on Embodiment 1, Embodiment 2 provides a deep learning-based transmission line fault cause identification device based on multimodal information fusion. This deep learning-based transmission line fault cause identification device corresponds to the aforementioned deep learning-based transmission line fault cause identification method based on multimodal information fusion, and specifically includes:

[0133] The acquisition module is used to acquire waveform recording data of transmission lines and external discrete information;

[0134] The first feature extraction module is used to convert the recorded waveform data into a two-dimensional image using the Gram angle field, and to extract fault time-frequency features from the two-dimensional image using the first model.

[0135] The second feature extraction module is used to extract and encode keywords from the external discrete information to obtain encoded information.

[0136] The representation processing module is used to extract features from the fault time-frequency features using the first branch in the second model to obtain image representation, and to extract features from the encoded information using the second branch in the second model to obtain discrete representation, wherein the second model is a CNN-BiLSTM based model.

[0137] The fusion module is used to fuse the image representation with the discrete representation through the second model, and obtain the cause of the fault based on the fusion result.

[0138] For specific details, please refer to the description in the section on deep learning-based methods for identifying fault causes in transmission lines based on multimodal information fusion, which will not be repeated here.

[0139] Example 3

[0140] Embodiment 3 of the present invention provides an electronic device, including a memory and a processor, which communicate with each other. The memory stores program instructions that can be executed by the processor. The processor calls the program instructions to execute a deep learning-based method for identifying the cause of transmission line faults based on multimodal information fusion. The method includes the following steps:

[0141] Acquire waveform data and external discrete information of transmission lines;

[0142] The recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model;

[0143] Keyword extraction and encoding are performed on the external discrete information to obtain encoded information;

[0144] The first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation, wherein the second model is a CNN-BiLSTM-based model.

[0145] The second model fuses the image representation with the discrete representation, and the cause of the fault is obtained based on the fusion result.

[0146] Example 4

[0147] Embodiment 4 of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion. The method includes the following steps:

[0148] Acquire waveform data and external discrete information of transmission lines;

[0149] The recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model;

[0150] Keyword extraction and encoding are performed on the external discrete information to obtain encoded information;

[0151] The first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation, wherein the second model is a CNN-BiLSTM-based model.

[0152] The second model fuses the image representation with the discrete representation, and the cause of the fault is obtained based on the fusion result.

[0153] Example 5

[0154] Embodiment 5 of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion. The method includes the following steps:

[0155] Acquire waveform data and external discrete information of transmission lines;

[0156] The recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model;

[0157] Keyword extraction and encoding are performed on the external discrete information to obtain encoded information;

[0158] The first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation, wherein the second model is a CNN-BiLSTM-based model.

[0159] The second model fuses the image representation with the discrete representation, and the cause of the fault is obtained based on the fusion result.

[0160] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing the present invention.

[0161] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for method or system embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the description of the method embodiments. The method and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0162] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A deep learning-based method for identifying the causes of transmission line faults based on multimodal information fusion, characterized in that, include: Acquire waveform data and external discrete information of transmission lines; The recorded waveform data is converted into a two-dimensional image using the Gram angle field, and the fault time-frequency features are extracted from the two-dimensional image using the first model; Keyword extraction and encoding are performed on the external discrete information to obtain encoded information; The first branch of the second model is used to extract features from the time-frequency features of the fault to obtain an image representation, and the second branch of the second model is used to extract features from the encoded information to obtain a discrete representation, wherein the second model is a CNN-BiLSTM-based model. The second model fuses the image representation with the discrete representation, and the cause of the fault is obtained based on the fusion result.

2. The method according to claim 1, characterized in that, The first branch is a convolutional neural network branch. The step of using the first branch in the second model to extract features from the time-frequency features of the fault to obtain an image representation includes: Based on the fault time-frequency characteristics, the image features are obtained using the following formula: ； In the formula, It is the feature vector output by the l-th network layer in the first branch, and the dimension of the feature vector is... , These are the CNN features input to the l-th network layer. It is a bias parameter. It is a vectorization operator or a flattening operator, which will Mapped to , It is a non-linear activation function, ReLU. The forward pass of the j-th feature map in the l-th layer of the first branch is: ； In the formula, This is the output of the j-th feature map in the l-th layer. The index of the input feature map. This is the set of feature maps used to compute the j-th feature vector in the l-th layer. It is the connection between the lth layer and the first layer. The input and the first Each output convolution kernel, It is the first Layer One input feature map, For the lth layer Bias terms of each feature map, For convolution, This is the activation function.

3. The method according to claim 1, characterized in that, The second branch is a BiLSTM branch. The step of using the second branch in the second model to extract features from the encoded information to obtain discrete representations includes: Based on the encoded information, the discrete representation is obtained using the following formula: , or ; ； ； ； ； ； ； In the formula, Let be the hidden layer state vector at time t; and Forward and backward LSTMs at time [times] The hidden states capture past and future information respectively; This is the hidden state from the previous moment; This is the input gate at time t; For the forget gate at time t; This is the output gate at time t; Let be the cell state vector at time t; The cell state at the previous moment; Let be the candidate cell state at time t; Let be the input vector at time t; , , , These are the weight matrices input to each gate; , , , These are the bias terms for each gate; It is the sigmoid activation function; It is a hyperbolic activation function; For Hadamrad product.

4. The method according to claim 1, characterized in that, The step of fusing the image representation with the discrete representation through the second model and obtaining the cause of the fault based on the fusion result includes: In the same dimension, the image representation and the discrete representation are spatially aligned, and the aligned two representations are added and fused: ； ； In the formula, The image representation of the first branch output. This is a discrete representation of the output of the second branch; , These are the aligned image representation and the discrete representation, respectively; , These are the alignment weight matrices for the image representation and the discrete representation, respectively; , These are the bias vectors for the image representation and the discrete representation, respectively; For fusion weighting coefficients; The fused feature vector; To calculate the weight matrix for the fusion weights; Calculate the bias term for the fusion weights.

5. The method according to claim 1, characterized in that, During training, the second model calculates the logit value through the fully connected layers, and introduces learnable boundary parameters on the logit value to dynamically adjust the decision boundaries of each branch. ； In the formula, Adjusting the logit value after introducing boundary parameters; , These are the original logit outputs for the first and second branches, respectively. , The adaptive boundary parameters for the first and second branches are respectively learned automatically through training and used to adjust the decision boundary; then the optimization objective of the model is defined to minimize the cross-entropy between the predicted distribution and the true label, the model is trained, and the loss is calculated; The loss function is: ； ； In the formula, The total loss function value is N; N is the total number of training samples. For the first The true label of each sample; For the first The predicted probability of each sample; For the previous formula The logit value of each sample after boundary adjustment; Total number of fault categories It is the natural logarithm function; It is an exponential function.

6. The method according to claim 1, characterized in that, The recorded waveform data includes time-domain voltage, current, and zero-sequence current time series. Before converting the recorded waveform data into a two-dimensional image using Gram angle field and extracting fault time-frequency features from the two-dimensional image using the first model, the method further includes: The Karenbauer transform is used to perform phase-mode transformation on the time-domain voltage and current to obtain the fault point voltage and the current modulus flowing through the transition resistor, and the transition resistor value is obtained based on the fault point voltage and the current modulus. The zero-sequence current time series is decomposed using discrete Fourier transform and converted into harmonic components.

7. A deep learning-based transmission line fault cause identification device based on multimodal information fusion, characterized in that, include: The acquisition module is used to acquire waveform recording data of transmission lines and external discrete information; The first feature extraction module is used to convert the recorded waveform data into a two-dimensional image using the Gram angle field, and to extract fault time-frequency features from the two-dimensional image using the first model. The second feature extraction module is used to extract and encode keywords from the external discrete information to obtain encoded information. The representation processing module is used to extract features from the fault time-frequency features using the first branch in the second model to obtain image representation, and to extract features from the encoded information using the second branch in the second model to obtain discrete representation, wherein the second model is a CNN-BiLSTM based model. The fusion module is used to fuse the image representation with the discrete representation through the second model, and obtain the cause of the fault based on the fusion result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-6.

Images