Overhead line fault detection method based on multi-source three-mode data fusion network

Abstract

The overhead line fault detection method based on the multi-source three-mode data fusion network comprises the following steps: acquiring historical fault data of an overhead line, constructing a multi-source three-mode data set containing waves, images and texts, and performing space-time dual-scale data enhancement to obtain a data set for fault detection; building different fault feature extraction network models for waves, images and texts for feature coding of each modal data, performing semantic alignment on feature vectors of the output waves, images and texts, and performing feature fusion to obtain fusion features of the multi-source three-modal data; generating a graph structure based on the fusion features of the multi-source tri-modal data, and inputting the graph structure into a dynamic graph attention neural network for training; and three-mode fault data of faults are collected to form a multi-source three-mode data set, a graph structure is generated after feature fusion, the graph structure is input into the trained dynamic graph attention neural network, a corresponding fault type is obtained, and parts and parts where the faults occur are positioned.

Description

Technical Field

[0001] The invention belongs to the technical field of overhead line fault detection, and relates to an overhead line fault detection method based on a multi-source three-modal data fusion network. Background Art

[0002] With the continuous development of long-distance overhead power transmission, energy resources have been optimally allocated over a larger range, solving the problem of uneven geographical distribution of energy production and consumption. However, it also puts forward higher requirements for security defense technology, especially as overhead lines are an important part of the power system. Since they are exposed to the external environment, they are more susceptible to line aging, dirty insulator flashover, lightning overvoltage, etc., which makes short-circuit faults often occur. At the same time, in actual systems, the duration of faults is short and there is little fault information from a single source, which is not conducive to reliable fault detection of overhead lines. Therefore, completing effective fault detection based on multi-source and multi-modal data at the time of fault has become the key to ensuring reliable and stable operation of the system.

[0003] Overhead line fault detection methods are usually divided into two methods: model-driven and data-driven. Among them, artificial intelligence technology that extracts fault data features based on deep learning and relies on data mining to achieve fault detection is the most widely studied. However, with the continuous maturity of UHV AC / DC networking technology, the structure of the power system has become more complex, and the failure rate of overhead line parts has continued to increase. This has led to many problems with existing artificial intelligence technology. For example, the lack of fault information leads to poor detection accuracy. At the same time, the lack of feature extraction capabilities of related methods can easily cause fault detection failure.

[0004] At present, under the background of continuous innovation and development of data-driven artificial intelligence technology, based on deep learning in multi-source parts of overhead lines such as overhead conductors and insulators, multi-source multi-modal data fusion neural networks are constructed based on multi-modal data such as images and texts to extract data features and realize fault detection of overhead lines, which can provide guarantee for the robust operation of the system. With the significant increase in the complexity of the power system structure, the failure rate of overhead line components has also increased. This trend exposes the limitations of current artificial intelligence technology in application, which are mainly reflected in: limited by insufficient fault information, the detection accuracy needs to be improved; and due to insufficient feature extraction capabilities, there is a risk of failure in fault detection. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides an overhead line fault detection method based on a multi-source three-modal data fusion network, so as to achieve the purpose of accurately determining the overhead line fault type and detecting faulty parts and locations.

[0006] The present invention provides an overhead line fault detection method based on a multi-source three-modal data fusion network, comprising:

[0007] Step 1: Obtain historical fault data of overhead lines, construct a multi-source tri-modal dataset containing waves, images, and texts, and obtain a dataset for fault detection through spatiotemporal dual-scale data enhancement;

[0008] Step 2: Build different fault feature extraction network models for waves, images, and texts for feature encoding of each modal data, perform semantic alignment on the feature vectors of the output waves, images, and texts, and perform feature fusion to obtain the fusion features of multi-source three-modal data;

[0009] Step 3: Generate a graph structure based on the fusion features of multi-source trimodal data, and input the graph structure into the dynamic graph attention neural network for training;

[0010] Step 4: Collect the trimodal fault data of the fault to form a multi-source trimodal data set and generate a graph structure after feature fusion. Input the graph structure into the trained dynamic graph attention neural network to obtain the corresponding fault type and locate the parts and parts where the fault occurs.

[0011] Furthermore, the step 1 is specifically as follows:

[0012] Step 1.1: The voltage waves at both ends of the overhead line in historical faults are taken as wave modal data, the multi-source fault images of faulty parts collected by drone inspections are taken as image modal data, and the description texts of the fault types and the impacts are summarized as text modal data to construct a multi-source tri-modal dataset containing waves, images, and texts.

[0013] Step 1.2: Based on the dual scale of time and space, the multi-source tri-modal data set is enhanced. That is, the wave modal data is enhanced in the time dimension according to the overlapping sliding window with a fixed step size; the information expressed by the image is enhanced in the spatial dimension by the shear rotation method; the text modal data is enhanced based on the synonymous description, so as to obtain the data set for training: This dataset contains labels for four types of typical faults: single-phase grounding fault SPG, two-phase grounding fault TPG, two-phase interphase short circuit fault TPSC, and three-phase interphase short circuit fault ThPSC. n Represents the nth wave modal data, M n Represents the nth image modality data, T n Represents the nth text modal data, n∈[1,N].

[0014] Furthermore, the step 2 is specifically as follows:

[0015] Step 2.1: Establish a wave data fault feature extraction network model, extract features from the wave modal data after data enhancement in step 1.2, and obtain the wave feature vector

[0016] Step 2.2: Establish an image data fault feature extraction network model, extract features from the image modal data after data enhancement in step 1.2, and obtain the image feature matrix

[0017] Step 2.3: Establish a text data fault feature extraction network model, extract features from the text modal data after data enhancement in step 1.2, and obtain the text feature vector

[0018] Step 2.4: Based on the mean multi-head attention mechanism, semantic alignment is performed on the trimodal data whose features are extracted in steps 2.1, 2.2, and 2.3, and feature fusion is performed to obtain the fused features of the multi-source trimodal data.

[0019] Furthermore, the step 2.1 is specifically as follows:

[0020] Step 2.1.1: Establish a wave data fault feature extraction network model including a first network normalization reconstruction layer, a non-uniform discrete Fourier transform layer, a first gated recurrent convolution layer and a Dropout layer connected in sequence;

[0021] Step 2.1.2 accelerates the training convergence through the first network standardization reconstruction layer, and transforms and reconstructs the wave modal data after network layer standardization according to the following formula:

[0022]

[0023] in, is the reconstructed wave modal data, is the standardized wave modal data, α and β are the learning parameters of the network standardized reconstruction layer, I is the unit matrix, and δ represents the standard deviation of U;

[0024]

[0025] in, is the nth wave modal data after standardization, U=[U1,U2...U N ] is the wave modal data, δ represents the standard deviation of U, E represents the expectation, and Var represents the average value of the matrix elements;

[0026] Step 2.1.3: Standardized reconstruction will lead to uneven time and frequency domains of wave data, so the standardized reconstructed wave modal data is subjected to non-uniform discrete Fourier transform based on overlapping feature information:

[0027]

[0028] Among them, x τ ∈R 1×s represents the overlapping features of the fixed-step overlapping sliding windows, ω represents the custom frequency point, R represents the dimension of the discrete Fourier transform, and ζ represents the time coordinate of the sampling;

[0029] Step 2.1.4: Feature extraction is performed on the wave modal data after the non-uniform discrete Fourier transform through the first gated recurrent convolution layer, which is based on the overlapping window feature x τ The extraction process is:

[0030]

[0031] in, Represents a sliding window of size τ The features of the corresponding window are extracted. τ>s; express With x τ The matrix multiplication of τ×s obtained by replication and expansion; W1, W2, W U To update the weight, b U is the bias term;

[0032] Step 2.1.5: Output features through the ReLU function Perform nonlinear transformation and use the transformation result as the input of the next layer;

[0033] Step 2.1.6: Use the Adam optimizer to optimize the parameters, update the network parameters to save the parameters that are best for wave fault feature extraction, and cut neurons through the Dropout layer to prevent overfitting of the feature extraction process, thereby obtaining a wave feature vector of size 1×c The subscript n represents the nth sample in the dataset.

[0034] Furthermore, the step 2.2 is specifically as follows:

[0035] Step 2.2.1: Establish an image data fault feature extraction network model including a second gated recurrent convolution layer, a first maximum pooling layer, a second network normalization reconstruction layer, a third gated recurrent convolution layer, a second maximum pooling layer and a Dropout layer connected in sequence;

[0036] Step 2.2.2: The image modality data is sequentially input into the second gated recurrent convolution layer, the first maximum pooling layer, the second network normalization reconstruction layer, the third gated recurrent convolution layer, the second maximum pooling layer and the Dropout layer for feature extraction;

[0037] The process of standardizing and reconstructing the data by the second network normalization and reconstruction layer is the same as the operation process of the first network normalization and reconstruction layer in step 2.1.2; the process of feature extraction by the second gated recurrent convolution layer and the third gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.4;

[0038] Step 2.2.3: Use Adam optimizer to optimize the network parameters and obtain an image feature matrix of size c×c

[0039] Furthermore, the step 2.3 is specifically as follows:

[0040] Step 2.3.1: Establish a text data fault feature extraction network model including a third network standardization reconstruction layer, an encoding-decoding large language module, a semantic feature self-encoding layer, a fourth gated recurrent convolution layer and a Dropout layer connected in sequence;

[0041] Step 2.3.2: Input the text modality data into the text data fault feature extraction network model, reconstruct it through the third network standardization reconstruction layer, perform semantic extraction and feature reconstruction through the encoding-decoding large language parsing module, and normalize the features through the fourth gated recurrent convolution layer after semantic feature self-encoding, and finally output a text modality feature vector of size 1×c

[0042] Among them, the process of standardizing and reconstructing the data by the third network standardization and reconstruction layer is the same as the operation process of the first network standardization and reconstruction layer in step 2.1.2; the process of feature extraction by the fourth gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.4.

[0043] Furthermore, the step 2.4 is specifically as follows:

[0044] Step 2.4.1: Semantically align the image feature matrix and wave feature vector based on the mean multi-head attention mechanism to obtain feature output Semantically align the text feature vector and the wave feature vector to obtain feature output Semantically align the image feature matrix and text feature vector to obtain feature output The formula for feature alignment is as follows:

[0045]

[0046]

[0047] Among them, a=m,t, b=u,t, a≠b, W Attis the linear transformation matrix of multi-head attention; M is the number of heads of multi-head attention; Concat(*) means that the features extracted by different heads of attention are combined in a splicing manner; Softmax is the activation function, W Q is the update weight of the Q vector, is the update weight of K vector, W V is the update weight of the V vector, d k is the scaling factor of the multi-head attention mechanism;

[0048] Step 2.4.2: Align the features after semantic alignment in step 2.4.1 and The features are combined according to the following formula, and the features are fused through a single-layer convolution to obtain the fusion features of multi-source tri-modal data.

[0049]

[0050] Among them, W3 and W4 are update weights.

[0051] Furthermore, the step 3 is specifically as follows:

[0052] Step 3.1: Fusion features of multi-source tri-modal data According to the fault label type, it is divided into single-phase grounding fault characteristics Two-phase ground fault characteristics Two-phase short circuit fault characteristics And three-phase short circuit fault characteristics

[0053] Step 3.2: Construct a graph structure for the four types of fault features, represented as G(V,L), where V = (V SPG ,V TPG ,V TPSC ,V ThPSC ) represents each node in the graph, Represents the connection lines between nodes, where:

[0054]

[0055] Weight them according to the cosine distance of the angle, and find the corresponding connection line according to the following formula:

[0056]

[0057] Among them, A and B represent four types of faults SPG, TPG, TPSC and ThPSC;

[0058] Step 3.3: Aggregate each fault type in the graph G(V,L) according to the following features:

[0059]

[0060] Where C∈Ν(C) represents the adjacent nodes related to the fault type A node in the graph G; W AA and W AC represents the learnable weight; w AA and w AC represents the self-node attention weight and the adjacent node weight, w AA Update according to the following formula:

[0061]

[0062] w AC Update according to the following formula:

[0063]

[0064] Step 3.4: The four types of fault features are aggregated through the dynamic graph attention layer, and the fault detection results of the dynamic graph attention network are output by the Softmax function after the fully connected layer.

[0065] The overhead line fault detection method based on a multi-source three-modal data fusion network of the present invention has the following beneficial effects:

[0066] (1) By using multi-source trimodal data including wave, image, and text description as the data source for overhead line fault detection, the problem of fault detection failure under a single data source is effectively eliminated. By enhancing the data of the trimodal information related to the fault, the impact of insufficient information caused by the short maintenance time during the actual fault is reduced.

[0067] (2) The multi-source tri-modal data fusion neural network method described in the present invention fuses different modal data from the perspective of the model, and encodes the data according to a model that is more suitable for modal feature extraction, such as an overlapping window feature module, thereby retaining the original features of the data to the greatest extent and ensuring the effectiveness and reliability of fault detection;

[0068] (3) Compared with the existing overhead line fault detection method, the method of the present invention further improves the differentiation by weighting different fault types through a dynamic graph attention neural network, providing multiple guarantees for the reliability output of the faulty parts and locations of the overhead line and their fault types. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a flow chart of an overhead line fault detection method based on a multi-source three-modal data fusion network of the present invention;

[0070] Figure 2It is a schematic diagram of extracting features through the fault feature extraction network model of each modal data and performing feature fusion;

[0071] Figure 3 It is a diagram of the power system structure. DETAILED DESCRIPTION

[0072] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0073] In this embodiment, the historical fault wave data is derived from Figure 3 The power system structure contains 36 nodes. The overhead line fault detection method based on multi-source three-modal data fusion network has the following process: Figure 1 As shown, the following steps are included:

[0074] Step 1: Obtain historical fault data of overhead lines, construct a multi-source tri-modal data set containing waves, images, and texts, and obtain a data set for fault detection through spatiotemporal dual-scale data enhancement. Step 1 is specifically as follows:

[0075] Step 1.1: The voltage waves at both ends of the overhead line nodes in historical faults are taken as wave modal data, and the multi-source fault images of faulty parts such as tower hardware, insulators and conductors collected by drone inspections are taken as image modal data. The fault types and description texts of the impacts are summarized as text modal data to construct a multi-source tri-modal dataset containing waves, images and texts.

[0076] In the specific implementation, the multi-source fault images of 248 groups of faulty parts are divided into single-phase grounding fault, two-phase grounding fault, two-phase interphase fault and three-phase interphase fault according to the text labels. Figure 3 The fault voltage wave data between the overhead lines between the middle node 11 and the node 12 correspond to the four types of faults, and a multi-source three-modal data set D containing waves, images, and texts is obtained.

[0077] Step 1.2: Based on the dual scale of time and space, the multi-source tri-modal data set is enhanced. That is, the wave modal data is enhanced in the time dimension according to the overlapping sliding window with a fixed step size; the information expressed by the image is enhanced in the spatial dimension by the shear rotation method; the text modal data is enhanced based on the synonymous description, so as to obtain the data set for training: This dataset contains labels for four types of typical faults: single-phase grounding fault SPG, two-phase grounding fault TPG, two-phase interphase short circuit fault TPSC, and three-phase interphase short circuit fault ThPSC. n Represents the nth wave modal data, M n Represents the nth image modality data, T nRepresents the nth text modal data, n∈[1,N].

[0078] In the specific implementation, each image of size (4096, 2160) is cut and enhanced into 16 images with random angle rotation by using a window of (128, 128), and a total of 3968 image samples M are obtained; for the text labels required for the 16 images cut from the same image, data enhancement is performed based on the synonymous description to obtain the corresponding text data T; based on the corresponding relationship between the fault types, the wave data is enhanced according to the sliding window of (1, 128) to obtain the wave data U, and finally the data pair is obtained.

[0079] Step 2: Build different fault feature extraction network models for waves, images, and texts for feature encoding of each modal data, perform semantic alignment on the output feature vectors of waves, images, and texts, and perform feature fusion to obtain the fusion features of multi-source three-modal data, such as Figure 2 As shown, specifically:

[0080] Step 2.1: Establish a wave data fault feature extraction network model, extract features from the wave modal data after data enhancement in step 1.2, and obtain the wave feature vector

[0081] In the specific implementation, the input size of the wave data fault feature extraction network model is 1×128, and the middle layer is two convolutional layers after batch normalization. Dropout is used to prevent overfitting, including the following steps:

[0082] Step 2.1.1: Establish a wave data fault feature extraction network model including a first network normalization reconstruction layer, a non-uniform discrete Fourier transform layer, a first gated recurrent convolution layer and a Dropout layer connected in sequence.

[0083] Step 2.1.2 accelerates the training convergence through the first network standardization reconstruction layer, and transforms and reconstructs the wave modal data after network layer standardization according to the following formula:

[0084]

[0085] in, is the reconstructed wave modal data, is the standardized wave modal data, α and β are the learning parameters of the network standardized reconstruction layer, I is the unit matrix, and δ represents the standard deviation of U.

[0086]

[0087] in, is the nth wave modal data after standardization, U=[U1,U2...U N] is the wave modal data, δ represents the standard deviation of U, E represents the expectation, and Var represents the average value of the matrix elements.

[0088] Step 2.1.3: Standardized reconstruction will lead to uneven time and frequency domains of wave data, so the standardized reconstructed wave modal data is subjected to non-uniform discrete Fourier transform based on overlapping feature information:

[0089]

[0090] Among them, x τ ∈R 1×s represents the overlapping features of the fixed-step overlapping sliding windows, ω represents the custom frequency point, R represents the dimension of the discrete Fourier transform, and ζ represents the time coordinate of the sampling.

[0091] Step 2.1.4: Feature extraction is performed on the wave modal data after the non-uniform discrete Fourier transform through the first gated recurrent convolution layer, which is based on the overlapping window feature x τ The extraction process is:

[0092]

[0093] in, Represents a sliding window of size τ The features of the corresponding window are extracted. τ>s; express With x τ The matrix multiplication of τ×s obtained by replication and expansion; W1, W2, W U To update the weight, b U is the bias term.

[0094] Step 2.1.5: Output features through the ReLU function Perform nonlinear transformation and use the transformation result as the input of the next layer.

[0095] Step 2.1.6: Use the Adam optimizer to optimize the parameters, update the network parameters to save the parameters that are best for wave fault feature extraction, and cut neurons through the Dropout layer to prevent overfitting of the feature extraction process, and obtain a wave feature vector of size 1×128 The subscript n represents the nth sample in the dataset.

[0096] Step 2.2: Establish an image data fault feature extraction network model with a model input size of 128×128. Perform feature extraction on the image modal data after data enhancement in step 1.2 to obtain the image feature matrix The step 2.2 is specifically as follows:

[0097] Step 2.2.1: Establish an image data fault feature extraction network model including a second gated recurrent convolution layer, a first maximum pooling layer, a second network normalization reconstruction layer, a third gated recurrent convolution layer, a second maximum pooling layer and a Dropout layer connected in sequence.

[0098] Step 2.2.2: The image modality data is sequentially input into the second gated recurrent convolution layer, the first maximum pooling layer, the second network normalization reconstruction layer, the third gated recurrent convolution layer, the second maximum pooling layer and the Dropout layer for feature extraction.

[0099] Among them, the process of standardizing and reconstructing the data by the second network normalization and reconstruction layer is the same as the operation process of the first network normalization and reconstruction layer in step 2.1.2; the process of feature extraction by the second gated recurrent convolution layer and the third gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.4.

[0100] Step 2.2.3: Use Adam optimizer to optimize the network parameters and obtain an image feature matrix of size 128×128

[0101] Step 2.3: Establish a text data fault feature extraction network model with a model input size of 1×128. Perform feature extraction on the text modal data after data enhancement in step 1.2 to obtain the text feature vector The step 2.3 is specifically as follows:

[0102] Step 2.3.1: Establish a text data fault feature extraction network model including a third network standardization reconstruction layer, an encoding-decoding large language module, a semantic feature self-encoding layer, a fourth gated recurrent convolutional layer and a Dropout layer connected in sequence.

[0103] Step 2.3.2: Input the text modality data into the text data fault feature extraction network model, reconstruct it through the third network standardization reconstruction layer, perform semantic extraction and feature reconstruction on it through the encoding-decoding large language parsing module, and normalize the features through the fourth gated recurrent convolution layer after semantic feature self-encoding, and finally output a text modality feature vector of size 1×128

[0104] Among them, the process of standardizing and reconstructing the data by the third network standardization and reconstruction layer is the same as the operation process of the first network standardization and reconstruction layer in step 2.1.2; the process of feature extraction by the fourth gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.4.

[0105] Step 2.4: semantically align the trimodal data whose features are extracted in steps 2.1, 2.2 and 2.3 according to the mean multi-head attention mechanism, and perform feature fusion to obtain the fusion features of the multi-source trimodal data. Step 2.4 is specifically as follows:

[0106] Step 2.4.1: Semantically align the image feature matrix and wave feature vector based on the mean multi-head attention mechanism to obtain feature output Semantically align the text feature vector and the wave feature vector to obtain feature output Semantically align the image feature matrix and text feature vector to obtain feature output The formula for feature alignment is as follows:

[0107]

[0108]

[0109] Among them, a=m,t, b=u,t, a≠b, W Att is the linear transformation matrix of multi-head attention; M is the number of heads of multi-head attention, and the value is M=6; Concat(*) means that the features extracted by different heads of attention are combined in a splicing manner; Softmax is the activation function, W Q is the update weight of the Q vector, is the update weight of K vector, W V is the update weight of the V vector, d k is the scaling factor of the multi-head attention mechanism.

[0110] Step 2.4.2: Align the features after semantic alignment in step 2.4.1 and The features are combined according to the following formula, and the features are fused through a single-layer convolution to obtain the fusion features of multi-source tri-modal data.

[0111]

[0112] Among them, W3 and W4 are update weights.

[0113] Step 3: Generate a graph structure based on the fusion features of multi-source trimodal data, and input the graph structure into the dynamic graph attention neural network for training. Step 3 is as follows:

[0114] Step 3.1: Fusion features of multi-source tri-modal data According to the fault label type, it is divided into single-phase grounding fault characteristics Two-phase ground fault characteristics Two-phase short circuit fault characteristics And three-phase short circuit fault characteristics

[0115] Step 3.2: Construct a graph structure for the four types of fault features, represented as G(V,L), where V = (V SPG ,V TPG ,V TPSC ,V ThPSC ) represents each node in the graph, Represents the connection lines between nodes, where:

[0116]

[0117] Weight them according to the cosine distance of the angle, and find the corresponding connection line according to the following formula:

[0118]

[0119] Among them, A and B represent four types of faults: SPG, TPG, TPSC and ThPSC.

[0120] Step 3.3: Aggregate each fault type in the graph G(V,L) according to the following features:

[0121]

[0122] Where C∈Ν(C) represents the adjacent nodes related to the fault type A node in the graph G; W AA and W AC represents the learnable weight; w AA and w AC represents the self-node attention weight and the adjacent node weight, w AA Update according to the following formula:

[0123]

[0124] w AC Update according to the following formula:

[0125]

[0126] Step 3.4: The four types of fault features are aggregated through the dynamic graph attention layer, and the fault detection results of the dynamic graph attention network are output by the Softmax function after the fully connected layer.

[0127] Step 4: Collect the trimodal fault data of the fault to form a multi-source trimodal data set and generate a graph structure after feature fusion. Input the graph structure into the trained dynamic graph attention neural network to obtain the corresponding fault type and locate the parts and parts where the fault occurs.

[0128] In specific implementation, Figure 3 The voltage operation data of the two ends of the line at the middle node 8 and node 9, 106 fault images different from the training data and the corresponding description text are obtained after data enhancement. This is input into the dynamic graph attention neural network trained in step 3, and based on its fault detection of overhead lines, the corresponding fault type, the parts and locations where the fault occurs are located. ,According to the calculation, 1696 sets of data output results are obtained, of which 88% of the fault detection results can correspond to the actual labels, and the correct fault type, parts and locations where the fault occurs are located are obtained.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the concept of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. An overhead line fault detection method based on a multi-source three-modal data fusion network is characterized in that: include: Step 1: Obtain historical fault data of overhead lines, construct a multi-source tri-modal dataset containing waves, images, and texts, and obtain a dataset for fault detection through spatiotemporal dual-scale data enhancement; Step 2: Build different fault feature extraction network models for waves, images, and texts for feature encoding of each modal data, perform semantic alignment on the feature vectors of the output waves, images, and texts, and perform feature fusion to obtain the fusion features of multi-source three-modal data; Step 3: Generate a graph structure based on the fusion features of multi-source trimodal data, and input the graph structure into the dynamic graph attention neural network for training; Step 4: Collect the trimodal fault data of the fault to form a multi-source trimodal data set and generate a graph structure after feature fusion. Input the graph structure into the trained dynamic graph attention neural network to obtain the corresponding fault type and locate the parts and parts where the fault occurs.

2. The overhead line fault detection method based on a multi-source three-modal data fusion network according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1: The voltage waves at both ends of the overhead line in historical faults are taken as wave modal data, the multi-source fault images of faulty parts collected by drone inspections are taken as image modal data, and the description texts of the fault types and the impacts are summarized as text modal data to construct a multi-source tri-modal dataset containing waves, images, and texts. Step 1.2: Based on the dual scale of time and space, data enhancement is performed on the multi-source tri-modal data set, that is, data enhancement is performed on the wave modal data in the time dimension based on an overlapping sliding window with a fixed step size; data enhancement is performed on the information expressed by the image in the spatial dimension by the shear rotation method; The text modality data is augmented based on the synonymous description to obtain the dataset for training: This dataset contains labels for four types of typical faults: single-phase grounding fault SPG, two-phase grounding fault TPG, two-phase interphase short circuit fault TPSC, and three-phase interphase short circuit fault ThPSC. n Represents the nth wave mode data, M n Represents the nth image modality data, T n Represents the nth text modal data, n∈[1,N].

3. The overhead line fault detection method based on a multi-source three-modal data fusion network according to claim 2, characterized in that: The step 2 is specifically as follows: Step 2.1: Establish a wave data fault feature extraction network model, extract features from the wave modal data after data enhancement in step 1.2, and obtain the wave feature vector Step 2.2: Establish an image data fault feature extraction network model, extract features from the image modal data after data enhancement in step 1.2, and obtain the image feature matrix Step 2.3: Establish a text data fault feature extraction network model, extract features from the text modal data after data enhancement in step 1.2, and obtain the text feature vector Step 2.4: Based on the mean multi-head attention mechanism, semantic alignment is performed on the trimodal data whose features are extracted in steps 2.1, 2.2, and 2.3, and feature fusion is performed to obtain the fused features of the multi-source trimodal data.

4. The overhead line fault detection method based on a multi-source three-modal data fusion network according to claim 3 is characterized in that: The step 2.1 is specifically as follows: Step 2.1.1: Establish a wave data fault feature extraction network model including a first network normalization reconstruction layer, a non-uniform discrete Fourier transform layer, a first gated recurrent convolution layer and a Dropout layer connected in sequence; Step 2.1.2 accelerates the training convergence through the first network standardization reconstruction layer, and transforms and reconstructs the wave modal data after network layer standardization according to the following formula: in, is the reconstructed wave modal data, is the standardized wave modal data, α and β are the learning parameters of the network standardized reconstruction layer, I is the unit matrix, and δ represents the standard deviation of U; in, is the nth wave modal data after standardization, U=[U1,U2...U N ] is the wave modal data, δ represents the standard deviation of U, E represents the expectation, and Var represents the average value of the matrix elements; Step 2.1.3: Standardized reconstruction will lead to uneven time and frequency domains of wave data, so the standardized reconstructed wave modal data is subjected to non-uniform discrete Fourier transform based on overlapping feature information: Among them, x τ ∈R 1×s represents the overlapping features of the fixed-step overlapping sliding windows, ω represents the custom frequency point, R represents the dimension of the discrete Fourier transform, and ζ represents the time coordinate of the sampling; Step 2.1.4: Feature extraction is performed on the wave modal data after the non-uniform discrete Fourier transform through the first gated recurrent convolution layer, which is based on the overlapping window feature x τ The extraction process is: in, Represents a sliding window of size τ The features of the corresponding window are extracted. express With x τ The matrix multiplication of τ×s obtained by replication and expansion; W1, W2, W U To update the weight, b U is the bias term; Step 2.1.5: Output features through the ReLU function Perform nonlinear transformation and use the transformation result as the input of the next layer; Step 2.1.6: Use the Adam optimizer to optimize the parameters, update the network parameters to save the parameters that are best for wave fault feature extraction, and cut neurons through the Dropout layer to prevent overfitting of the feature extraction process, thereby obtaining a wave feature vector of size 1×c The subscript n represents the nth sample in the dataset.

5. The overhead line fault detection method based on multi-source tri-modal data fusion network according to claim 4 is characterized in that: The step 2.2 is specifically as follows: Step 2.2.1: Establish an image data fault feature extraction network model including a second gated recurrent convolution layer, a first maximum pooling layer, a second network normalization reconstruction layer, a third gated recurrent convolution layer, a second maximum pooling layer and a Dropout layer connected in sequence; Step 2.2.2: The image modality data is sequentially input into the second gated recurrent convolution layer, the first maximum pooling layer, the second network normalization reconstruction layer, the third gated recurrent convolution layer, the second maximum pooling layer and the Dropout layer for feature extraction; The process of standardizing and reconstructing the data by the second network normalization and reconstruction layer is the same as the operation process of the first network normalization and reconstruction layer in step 2.1.2; the process of feature extraction by the second gated recurrent convolution layer and the third gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.4; Step 2.2.3: Use Adam optimizer to optimize the network parameters and obtain an image feature matrix of size c×c 6. The overhead line fault detection method based on multi-source tri-modal data fusion network according to claim 4, characterized in that: The step 2.3 is specifically as follows: Step 2.3.1: Establish a text data fault feature extraction network model including a third network standardization reconstruction layer, an encoding-decoding large language module, a semantic feature self-encoding layer, a fourth gated recurrent convolution layer and a Dropout layer connected in sequence; Step 2.3.2: Input the text modality data into the text data fault feature extraction network model, reconstruct it through the third network standardization reconstruction layer, perform semantic extraction and feature reconstruction through the encoding-decoding large language parsing module, and normalize the features through the fourth gated recurrent convolution layer after semantic feature self-encoding, and finally output a text modality feature vector of size 1×c Among them, the process of standardizing and reconstructing the data by the third network standardization and reconstruction layer is the same as the operation process of the first network standardization and reconstruction layer in step 2.1.2; the process of feature extraction by the fourth gated recurrent convolution layer is the same as the first gated recurrent convolution layer in step 2.1.

4.

7. The overhead line fault detection method based on a multi-source tri-modal data fusion network according to claim 4, characterized in that: The step 2.4 is specifically as follows: Step 2.4.1: Semantically align the image feature matrix and wave feature vector based on the mean multi-head attention mechanism to obtain feature output Semantically align the text feature vector and the wave feature vector to obtain feature output Semantically align the image feature matrix and text feature vector to obtain feature output The formula for feature alignment is as follows: Among them, a=m,t, b=u,t, a≠b, W Att is the linear transformation matrix of multi-head attention; M is the number of heads of multi-head attention; Concat(*) means that the features extracted by different heads of attention are combined in a splicing manner; Softmax is the activation function, W Q is the update weight of the Q vector, is the update weight of K vector, W V is the update weight of the V vector, d k is the scaling factor of the multi-head attention mechanism; Step 2.4.2: Align the features after semantic alignment in step 2.4.1 and The features are combined according to the following formula, and the features are fused through a single-layer convolution to obtain the fusion features of multi-source tri-modal data. Among them, W3 and W4 are update weights.

8. The overhead line fault detection method based on a multi-source three-modal data fusion network according to claim 7, characterized in that: The step 3 is specifically as follows: Step 3.1: Fusion features of multi-source tri-modal data According to the fault label type, it is divided into single-phase grounding fault characteristics Two-phase ground fault characteristics Two-phase short circuit fault characteristics And three-phase short circuit fault characteristics Step 3.2: Construct a graph structure for the four types of fault features, represented as G(V,L), where V = (V SPG ,V TPG ,V TPSC ,V ThPSC ) represents each node in the graph, Represents the connection lines between nodes, where: Weight them according to the cosine distance of the angle, and find the corresponding connection line according to the following formula: Among them, A and B represent four types of faults SPG, TPG, TPSC and ThPSC; Step 3.3: Aggregate each fault type in the graph G(V,L) according to the following features: Where C∈Ν(C) represents the adjacent nodes related to the fault type A node in the graph G; W AA and W AC represents the learnable weight; w AA and w AC represents the self-node attention weight and the adjacent node weight, w AA Update according to the following formula: w AC Update according to the following formula: Step 3.4: The four types of fault features are aggregated through the dynamic graph attention layer, and the fault detection results of the dynamic graph attention network are output by the Softmax function after the fully connected layer.

Images