Power transmission line fault cause identification method and device based on multi-modal information, and medium

Abstract

The application discloses a power transmission line fault cause identification method and device based on multi-modal information and a medium, relates to the technical field of power transmission line fault cause identification, and comprises the following steps: acquiring a fault recording image, real-time weather information and time information of a current power transmission line; inputting the multi-modal data into a fault cause identification model constructed by a joint loss function optimization to obtain a final fault cause; wherein the fault cause identification model is obtained through multi-source feature fusion and loss function collaborative training; and the identification method can improve the accuracy of power transmission line fault analysis results and improve the fault identification capability in complex power grid scenarios.

Description

Technical Field

[0001] This invention generally relates to the field of transmission line fault cause identification technology, and specifically to a method, device and medium for transmission line fault cause identification based on multimodal information. Background Technology

[0002] In recent years, significant progress has been made in deep learning-based methods for identifying the causes of transmission line faults. Some researchers have used convolutional neural networks to extract features from fault waveform images, capturing fault features through multi-layer convolution operations. However, existing technologies rely entirely on a single fault manifestation to determine transmission line faults. This fault identification often employs single or simple loss optimization strategies, resulting in insufficient feature representation. This makes it difficult to cope with the diversity of complex power grid scenarios, hindering the elimination of potential transmission line faults and slowing down transmission line operation and maintenance efficiency. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide a method, device and medium for identifying the cause of transmission line faults based on multimodal information.

[0004] In a first aspect, the present invention provides a method for identifying the cause of transmission line faults based on multimodal information, comprising:

[0005] Acquire the current fault waveform image, weather information, and time information; input the fault waveform image, weather information, and time information into the fault cause identification model to obtain the final fault cause; wherein, the fault cause identification model is optimized using a set loss function;

[0006] The method for obtaining the loss function includes the following steps:

[0007] Feature information is extracted from training fault waveform images, training weather information, and training time information to obtain training fault waveform features, training weather features, and training time features.

[0008] Based on the training fault recording features, training weather features, and training time features, the training multimodal fusion features are calculated.

[0009] Calculate the loss value corresponding one-to-one with the training multimodal fusion feature, training fault recording feature, training weather feature, and training time feature to obtain the final loss function.

[0010] According to the technical solution provided by the present invention, the process of obtaining the training fault recording features is as follows:

[0011] Acquire training fault waveform images and construct a fault waveform sub-network; input the training fault waveform images into the fault waveform sub-network, and use the fault waveform sub-network to extract features from the input training fault waveform images to obtain the training fault waveform features.

[0012] According to the technical solution provided by the present invention, the process of obtaining the training weather characteristics is as follows:

[0013] Acquire training weather information and construct a weather information subnetwork. Input the training weather information into the weather information subnetwork to obtain a transformed high-dimensional weather feature vector. Process the high-dimensional weather feature vector to obtain the training weather features.

[0014] According to the technical solution provided by the present invention, the process of obtaining the training time features is as follows:

[0015] Acquire training time information and construct a time information subnetwork; input the training time information into the time information subnetwork, which integrates... Transformer The module utilizes a multilayer perceptron to perform high-dimensional mapping on the training time information, and inputs the high-dimensional time feature vector obtained after high-dimensional mapping into the module. Transformer The module obtains the training time features.

[0016] According to the technical solution provided by the present invention, the training multimodal fusion features are calculated, including:

[0017] Extract the last dimension vector of the training fault recording feature as the recording fusion feature of the training fault recording feature; extract the category token vector of the training weather feature as the weather fusion feature of the training weather feature; extract the category token vector of the training time feature as the time fusion feature of the training time feature.

[0018] After aligning the waveform fusion features, weather fusion features, and time fusion features, a preliminary fusion feature is obtained through bidirectional additive fusion. The preliminary fusion feature is then subjected to convolutional sampling and weighted fusion processing to obtain the training multimodal fusion feature.

[0019] According to the technical solution provided by the present invention, after aligning the waveform fusion features, weather fusion features, and time fusion features, a preliminary fusion feature is obtained through bidirectional additive fusion, including:

[0020] The first intermediate feature is obtained by aligning and connecting the waveform fusion feature and the weather fusion feature in parallel.

[0021] The second intermediate feature is obtained by aligning and concatenating the first intermediate feature and the time-fused feature in parallel.

[0022] The third intermediate feature is obtained by aligning and connecting the first intermediate feature, the second intermediate feature, and the weather fusion feature in parallel.

[0023] The preliminary fusion feature is obtained by aligning and connecting the third intermediate feature and the waveform fusion feature in parallel.

[0024] According to the technical solution provided by the present invention, the preliminary fusion features are subjected to convolutional sampling and weighted fusion processing to obtain the training multimodal fusion features, including:

[0025] The preliminary fusion features are convolutionally sampled to obtain a multi-scale feature layer containing different sampling features; the sampling features under the multi-scale feature layer and the preliminary fusion features are aligned and then weighted and concatenated to obtain the training multimodal fusion features.

[0026] According to the technical solution provided by the present invention, calculating the loss value corresponding one-to-one with the training multimodal fusion feature, training fault recording feature, training weather feature, and training time feature to obtain the final loss function includes:

[0027] The waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features are obtained. The waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features are mapped to the predicted probabilities of multiple types of faults through a multilayer perceptron. The loss values ​​corresponding to the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features are calculated respectively through the cross-entropy loss function.

[0028] The loss function is calculated based on the loss values ​​corresponding to the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features.

[0029] In a second aspect, the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0030] Thirdly, the present invention provides a computer-readable storage medium having a computer program that, when executed by a processor, implements the steps of the above-described method.

[0031] Fourthly, the present invention provides a transmission line fault cause identification system based on multimodal information, comprising:

[0032] The identification module is used to acquire the current fault waveform image, weather information, and time information; input the fault waveform image, weather information, and time information into the fault cause identification model to obtain the final fault cause; wherein, the fault cause identification model is optimized using a set loss function;

[0033] An optimization module is used to obtain the loss function, and the optimization module includes the following:

[0034] The feature extraction unit is used to extract feature information from training fault waveform images, training weather information and training time information to obtain training fault waveform features, training weather features and training time features.

[0035] The calculation unit is configured to calculate the training multimodal fusion features based on the training fault recording features, training weather features, and training time features; and to calculate the loss values ​​corresponding one-to-one with the training multimodal fusion features, training fault recording features, training weather features, and training time features, so as to calculate the final loss function.

[0036] In some embodiments, the process of obtaining the training fault recording features is as follows:

[0037] The feature extraction unit is used to acquire training fault waveform images and construct a fault waveform sub-network; the training fault waveform images are input into the fault waveform sub-network, and the fault waveform sub-network is used to extract features from the input training fault waveform images to obtain the training fault waveform features.

[0038] In some embodiments, the process of obtaining the training weather features is as follows:

[0039] The feature extraction unit is used to acquire training weather information and construct a weather information sub-network. The training weather information is input into the weather information sub-network to obtain a transformed high-dimensional weather feature vector. The training weather features are obtained by processing the high-dimensional weather feature vector.

[0040] In some embodiments, the process of obtaining the training time features is as follows:

[0041] The feature extraction unit is used to acquire training time information and construct a time information subnetwork; the training time information is input into the time information subnetwork, which integrates... Transformer The module utilizes a multilayer perceptron to perform high-dimensional mapping on the training time information, and inputs the high-dimensional time feature vector obtained after high-dimensional mapping into the module. Transformer The module obtains the training time features.

[0042] In some embodiments, the training multimodal fusion features are calculated, including:

[0043] The feature extraction unit is used to extract the last dimension vector of the training fault recording feature as the recording fusion feature of the training fault recording feature; extract the category token vector of the training weather feature as the weather fusion feature of the training weather feature; extract the category token vector of the training time feature as the time fusion feature of the training time feature; and align the recording fusion feature, weather fusion feature and time fusion feature, and then fuse them through bidirectional addition to obtain a preliminary fusion feature; perform convolution sampling and weighted fusion processing on the preliminary fusion feature to obtain the training multimodal fusion feature.

[0044] In some embodiments, after aligning the waveform fusion features, weather fusion features, and time fusion features, a preliminary fusion feature is obtained through bidirectional additive fusion, including:

[0045] The feature extraction unit is used to align and connect the waveform fusion feature and the weather fusion feature in parallel to obtain a first intermediate feature; align and connect the first intermediate feature and the time fusion feature in parallel to obtain a second intermediate feature; align and connect the first intermediate feature, the second intermediate feature and the weather fusion feature in parallel to obtain a third intermediate feature; and align and connect the third intermediate feature and the waveform fusion feature in parallel to obtain a preliminary fusion feature.

[0046] In some embodiments, the preliminary fused features are subjected to convolutional sampling and weighted fusion processing to obtain the trained multimodal fused features, including:

[0047] The feature extraction unit is used to perform convolutional sampling on the preliminary fused features to obtain a multi-scale feature layer containing different sampling features; the sampling features under the multi-scale feature layer and the preliminary fused features are aligned and then weighted and concatenated to obtain the training multimodal fused features.

[0048] In some embodiments, loss values ​​corresponding one-to-one with the training multimodal fusion features, training fault recording features, training weather features, and training time features are calculated to obtain the final loss function, including:

[0049] The computing unit is used to acquire waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features, and maps these features to predicted probabilities of multiple fault types using a multilayer perceptron. It then calculates the loss values ​​corresponding to each feature using a cross-entropy loss function. Finally, it calculates the loss function based on the loss values ​​corresponding to these features.

[0050] In summary, this technical solution specifically discloses a method, device, and medium for identifying the cause of transmission line faults based on multimodal information. The identification method includes: acquiring current fault waveform images, weather information, and time information; inputting the fault waveform images, weather information, and time information into a fault cause identification model to obtain the final fault cause; wherein the fault cause identification model is optimized using a set loss function; the process of obtaining the loss function includes: extracting feature information from training fault waveform images, training weather information, and training time information to obtain training fault waveform features, training weather features, and training time features; calculating training multimodal fusion features based on the training fault waveform features, training weather features, and training time features; and calculating the loss value corresponding one-to-one with the training multimodal fusion features, training fault waveform features, training weather features, and training time features to calculate the final loss function.

[0051] Existing methods for identifying the causes of transmission line faults rely entirely on a single fault symptom, making it difficult to cope with the diversity of complex power grid scenarios. This invention acquires current fault waveform images, weather information, and time information, then inputs these into a fault cause identification model. The model, optimized by a loss function, extracts and deeply fuses features from the fault waveform images, weather information, and time information. By fully integrating these three information dimensions—fault waveform images, weather information, and time information—which are strongly correlated with fault causes, the model can analyze and process multi-source feature information strongly correlated with transmission line faults. This effectively improves the accuracy of transmission line fault analysis results and enhances the ability to identify faults in complex power grid scenarios. Attached Figure Description

[0052] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0053] Figure 1This is a flowchart illustrating a method for identifying the causes of transmission line faults based on multimodal information.

[0054] Figure 2 This is a schematic diagram of a transmission line fault cause identification system based on multimodal information.

[0055] Figure 3 This is a schematic diagram of a terminal device.

[0056] The diagram labels are as follows: 500, Terminal device; 501, CPU; 502, ROM; 503, RAM; 504, Bus; 505, I / O interface; 506, Input section; 507, Output section; 508, Storage section; 509, Communication section; 510, Driver; 511, Removable media; 600, Identification system; 601, Identification module; 602, Optimization module; 6021, Feature extraction unit; 6022, Calculation unit. Detailed Implementation

[0057] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0058] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0059] Example 1

[0060] To make the technical solutions of the embodiments of the present invention clearer and easier to understand, the application background of the embodiments of the present invention will be introduced below.

[0061] In recent years, deep learning-based methods for identifying the causes of transmission line faults have made progress. Some researchers have used convolutional neural networks (CNNs) to extract features from fault recording images, capturing fault features through multi-layer convolutional operations. Others have proposed a fault classification model based on CNNs, extracting key fault features through deep autoencoders and pooling operations. However, these methods ignore the complexity and diversity of transmission line fault causes, and considering only the influence of a single factor cannot accurately identify transmission line faults. To address this issue, some researchers have designed a multimodal fusion method, combining CNNs and long short-term memory networks to process multi-source data, fusing image features with other temporal information for classification. However, this method lacks deep interaction between features, limiting the expressive power of multimodal features and resulting in insufficient feature representation. Furthermore, some have proposed a joint loss optimization strategy, classifying transmission line faults through cross-entropy loss, but a single loss function is insufficient to handle the challenges of complex scenarios.

[0062] In view of this, the present invention proposes a method for identifying the cause of transmission line faults based on multimodal information. The method includes: acquiring current fault waveform images, weather information, and time information; inputting the fault waveform images, weather information, and time information into a fault cause identification model to obtain the final fault cause; wherein the fault cause identification model is optimized using a set loss function; the process of obtaining the loss function includes: extracting feature information from training fault waveform images, training weather information, and training time information to obtain training fault waveform features, training weather features, and training time features; calculating training multimodal fusion features based on the training fault waveform features, training weather features, and training time features; and calculating the loss value corresponding one-to-one with the training multimodal fusion features, training fault waveform features, training weather features, and training time features to obtain the final loss function.

[0063] As can be seen, the identification method proposed in this invention introduces weather and time information on the basis of fault recording images. After obtaining training fault recording features, training weather features, and training time features, multimodal feature fusion is performed to deeply explore the potential interaction relationship between weather factors, time factors, and fault features, thereby improving the performance of transmission line fault cause identification. In addition, for this multimodal feature fusion method, the loss function is optimized based on the loss value calculated by multimodal features, thereby effectively integrating multi-source features to cope with the diversity of complex power grid scenarios and enhance the robustness and classification accuracy of the model.

[0064] For details, please refer to Figure 1 The flowchart shown in this embodiment illustrates a method for identifying the cause of transmission line faults based on multimodal information, including:

[0065] S100: Obtain the current fault waveform image, weather information, and time information;

[0066] S200. Input the fault waveform image, weather information and time information into the fault cause identification model to obtain the final fault cause; wherein, the fault cause identification model is optimized using a set loss function.

[0067] In this embodiment of the invention, the executing entity can be an upper-level power grid control system capable of collecting the status of transmission lines in real time. This power grid control system can acquire real-time fault waveform images, weather information, and time information through devices such as fault recorders, image acquisition devices / mini weather stations, and built-in clocks.

[0068] Taking the acquisition of fault waveform images by a fault recorder as an example, the fault recorder can monitor key electrical quantities such as three-phase voltage, three-phase current, and zero-sequence voltage / current in real time by connecting the voltage transformer (PT) and current transformer (CT) of the transmission line. When a short circuit or grounding fault occurs in the line, the device will automatically trigger the waveform recording mode and record the electrical waveform data before, during, and after the fault at a high frequency sampling rate, thereby forming a fault waveform image.

[0069] Weather and time information can be collected simultaneously with fault recording images. Weather information can be obtained through a mini weather station, which is used to reflect weather data along the transmission line (e.g., temperature, humidity, precipitation, wind speed, and wind direction), while time information can be obtained directly through the timestamp of the acquisition equipment.

[0070] Next, after obtaining the fault waveform image, weather information, and time information, the fully connected layer in the transmission line fault cause identification model can be used to map the multimodal fusion features obtained based on the fault waveform image, weather information, and time information into the probability distribution of N types of faults. Finally, the fault cause identification model selects the type with the highest probability as the transmission line fault cause identification result for output. Among them, the optimized fault cause identification model can capture the correlation features of weather-time-fault and output accurate fault causes in complex scenarios.

[0071] Specifically, the fault cause identification model is optimized using a set loss function. The method for obtaining the loss function includes the following steps: extracting feature information from training fault waveform images, training weather information, and training time information to obtain training fault waveform features, training weather features, and training time features; calculating training multimodal fusion features based on training fault waveform features, training weather features, and training time features; and finally, combining the training multimodal fusion features, training fault waveform features, training weather features, and training time features to calculate the loss value corresponding to each feature, thereby obtaining the final loss function.

[0072] As can be seen, the loss function in this embodiment of the invention is obtained by using the loss values ​​corresponding to multimodal fusion features, training fault recording features, training weather features, and training time features. This enables the model to simultaneously grasp local features (single-modal) and global correlations (multimodal), solving the problems of insufficient feature representation and weak ability to cope with complex scenarios in traditional models. Ultimately, this makes the fault cause identification model more accurate and robust.

[0073] The following is an introduction to the fault cause identification model:

[0074] (1) In the fault cause identification model, the process of obtaining the training fault recording features is as follows:

[0075] Step A1: Obtain training fault waveform images;

[0076] This training fault waveform image was generated from the original fault waveform data and contains waveforms of eight key electrical quantities extracted from the original fault waveform data: three-phase phase voltage U a U b U c Three-phase phase current I a I b I c Zero-sequence voltage 3U0 and zero-sequence current 3I0.

[0077] Specifically, three-phase voltage and current (U a U b U c I a I b I c These quantities directly reflect the operating status of each phase of the line. Faults (such as short circuits and open circuits) usually cause a sudden drop in voltage or an abnormal surge in current. By analyzing the phase-to-phase or phase-to-ground relationship, electrical faults such as "line open circuit (complete open circuit)," "line open circuit (sub-conductor open circuit)," "foreign object (high resistance)," and "foreign object (low resistance)" can be distinguished. In addition, for "lightning strike," "flashover (bird droppings)," and "flashover (other)," the instantaneous changes in voltage / current also provide important clues. Zero-sequence voltage and current (3U0, 3I0): The zero-sequence component is significantly increased in grounding faults and is a key indicator for identifying "lightning strike," "flashover (bird droppings)," and "flashover (other)," especially highlighting the asymmetric characteristics in complex power grids. It is suitable for detecting grounding-related faults and together covers all typical fault scenarios of transmission lines, improving the identification accuracy of fault cause identification models.

[0078] Step A2: Construct the fault recording subnetwork;

[0079] A pre-trained deep learning network is determined, and the last fully connected layer in the pre-trained deep learning network is removed to obtain the fault waveform sub-network. In this embodiment of the invention, the pre-trained deep learning network can be ViT-B / 32, and only the network with the last fully connected layer removed in the pre-trained deep learning network is selected as the final fault waveform sub-network.

[0080] Step A3: Input the training fault waveform image into the fault waveform sub-network, and use the fault waveform sub-network to extract features from the input training fault waveform image to obtain the training fault waveform features.

[0081] After inputting the training fault waveform images into the fault waveform sub-network, the original size of the waveform images will vary due to differences in the sampling resolution and data storage format of the fault waveform recorder. Since deep learning models have fixed requirements for the size of the input images, size unification is necessary. Therefore, the fault waveform sub-network first adjusts and normalizes the size of the input training fault waveform images to obtain standardized training fault waveform images. Then, it performs random erasure and random occlusion on some regions of the standardized training fault waveform images to ensure training efficiency and model robustness, resulting in enhanced training fault waveform images. Specifically, in the design and application, the size of the training fault waveform image can be adjusted to 224×224, and all pixel values ​​of the training fault waveform image can be scaled to the [0, 1] range to obtain a standardized training fault waveform image. Random erasure is then performed on all pixel values ​​of the standardized training fault waveform image, for example, with an erasure probability of 0.3, an erasure area ratio ranging from 0.02 to 0.33, and an erasure region aspect ratio ranging from 0.3 to 3.3.

[0082] Next, after obtaining the enhanced training fault waveform images, feature extraction can be performed. Here, the fault waveform sub-network directly performs image patch segmentation based on the enhanced training fault waveform images, and then the image patch embedding layer within the fault waveform sub-network and... Transformer Feature extraction is performed on the encoder layer, etc., to finally obtain the training fault recording features. ;in, This is represented as the feature representation of each training fault record image after it has been segmented into multiple image blocks; n This is expressed as the number of images per block. Represented as the feature dimension of each image patch, optionally, , No specific restrictions are imposed.

[0083] (2) In the fault cause identification model, the process of obtaining the training weather features is as follows:

[0084] Step B1: Obtain training weather information;

[0085] Specifically, the training weather information is obtained from fault inspection reports, including categories such as sunny, rainy, snowy, and foggy, which are processed to generate feature dimensions. Meanwhile, there are strict classifications for the weather; there is no "thunderstorm and strong wind" weather type. If both "thunderstorm" and "strong wind" weather types exist, "thunderstorm" is prioritized because its classification boundary is clearer than that of "strong wind." This retains the weather category with the clearest and most typical causal relationship to power system faults, excluding composite types and weather types with extremely low correlation to faults. This avoids irrelevant features interfering with the model's learning of the "critical weather-fault" mapping relationship, thus improving model training efficiency.

[0086] It should be noted that when both "thunderstorm" and "strong wind" weather types are present, thunderstorm is prioritized as the primary weather type. This is because the boundaries and identification criteria for thunderstorms are relatively clear, accurately reflecting the occurrence of severe convective weather. While strong winds do have wind speed standards (e.g., a blue strong wind warning indicates average wind speeds of level 6 or higher, or gusts of level 7 or higher), their boundaries are not as clear as those for thunderstorms. Therefore, when these weather phenomena occur simultaneously, thunderstorms are prioritized as the primary weather type. Furthermore, the final annotations in the above weather information are based on the annotations made by staff in the patrol reports; that is, the final weather information was filled in by staff. While staff can visually perceive thunderstorm conditions as severe convective weather, they cannot visually perceive wind speed, as there are no clear boundaries. Other categories follow a similar pattern.

[0087] Step B2: Construct a weather information subnetwork, which integrates... CLIP Text Encoder ;

[0088] In the transmission line fault cause identification model, an integrated system is constructed. CLIP Text Encoder The weather information subnetwork can break through the limitations of traditional weather feature encoding, and at the same time enable weather features and recorded image features to form semantic-visual cross-modal alignment, which greatly improves the effectiveness of multimodal fusion, especially adapting to the diversity of weather-fault associations in complex power grid scenarios.

[0089] For example, the weather information sub-network structure may include: an input preprocessing layer, CLIP Text Encoder The core layer and feature adaptation layer are explained in detail below:

[0090] The input preprocessing layer converts structured weather data (such as numerical values ​​for temperature, wind speed, and lightning intensity) into text descriptions and adds power grid-specific labels to unstructured descriptions (such as "severe convective weather"). The input preprocessing layer also ensures text standardization, guaranteeing that the input text conforms to... CLIP Processing requirements;

[0091] CLIP Text Encoder The core layer includes: a text embedding layer, a positional encoding layer, and Transformer Encoder stacking layer and category token extraction layer; text embedding layer for preprocessed text Token Convert to an initial vector; the positional encoding layer is used to add absolute positional information while preserving the sequential features of the text sequence; Transformer The encoder stacking layer is used to capture semantic relationships between weather elements (such as the synergistic effect of "thunderstorm" and "high wind speed") through multi-head self-attention (e.g., 8 heads). Each layer contains an attention layer, residual connections, layer normalization, and a feedforward network; the category token extraction layer is used to extract... Transformer The first output sequence Token (CLS Token) , as the semantic feature vector of the overall weather information;

[0092] The feature adaptation layer consists of an attention weighting layer, a dimension mapping layer, and a normalization layer. The attention weighting layer is used to... MLP The system learns the importance weights of features in the power grid scenario (e.g., assigning higher weights to dimensions related to "lightning level" and "icing"), thereby strengthening semantics closely associated with faults; the dimension mapping layer uses a fully connected layer to map 512-dimensional features to 768 dimensions (consistent with the feature dimensions of the waveform image), ensuring dimension matching during subsequent fusion; the normalization layer is used to... Layer Norm Standardized feature distribution enhances cross-scenario generalization ability.

[0093] In practical applications, the weather information subnetwork structure here can utilize a contrastive language-image pre-trained model (…). CLIP In the model Text Encoder As an existing structure of the weather information subnetwork, it is based on Transformer Architecture construction, CLIP Text Encoder That is CLIP The text encoder in the code captures the semantic features of the input text. Generally, it first segments the input text into words, converts it into a list of integer indices, and then obtains the corresponding continuous vector representation through an embedding matrix. In practical applications, it can be directly used... CLIP Good pre-training Text Encoder The weather information is input as text into the encoder to obtain weather feature vectors, which can then be used... CLIP The semantic understanding capabilities obtained by pre-training on a large amount of image and text data facilitate cross-modal alignment and fusion of features from other modalities.

[0094] Step B3: Input the training weather information into the weather information sub-network to obtain the transformed high-dimensional weather feature vector; process the high-dimensional weather feature vector to obtain the training weather features.

[0095] Specifically, the weather information subnetwork can utilize CLIP Text EncoderThe word embedding process performs an initial mapping on the training weather information to generate a high-dimensional weather feature vector, which is then input into... CLIP-ViT / B-32 Text Encoder To obtain training weather features based on training weather information ;in, This is represented as a feature representation for each training weather information. Represented as a feature dimension for each piece of weather information, optionally, No specific restrictions are imposed.

[0096] (3) In the fault cause identification model, the process of obtaining the training time features is as follows:

[0097] Step C1: Obtain training time information;

[0098] Specifically, the training time information is categorized into 12 months, and after processing, feature dimensions are generated. It should be noted that the choice of monthly granularity is based on its effectiveness in transmission line fault analysis, thereby fully capturing seasonal patterns and providing sufficient data support to facilitate the model's learning of the correlation between faults and time. At the same time, the monthly granularity balances the time scale and data stability, can reflect the periodic changes in power system faults, and has moderate computational complexity, making it easy to train and apply. For example, "icing" occurs more frequently in January, and "lightning strikes" occur more frequently in July and August.

[0099] Step C2: Construct a time information subnetwork;

[0100] Step C3: Input the training time information into the time information subnetwork, which integrates... Transformer Module; utilizes a multilayer perceptron to perform high-dimensional mapping of training time information, and inputs the high-dimensional time feature vector obtained after high-dimensional mapping into... Transformer The module yields training time features.

[0101] Specifically, the constructed time information subnetwork adopts a "multilayer perceptron (MLP)" architecture. MLP )+ Transformer The "module" architecture design can be broken down into an input processing layer, MLP High-dimensional mapping layer Transformer The encoder consists of four main modules: the core layer, the feature output layer, and so on. These are explained in detail below:

[0102] The input processing layer is the feature encoding layer: it converts the original training time information (such as year / month / day / hour / minute / second, season label, time period label, etc.) into a low-dimensional numerical vector, and then concatenates it with the original numerical time information (such as hour=14 corresponding to 2 pm) to form a unified low-dimensional time input feature vector. MLPHigh-dimensional mapping layers can consist of stacked fully connected layers, activation function layers, and normalization layers. They typically contain 2 to 3 fully connected layers, each followed by a corresponding activation function layer (introducing non-linearity to enhance feature representation). In some scenarios, a normalization layer is added (to stabilize the training process and avoid fluctuations in feature distribution). Ultimately, the low-dimensional temporal features are transformed into a mapping that conforms to the desired shape. Transformer The required high-dimensional time feature vector is input.

[0103] Transformer Encoder core layer: It is based on Transformer The encoder layer is the core, and each encoder layer contains a multi-head self-attention layer, a residual connection layer, a layer normalization layer, and a feedforward network layer. Its function is to first capture the correlation of features at different time steps through the multi-head self-attention layer (such as the temporal dependency between the fault occurrence and various time points within the previous hour). After attention calculation, the input features and attention output features are directly superimposed through the residual connection layer (avoiding gradient vanishing). Then, the layer normalization layer unifies the feature distribution before entering the feedforward network layer to strengthen local feature representation. In practical applications, 2 to 6 such layers are typically stacked. Transformer The encoder layer progressively deepens the capture of temporal and sequential correlations.

[0104] Feature Output Layer: It includes a global pooling layer / category token extraction layer, which is used to extract features of pre-set learnable category tokens at the beginning of the sequence, transforming the temporal features into fixed-dimensional global temporal features as the final training temporal features for subsequent multimodal fusion.

[0105] In practical applications, after receiving training time information, the time information subnetwork can use a multilayer perceptron to perform high-dimensional mapping on the training time information, generating a high-dimensional time feature vector, and then input the high-dimensional time feature vector into... Transformer The module can then obtain training time features based on training time information. ;in, This is represented as a feature representation for each training time information. Represented as a feature dimension for each piece of time information, optionally, No specific restrictions are imposed.

[0106] (4) The process of confirming the training multimodal fusion features in the fault cause identification model is as follows:

[0107] For fault cause identification models, obtaining training multimodal fusion features can be accomplished through a multimodal progressive fusion module. In this module, the waveform fusion features based on training fault waveform images, the weather fusion features based on training weather information, and the time fusion features based on training time information are aligned and then bidirectionally added together to obtain preliminary fusion features. The specific principle is as follows:

[0108] Step D1: Extract the last dimension vector of the training fault recording features, and use it as the recording fusion feature of the training fault recording features. Extract the category token vectors of the training weather features, and use them as the weather fusion features of the training weather features. Extract the category token vectors of the training time features, and use them as the temporal fusion features of the training time features. .

[0109] in, This is represented as the feature representation of each training fault record image after it has been segmented into multiple image blocks; This is represented as a feature representation for each training weather information; This is represented as a feature representation for each training time information.

[0110] Specifically, training fault recording features typically involves outputting a "sequence-type feature matrix" from the fault recording sub-network (taking a common 196×768 matrix as an example: 196 represents the number of image patches, corresponding to 196 16×16 patches after segmenting a 224×224 fault recording image; 768 represents the feature dimension of each image patch). The operation of "extracting the last dimension vector" can be understood as compressing the global core information of this sequence feature. The extraction of weather and time features during training is done using "category token vectors." In deep learning (especially...),... Transformer In the architecture and related models, the category token vector is a feature vector specifically used to aggregate global information. Its main function is to extract overall semantics or global features from serialized features. In practice, the process of obtaining the category token vector is deeply tied to the model structure. Generally, for serialized features such as image patches and text words, its relationship with... class_token The spliced ​​sequence after Transformer After the encoder, it is possible to extract from the output sequence. class_token The corresponding vector (usually the first element of the sequence) is the category token vector of the corresponding feature vector. In this way, the preliminary fusion features obtained by processing the extracted fusion features can effectively solve the problems of multimodal feature dimension mismatch and information redundancy, and achieve efficient cross-modal fusion.

[0111] Step D2: Fuse the waveform recordings with features Weather fusion characteristics Features of time integration After alignment, preliminary fusion features are obtained through bidirectional additive fusion. Convolutional sampling and weighted fusion processing are performed on the preliminary fusion features to obtain the training multimodal fusion features.

[0112] Among them, preliminary fusion features are obtained through bidirectional additive fusion. The process is as follows:

[0113] Waveform fusion features Features integrated with weather The first intermediate feature is obtained after alignment and parallel connection. ; the first intermediate feature Features of time integration The second intermediate feature is obtained after alignment and parallel connection. ; the first intermediate feature Second intermediate features and weather integration characteristics The third intermediate feature is obtained after alignment and parallel connection. ; the third intermediate feature Synthesis and recording characteristics Preliminary fusion features are obtained after alignment and parallel connection. .

[0114] For the fault cause identification model, after obtaining the preliminary fusion features Finally, the training of multimodal fusion features can be completed using an adaptive multi-residual module. ; This is represented as a feature representation for each preliminary fusion feature. It can represent the extended feature dimension within a module, used to enhance feature representation in residual connections; This is represented as a feature representation for each preliminary fusion feature. This can represent the dimension of the final output feature, i.e., the dimension of the training multimodal fusion feature. Specifically, the principle of "convolutional sampling and weighted fusion processing of the preliminary fusion feature to obtain the training multimodal fusion feature" in step D2 is as follows: Convolutional sampling is performed on the preliminary fusion feature to obtain a multi-scale feature layer containing different sampling features; the sampling features under the multi-scale feature layer and the preliminary fusion feature are aligned and then weighted and concatenated to obtain the training multimodal fusion feature.

[0115] Specifically, regarding the initial fusion characteristics Downsampling is performed to obtain a multi-scale feature layer, which includes sampled features. and Here, "downsampling" refers to mapping high-dimensional features to a low-dimensional space using a specific algorithm, generating feature vectors with lower dimensions. The two downsampled features generated here... and This is to preserve information at different levels of compression in order to form multi-scale complementarity.

[0116] In the adaptive multi-residual module, these two sampling features are obtained. and Subsequently, the preliminary fusion characteristics obtained above... After alignment, weighted concatenation yields the multimodal fusion features, which is the basis for training the multimodal fusion features. For example, the multimodal fusion feature dimension here .

[0117] Here, we will explain the processing operations such as alignment and fusion, parallel connection, or weighted serial connection of the different features mentioned above: In the design and application process, whether it is waveform fusion feature, weather fusion feature, time fusion feature, sampling feature, or preliminary fusion feature, their dimensions (resolution, number of channels / length, modal dimension) are usually inconsistent. Therefore, feature alignment is required before parallel connection or serial connection. The core of feature alignment is to make all features to be fused completely identical in the specified dimension. This process can rely on transposed convolution, bilinear interpolation, etc. to expand and compress the feature dimension to ensure that different features can be aligned in a specific dimension.

[0118] Parallel fusion of multiple features refers to aligning all features to be fused, then adding them element by element to obtain the final first intermediate feature mentioned above. Second intermediate features Third intermediate feature and initial fusion characteristics Weighted concatenation refers to first assigning weights to each aligned feature, and then concatenating them sequentially along a specified dimension to obtain the final trained multimodal fusion feature. .

[0119] In a preferred embodiment, when the aforementioned trained multimodal fusion features are obtained... Waveform fusion characteristics Weather fusion characteristics and temporal fusion features Then, the loss values ​​corresponding one-to-one with the training multimodal fusion features, training fault recording features, training weather features, and training time features are calculated to obtain the final loss function. This includes the following steps:

[0120] Step 1: Obtain waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features. Then, use a multilayer perceptron to map the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features into predicted probabilities of multiple fault types. Calculate the loss values ​​corresponding to the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features respectively using the cross-entropy loss function.

[0121] Step 2: Calculate the preliminary loss function based on the loss values ​​corresponding to the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features.

[0122] For the fault cause identification model, the loss function can be optimized through a constructed loss calculation module. This module can be built based on the fully connected layer and the cross-entropy loss function connected to the fully connected layer. In the loss calculation module, four loss values ​​are calculated using the waveform fusion features based on the training fault waveform images, the weather fusion features based on the training weather information, the time fusion features based on the training time information, and the training multimodal fusion features. The four loss values ​​are then weighted to obtain the final loss function, thereby optimizing the transmission line fault cause identification model and obtaining the optimal transmission line fault cause identification model.

[0123] Specifically, waveform fusion characteristics The predicted probabilities of N types of faults are mapped using a multilayer perceptron, and the loss value of the recorded image is calculated using the cross-entropy loss function. Weather fusion characteristics The weather information loss value is calculated by mapping the predicted probabilities of N types of faults to a multilayer perceptron. Temporal fusion features The time information loss value is calculated by mapping the predicted probabilities of N types of faults to a multilayer perceptron. Multimodal fusion features The multimodal fusion loss value is calculated by mapping the multilayer perceptron to the predicted probabilities of N types of faults. It is used to optimize the fault cause identification model for power transmission lines.

[0124] Below, we will discuss the loss value of the waveform image. 、 Weather information loss value 、 Time information loss value and multimodal fusion loss value L The specific calculation process for 4 is as follows:

[0125] First, we define an MLP for waveform image features, which maps each feature to an N-dimensional vector using four different multilayer perceptrons, denoted as: MLP used for weather characteristics, denoted as MLPs used for time features are denoted as... MLP used for multimodal feature fusion is denoted as... .

[0126] Then, the four predicted probability distributions obtained by the Softmax function are as follows:

[0127] Waveform prediction probability: ;

[0128] Weather forecast probability: ;

[0129] Time prediction probability: ;

[0130] Multimodal fusion feature prediction probability: ;

[0131] in, , , and These are respectively represented as waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features.

[0132] Furthermore, the loss value of the recorded image It can be calculated using the following formula (1):

[0133] Formula (1);

[0134] in, Represented as ground truth fault labels, using one-hot encoding (e.g., the ground truth fault category of the sample is the first). i When class, then (If it is 1, then it is 0). This represents the total number of categories of causes of failure. This is represented as the fault cause predicted by the model based on waveform fusion features, belonging to the [missing information - likely a specific category or group]. i The probability of a class.

[0135] Furthermore, the value of weather information loss It can be calculated using the following formula (2):

[0136] Formula (2);

[0137] in, Represented as true fault labels, using one-hot encoding (e.g., the true fault category of the sample is the first fault). i When class, then (If it is 1, then it is 0). This represents the total number of categories of causes of failure; This is represented as the cause of the fault, predicted by the model based on weather fusion features, belonging to the [missing information]. i The probability of a class.

[0138] Furthermore, the time information loss value It can be calculated using the following formula (3):

[0139] Formula (3);

[0140] in, Represented as true fault labels, using one-hot encoding (e.g., the true fault category of the sample is the first fault). i When class, then (If it is 1, then it is 0). This represents the total number of categories of causes of failure; This is represented as the fault cause predicted by the model based on time fusion features, belonging to the [missing information]. i The probability of a class.

[0141] Furthermore, the multimodal fusion loss value It can be calculated using the following formula (4):

[0142] Formula (4);

[0143] in, Represented as true fault labels, using one-hot encoding (e.g., the true fault category of the sample is the first fault). i When class, then (If it is 1, then it is 0). This represents the total number of categories of causes of failure; This is represented as the fault cause predicted by the model based on multimodal fusion features, belonging to the [missing information - likely a specific category or group]. i The probability of a class.

[0144] Next, after obtaining the loss value of the recorded waveform image... Weather information loss value Time information loss value With multimodal fusion loss value Then, by weighting the four loss values, the final loss function is obtained. L For details, please refer to the following formula (5):

[0145] Formula (5).

[0146] in, L 1. , L 3 and L 4 represents the loss value of the waveform image, the loss value of weather information, the loss value of time information, and the loss value of multimodal fusion, respectively. , , and These are respectively represented as the weighting of the loss value proportions for waveform image loss, weather information loss, time information loss, and multimodal fusion loss; for example, , , and The values ​​can be 1, 0.1, 0.1, and 1, but there are no specific limitations. It can be seen that, by jointly calculating the loss values ​​between the multimodal fusion features and each individual modal feature, this embodiment of the invention constructs a loss function that enhances feature consistency and discriminativeness.

[0147] After obtaining the final loss function, the cross-entropy loss function is used to optimize the transmission line fault cause identification model to obtain the optimal transmission line fault cause identification model. In this optimization step, the stochastic gradient descent method can be used to iteratively optimize the parameters of the transmission line fault cause identification model to minimize the loss function, thereby optimizing the initial transmission line fault cause identification model. In the actual experiment, unlike existing methods, the transmission line fault cause identification model in this embodiment of the invention has the most significant improvement for the "lightning strike" fault type, with the recognition rate increasing from 52.5% to 63.2% on the test dataset.

[0148] In practical applications, the optimized transmission line fault cause identification model, after receiving fault waveform images, weather information, and time information, can obtain multimodal fusion features through feature extraction and fusion operations and input them into the fully connected layer, thereby mapping them into the probability distribution of N types of faults. Finally, the model selects the type with the highest probability as the transmission line fault cause identification result, such as "wind deviation" or "lightning strike", without any special restrictions.

[0149] In this embodiment of the invention, the invention defines N=12 types of fault causes, including: "1001 Line disconnection (complete disconnection)", "1002 Line disconnection (sub-conductor disconnection)", "1003 Wind deflection", "1004 Icing (insulator skirt bridging)", "1005 Icing (ice breakage)", "1006 Lightning strike", "1007 Wildfire", "1008 Pollution flashover (bird droppings)", "1009 Pollution flashover (other, other pollution flashover faults besides those caused by bird droppings)", "1010 Foreign object (high resistance)", "1011 Foreign object (low resistance)" and "1012 Other (i.e., other fault cause types besides the above 11 faults)". These classifications are based on historical fault statistics and combined with the experience of power system experts. The classifications are refined (e.g., distinguishing between sub-conductor disconnection and complete disconnection), and based on the correlation between voltage / current mutation patterns and environmental factors in waveform characteristics, ensuring that the model covers common and specific fault types.

[0150] For example, when fault waveform images, weather information, and time information are input into the fault cause identification model, the multimodal fusion feature mapping is mapped to the probability distribution of N types of faults as follows: "1001 Line disconnection (complete disconnection)", 1%; "1002 Line disconnection (sub-conductor disconnection)", 2%; "1003 Wind deflection", 1%; "1004 Icing (insulator skirt bridging)", 1%; "1005 Icing (ice breakage jump)", 3%; "1006 Lightning strike", 82%; "1007 Wildfire", 1%; "1008 Pollution flashover (bird droppings)", 2%; "1009 Pollution flashover (other)", 1%; "1010 Foreign object (high resistance)", 2%; "1011 Foreign object (low resistance)", 3%; "1012 Other", 1%. Then, the final fault cause identification result output by the fault cause identification model should be 1006 Lightning strike.

[0151] For downstream systems, when the fault cause identification result is obtained, the output result will trigger an automatic response. For example, after the fault cause is "1006 Lightning Strike", the system will automatically isolate the faulty segment and send an alarm to the dispatch center. At the same time, the maintenance team will arrange targeted actions based on the result. For example, when the fault cause is "1008 Pollution Flashover (Bird Droppings)", the insulators will be cleaned. When the fault cause is "1004 Icing (Insulator Skirt Bridging)", the faulty equipment will be inspected to ensure rapid restoration of power supply.

[0152] The above text combined Figure 1 The method for identifying the cause of transmission line faults based on multimodal information provided in the embodiments of the present invention has been described in detail below. Figure 2 , Figure 3 The system, device and medium provided in the embodiments of the present invention will be described.

[0153] Based on the transmission line fault cause identification method based on multimodal information shown in Embodiment 1, this embodiment of the invention proposes a transmission line fault cause identification system based on multimodal information. The identification system 600 includes:

[0154] The identification module 601 is used to acquire the current fault waveform image, weather information and time information; input the fault waveform image, weather information and time information into the fault cause identification model to obtain the final fault cause; wherein, the fault cause identification model is optimized using a set loss function;

[0155] Optimization module 602 is used to obtain the loss function. The optimization module includes the following:

[0156] The feature extraction unit 6021 is used to extract feature information from the training fault recording image, training weather information and training time information to obtain training fault recording features, training weather features and training time features.

[0157] The calculation unit 6022 is used to calculate the training multimodal fusion features based on the training fault recording features, training weather features, and training time features; and to calculate the loss values ​​corresponding one-to-one with the training multimodal fusion features, training fault recording features, training weather features, and training time features, so as to calculate the final loss function.

[0158] In some embodiments, the process of obtaining the training fault recording features is as follows:

[0159] The feature extraction unit 6021 is used to acquire training fault waveform images and construct a fault waveform sub-network; the training fault waveform images are input into the fault waveform sub-network, and the fault waveform sub-network is used to extract features from the input training fault waveform images to obtain training fault waveform features.

[0160] In some embodiments, the process of obtaining training weather features is as follows:

[0161] The feature extraction unit 6021 is used to acquire training weather information and construct a weather information sub-network. The training weather information is input into the weather information sub-network to obtain a transformed high-dimensional weather feature vector. The training weather features are obtained by processing the high-dimensional weather feature vector.

[0162] In some embodiments, the process of obtaining training time features is as follows:

[0163] The feature extraction unit 6021 is used to acquire training time information and construct a time information subnetwork; the training time information is input into the time information subnetwork, which integrates... Transformer Module; utilizes a multilayer perceptron to perform high-dimensional mapping of training time information, and inputs the high-dimensional time feature vector obtained after high-dimensional mapping into... Transformer The module yields training time features.

[0164] In some embodiments, the training multimodal fusion features are calculated, including:

[0165] The feature extraction unit 6021 is used to extract the last dimension vector of the training fault recording feature as the recording fusion feature of the training fault recording feature; extract the category token vector of the training weather feature as the weather fusion feature of the training weather feature; extract the category token vector of the training time feature as the time fusion feature of the training time feature; and after aligning the recording fusion feature, weather fusion feature and time fusion feature, obtain the preliminary fusion feature through bidirectional addition fusion; and perform convolution sampling and weighted fusion processing on the preliminary fusion feature to obtain the training multimodal fusion feature.

[0166] In some embodiments, after aligning the waveform fusion features, weather fusion features, and time fusion features, a preliminary fusion feature is obtained through bidirectional additive fusion, including:

[0167] The feature extraction unit 6021 is used to align and connect the waveform fusion feature and the weather fusion feature to obtain the first intermediate feature; align and connect the first intermediate feature and the time fusion feature to obtain the second intermediate feature; align and connect the first intermediate feature, the second intermediate feature and the weather fusion feature to obtain the third intermediate feature; and align and connect the third intermediate feature and the waveform fusion feature to obtain the preliminary fusion feature.

[0168] In some embodiments, convolutional sampling and weighted fusion processing are performed on the preliminary fused features to obtain training multimodal fused features, including:

[0169] The feature extraction unit 6021 is used to perform convolutional sampling on the preliminary fused features to obtain a multi-scale feature layer containing different sampling features; after aligning the sampling features under the multi-scale feature layer and the preliminary fused features, a weighted concatenation process is performed to obtain the training multimodal fused features.

[0170] In some embodiments, loss values ​​corresponding one-to-one with the training multimodal fusion features, training fault recording features, training weather features, and training time features are calculated to obtain the final loss function, including:

[0171] The computing unit 6022 is used to acquire waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features. It then uses a multilayer perceptron to map these features to predict the probabilities of various fault types. The cross-entropy loss function is used to calculate the loss values ​​corresponding to the waveform fusion features, weather fusion features, time fusion features, and training multimodal fusion features. Based on the loss values ​​corresponding to these features, the loss function is calculated.

[0172] The present invention also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a method for identifying the cause of transmission line faults based on multimodal information as described in Embodiment 1.

[0173] In this embodiment, as Figure 3As shown, the terminal device 500 includes a CPU 501 (Central Processing Unit), which can perform various appropriate actions and processes according to a program stored in ROM 502 (Read-Only Memory) or a program loaded from storage into RAM 503 (Random Access Memory). RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via bus 504. An I / O interface 505 (Input / Output Interface) is also connected to bus 504.

[0174] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0175] In particular, according to embodiments of the present invention, the above-described flowchart is as follows. Figure 1 The described process can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by CPU 501, it performs the functions defined above in the system of the present invention.

[0176] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM 503, ROM 502, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0177] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0178] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself. The described units or modules can also be located in a processor; for example, a processor can be described as including a first generation module, an acquisition module, a search module, a second generation module, and a merging module. The names of these units or modules do not necessarily limit the specific unit or module itself; for example, the acquisition module can also be described as "an acquisition module for acquiring multiple instances to be probed in the base table".

[0179] The present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the transmission line fault cause identification method based on multimodal information as described in the above embodiments.

[0180] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention is not limited to the specific combination of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in this invention.

Claims

1. A power transmission line fault cause identification method based on multi-modal information, characterized in that, The method comprises the following steps: obtaining current fault recording image, weather information and time information; inputting the fault recording image, weather information and time information into a fault cause identification model to obtain a final fault cause; wherein the fault cause identification model is obtained by using a set loss function for optimization; the loss function comprises the following steps: extracting feature information from training fault recording image, training weather information and training time information to obtain training fault recording feature, training weather feature and training time feature; based on the training fault recording feature, training weather feature and training time feature, a training multi-modal fusion feature is calculated; a loss value corresponding to the training multi-modal fusion feature, training fault recording feature, training weather feature and training time feature is calculated to obtain the final loss function; based on the training fault recording feature, training weather feature and training time feature, a training multi-modal fusion feature is calculated, comprising: extracting the last dimension vector of the training fault recording feature as the recording fusion feature of the training fault recording feature; extracting the category token vector of the training weather feature as the weather fusion feature of the training weather feature; extracting the category token vector of the training time feature as the time fusion feature of the training time feature; aligning the recording fusion feature, weather fusion feature and time fusion feature, and then obtaining a preliminary fusion feature through bidirectional addition fusion; performing convolution sampling and weighted fusion processing on the preliminary fusion feature to obtain a training multi-modal fusion feature; calculating a loss value corresponding to the training multi-modal fusion feature, training fault recording feature, training weather feature and training time feature to obtain the final loss function, comprising: obtaining recording fusion feature, weather fusion feature, time fusion feature and training multi-modal fusion feature, and mapping the recording fusion feature, weather fusion feature, time fusion feature and training multi-modal fusion feature into the prediction probability of multi-class fault through a multi-layer perception machine, and calculating the loss value corresponding to the recording fusion feature, weather fusion feature, time fusion feature and training multi-modal fusion feature through a cross-entropy loss function; According to the loss value corresponding to the recording fusion feature, weather fusion feature, time fusion feature and training multi-modal fusion feature, the loss function is calculated.

2. The method of claim 1, wherein, The process of obtaining the training fault recording feature is as follows: obtaining a training fault recording image and constructing a fault recording subnetwork; inputting the training fault recording image into the fault recording subnetwork, and using the fault recording subnetwork to extract features from the input training fault recording image to obtain the training fault recording feature. 3.The power transmission line fault cause identification method based on multi-modal information according to claim 1, characterized in that, The process of obtaining the training weather feature is as follows: obtaining training weather information and constructing a weather information subnetwork, inputting the training weather information into the weather information subnetwork to obtain a converted high-dimensional weather feature vector; by processing the high-dimensional weather feature vector, the training weather feature is obtained. 4.The power transmission line fault cause identification method based on multi-modal information according to claim 1, characterized in that, The process of obtaining the training time feature is as follows: Obtain training time information, and construct a time information subnetwork; input the training time information into the time information subnetwork, and the time information subnetwork is integrated with Transformer A module; use a multi-layer perception machine to perform high-dimension mapping on the training time information, input a high-dimension time feature vector obtained after the high-dimension mapping into Transformer A module, and obtain the training time feature.

5. The method of claim 1, wherein, After aligning the recording wave fusion features, weather fusion features and time fusion features, a preliminary fusion feature is obtained through bidirectional addition fusion, including: A first intermediate feature is obtained after aligning and connecting the recording wave fusion features and the weather fusion features in parallel; A second intermediate feature is obtained after aligning and connecting the first intermediate feature and the time fusion feature in parallel; A third intermediate feature is obtained after aligning and connecting the first intermediate feature, the second intermediate feature and the weather fusion feature in parallel; The third intermediate feature and the recording wave fusion feature are aligned and connected in parallel to obtain a preliminary fusion feature.

6. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 5.

Images