Power transmission line state monitoring method, device, system, electronic equipment and storage medium

Abstract

This application provides a method, apparatus, system, electronic device, and storage medium for monitoring the condition of transmission lines. The method includes: acquiring multimodal data collected by a terminal device; performing anomaly detection based on the multimodal data using multiple preset anomaly detection models to obtain anomaly status results; constructing a multi-relationship topology diagram based on the anomaly status results and the attribute information of the terminal device using a preset multimodal semantic model, and generating a natural language description of the multi-relationship topology diagram in a transmission line scenario; and determining the fault diagnosis result of the transmission line based on the natural language description using a preset large-scale power industry model. This application improves the accuracy and completeness of transmission line monitoring through the collaborative processing of multiple models.

Description

Technical Field

[0001] This application relates to the field of power grid technology, and in particular to a method, device, system, electronic device, and storage medium for monitoring the condition of transmission lines. Background Technology

[0002] With the development of the power industry and the continuous expansion of power grid coverage, the operation and maintenance of transmission lines face enormous challenges. Transmission line operation and maintenance systems deployed in complex terrain, remote areas, and other special environments mainly rely on terminal equipment such as sensors and drone-mounted devices for automated inspections. Based on data collected by these terminal devices, specific technical means are used to monitor line status. For example, the principle of light occlusion is used to detect changes in cable diameter to detect ice thickness; images captured by cameras are used to detect line defects; and multimodal fusion models are used to perceive the status after feature fusion based on multimodal data. However, none of these methods consider the dynamic correlation between the multimodal data collected by the terminal devices, resulting in unsatisfactory line monitoring performance. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, device, system, electronic device and storage medium for monitoring the condition of power transmission lines.

[0004] To achieve the above objectives, this application provides a method for monitoring the condition of a power transmission line, comprising:

[0005] Acquire multimodal data collected by terminal devices;

[0006] Anomalies are detected based on the multimodal data using multiple preset anomaly detection models to obtain anomaly state results;

[0007] Using a pre-defined multimodal semantic model, a multi-relationship topology diagram is constructed based on the abnormal state results and the attribute information of the terminal device, and a natural language description of the multi-relationship topology diagram in the power transmission line scenario is generated.

[0008] Based on the natural language description, the fault diagnosis results of the transmission line are determined using a pre-defined large-scale power industry model.

[0009] Optionally, the attribute information includes location information, and the terminal device includes environmental sensors and line sensors; the step of constructing a multi-relationship topology diagram based on the abnormal state result and the attribute information of the terminal device using a preset multimodal semantic model includes:

[0010] The physical connection relationships between terminal devices are determined based on the location information of each terminal device.

[0011] Based on the correlation of data changes among various terminal devices, determine the functional dependencies between terminal devices;

[0012] Based on the location information and data change correlation of the environmental sensors and line sensors, the spatiotemporal correlation between the environmental sensors and line sensors is determined;

[0013] Using the terminal device as a node, a multi-relationship topology diagram is constructed, including the physical connection relationship, functional dependency relationship and spatiotemporal association relationship.

[0014] Optionally, using the terminal device as a node, a multi-relationship topology diagram is constructed, including the physical connection relationship, functional dependency relationship, and spatiotemporal association relationship, including:

[0015] Based on the physical connection relationship, physical connection edges are generated between each node;

[0016] Connect any two nodes that do not have a physical connection edge with a functional dependency edge, calculate the weight of each functional dependency edge, and retain functional dependency edges whose weight is greater than or equal to a preset functional dependency threshold.

[0017] Based on the spatiotemporal correlation, spatiotemporal correlation edges are generated between the nodes corresponding to the environmental sensors and the line sensors, and the weight of each spatiotemporal correlation edge is calculated. Spatiotemporal correlation edges with weights greater than or equal to a preset spatiotemporal correlation threshold are retained.

[0018] Construct a multi-relationship topology graph based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal association edges.

[0019] Optionally, the method for calculating the weights of each functional dependency edge is as follows:

[0020]

[0021] in, Let be the weight of the functional dependency edge between nodes u and v. and Let be the projection matrix of the r-th trainable attention head. For query vector, For key vectors, Data collected at time t for nodes u and v are shown below, respectively, and H represents the number of attention heads. For the normalized attention weights, d k Let be the dimension of the key vector. The normalized attention weights W for the i-th attention head r,i Let be the trainable weight matrix for the i-th attention head.

[0022] Optionally, the method for calculating the weights of each spatiotemporal related edge is as follows:

[0023]

[0024] in, γ represents the weight of the spatiotemporal association edge between node u and node v; uv (τ) represents the time decay factor for nodes u and v, τ∈[t-Δt,t] is the time sliding window, and σ is a parameter used to control the time decay rate; β uv p is the spatial decay factor for nodes u and v. u p v These represent the location information of nodes u and v, respectively, and d0 is the distance scale parameter; ρ uv (τ) represents the temporal covariance between node u and node v, u τ For the data collected by node u at time τ, v t For the data collected by node v at time t, σ u σ v Let be the standard deviation of the data for node u and node v.

[0025] Optionally, generating a natural language description of the multi-relationship topology diagram in a transmission line scenario includes:

[0026] The node information of each node in the multi-relation topology graph is encoded to obtain the node feature representation of each node.

[0027] Retrieve prior knowledge related to each node from a pre-defined knowledge graph of the power sector;

[0028] Based on the prior knowledge of each node and the node feature representation of neighboring nodes, aggregated node features are obtained; wherein, the neighboring node is a node connected to the node through any kind of edge.

[0029] A natural language description is generated based on the features of the aggregated nodes.

[0030] Optionally, the node information includes the data collected by the node, the timestamp of the collected data, the node's location information, and the node's device type;

[0031] The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph. The method is as follows:

[0032]

[0033] Where GELU is the Gaussian linear unit activation function, W enc and b enc For learnable weights and bias parameters, onehot(d) u) represents the device type encoding d for node u. u Perform one-hot encoding, x u p u t u These are, respectively, the data collected by node u, the location information of node u, and the timestamp of the data collected by node u. Let be the initial node feature representation of node u.

[0034] Optionally, the multimodal data includes sensor data, visible light image data, infrared image data, and lidar data; the multiple anomaly detection models include sensor detection models, visible light image detection models, infrared image detection models, and lidar detection models.

[0035] The step of using multiple preset anomaly detection models to perform anomaly detection on the multimodal data to obtain anomaly state results includes:

[0036] The sensor detection model is used to detect anomalies in the sensor data to obtain the sensor data anomaly status results.

[0037] The visible light image detection model is used to detect anomalies in the visible light image data to obtain the results of the visible light image anomaly status.

[0038] The infrared image detection model is used to detect anomalies in the infrared image data to obtain the results of abnormal infrared image states.

[0039] The lidar detection model is used to detect anomalies in the lidar data, and the abnormal state results of the lidar data are obtained.

[0040] This application embodiment also provides a power transmission line condition monitoring device, including:

[0041] The acquisition module is used to acquire multimodal data collected by the terminal device;

[0042] An anomaly detection module is used to perform anomaly detection based on the multimodal data using multiple preset anomaly detection models to obtain anomaly state results;

[0043] The semantic generation module is used to construct a multi-relationship topology diagram based on the abnormal state results and the attribute information of the terminal device using a preset multimodal semantic model, and generate a natural language description of the multi-relationship topology diagram in the power transmission line scenario.

[0044] The fault diagnosis module is used to determine the fault diagnosis results of transmission lines based on the natural language description using a preset large power industry model.

[0045] Optionally, the attribute information includes location information, and the terminal device includes environmental sensors and line sensors;

[0046] The semantic generation module is used to determine the physical connection relationship between terminal devices based on the location information of each terminal device; determine the functional dependency relationship between terminal devices based on the data change correlation of each terminal device; determine the spatiotemporal correlation relationship between the environmental sensors and the line sensors based on the location information and data change correlation of the environmental sensors and the line sensors; and construct a multi-relationship topology diagram including the physical connection relationship, functional dependency relationship and spatiotemporal correlation relationship with the terminal devices as nodes.

[0047] Optionally, the semantic generation module is used to generate physical connection edges between nodes based on the physical connection relationship; connect any two nodes that do not have physical connection edges through functional dependency edges, and calculate the weight of each functional dependency edge, retaining functional dependency edges with a weight greater than or equal to a preset functional dependency threshold; generate spatiotemporal association edges between nodes corresponding to environmental sensors and line sensors based on the spatiotemporal association relationship, and calculate the weight of each spatiotemporal association edge, retaining spatiotemporal association edges with a weight greater than or equal to a preset spatiotemporal association threshold; and construct a multi-relationship topology graph based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal association edges.

[0048] Optionally, the method for calculating the weights of each functional dependency edge is as follows:

[0049]

[0050] in, Let be the weight of the functional dependency edge between nodes u and v. and Let be the projection matrix of the r-th trainable attention head. For query vector, For key vectors, Data collected at time t for nodes u and v are shown below, respectively, and H represents the number of attention heads. For the normalized attention weights, d k Let be the dimension of the key vector. The normalized attention weights W for the i-th attention head r,i Let be the trainable weight matrix for the i-th attention head.

[0051] Optionally, the method for calculating the weights of each spatiotemporal related edge is as follows:

[0052]

[0053] in, γ represents the weight of the spatiotemporal association edge between node u and node v; uv (τ) represents the time decay factor for nodes u and v, τ∈[t-Δt,t] is the time sliding window, and σ is a parameter used to control the time decay rate; β uv p is the spatial decay factor for nodes u and v. u p v These represent the location information of nodes u and v, respectively, and d0 is the distance scale parameter; ρ uv (τ) represents the temporal covariance between node u and node v, u τ For the data collected by node u at time τ, v t For the data collected by node v at time t, σ u σ v Let be the standard deviation of the data for node u and node v.

[0054] Optionally, the semantic generation module is used to encode the node information of each node in the multi-relation topology graph to obtain the node feature representation of each node; retrieve prior knowledge related to each node from a preset power field knowledge graph; obtain aggregated node features based on the prior knowledge of each node and the node feature representation of neighboring nodes; wherein, the neighboring nodes are nodes connected to the node through any kind of edge; and generate a natural language description based on the aggregated node features.

[0055] Optionally, the node information includes the data collected by the node, the timestamp of the collected data, the node's location information, and the node's device type;

[0056] The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph. The method is as follows:

[0057]

[0058] Where GELU is the Gaussian linear unit activation function, W enc and b enc For learnable weights and bias parameters, onehot(d) u ) represents the device type encoding d for node u. u Perform one-hot encoding, x u p u t u These are, respectively, the data collected by node u, the location information of node u, and the timestamp of the data collected by node u. Let be the initial node feature representation of node u.

[0059] Optionally, the multimodal data includes sensor data, visible light image data, infrared image data, and lidar data; the multiple anomaly detection models include sensor detection models, visible light image detection models, infrared image detection models, and lidar detection models.

[0060] The anomaly detection module is used to perform anomaly detection on the sensor data using the sensor detection model to obtain anomaly status results for the sensor data; to perform anomaly detection on the visible light image data using the visible light image detection model to obtain anomaly status results for the visible light image; to perform anomaly detection on the infrared image data using the infrared image detection model to obtain anomaly status results for the infrared image; and to perform anomaly detection on the lidar data using the lidar detection model to obtain anomaly status results for the lidar data.

[0061] This application embodiment also provides a transmission line condition monitoring system for implementing the aforementioned transmission line condition monitoring method. The system includes:

[0062] Terminal devices deployed at the edge, and multiple anomaly detection models, multimodal semantic models, and large-scale power industry models deployed in the cloud.

[0063] This application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the transmission line condition monitoring method.

[0064] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the transmission line status monitoring method.

[0065] As can be seen from the above, the transmission line condition monitoring method, device, system, electronic equipment, and storage medium provided in this application's embodiments utilize multiple anomaly detection models to perform anomaly detection based on multimodal data collected by terminal devices, obtaining anomaly state results. A multimodal semantic model is then used to construct a multi-relationship topology diagram based on the anomaly state results and the attribute information of the terminal devices, generating a natural language description of the multi-relationship topology diagram in the transmission line scenario. Finally, a large-scale power industry model is used to determine the fault diagnosis results of the transmission line based on the natural language description. This application, through the collaborative processing of multiple models and the dynamic correlation analysis of multimodal data, can improve the accuracy and completeness of line monitoring results. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of the method flow of an embodiment of this application;

[0068] Figure 2 This is a schematic diagram of a method flow according to another embodiment of this application;

[0069] Figure 3 This is a schematic diagram illustrating the collaborative processing flow of multiple models in an embodiment of this application;

[0070] Figure 4 This is a schematic diagram illustrating the processing procedure of the multimodal semantic model in an embodiment of this application;

[0071] Figure 5 A schematic diagram illustrating the process of constructing a multi-relationship topology graph for embodiments of this application;

[0072] Figure 6 This is a multi-relationship topology diagram constructed according to one embodiment of this application;

[0073] Figure 7 This is a schematic diagram illustrating the processing procedure of a large-scale power industry model according to an embodiment of this application;

[0074] Figure 8 This is a schematic diagram comparing the monitoring effects of embodiments of this application;

[0075] Figure 9 This is a block diagram of the device structure according to an embodiment of this application;

[0076] Figure 10 This is a schematic diagram of the system deployment architecture according to an embodiment of this application;

[0077] Figure 11 This is a schematic diagram of some functions of the system according to an embodiment of this application;

[0078] Figure 12 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation

[0079] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0080] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0081] In related technologies, lightweight, low-latency small models deployed on edge devices can detect single states of transmission lines in real time, such as using sensor detection models to detect whether indicators like temperature and humidity are abnormal. However, limited by computing power, small models struggle to integrate the complex interactions between multimodal data for multiphysics coupling analysis (such as simultaneously processing multiple data points like wind speed, conductor temperature, and mechanical stress, and analyzing their coupling relationships). They cannot deeply extract associated features, and many states remain unidentified, resulting in low accuracy and completeness of line monitoring, impacting the efficiency of transmission line operation and maintenance. Large multimodal models with higher processing power can achieve feature fusion and multiphysics coupling analysis of multimodal data, but they consume significant resources, have high latency, struggle to achieve real-time detection and response, and have poor adaptability to specific scenarios.

[0082] In view of this, this application provides a method for monitoring the condition of transmission lines. It utilizes multiple lightweight anomaly detection models to perform real-time detection of corresponding modal data, initially determining the abnormal state results. Then, based on the abnormal state results and the attribute information of terminal devices, a multimodal semantic model is used to construct a multi-relationship topology diagram among the terminal devices and generate corresponding natural language descriptions. Finally, a large-scale power industry model is used to determine the fault diagnosis results for the power industry scenario based on the natural language descriptions. Through the collaborative processing of multiple models and the dynamic correlation analysis of multimodal data, the accuracy and completeness of line monitoring results can be improved.

[0083] The technical solution of this application will be further described in detail below through specific embodiments.

[0084] like Figure 1-3 As shown in the figure, this application provides a method for monitoring the condition of a power transmission line, including:

[0085] S101: Acquire multimodal data collected by the terminal device;

[0086] In some embodiments, multiple terminal devices are deployed in the power transmission line condition monitoring system to collect line condition data and environmental data. These terminal devices include various sensors, image acquisition units, lidar, etc., deployed at specific locations along the line, and sensors, image acquisition units, lidar, etc., carried by drones. Sensors include, but are not limited to, temperature sensors, humidity sensors, vibration sensors, wind speed sensors, air pressure sensors, and current sensors. Sensors can be categorized into environmental sensors for collecting environmental data and line sensors for collecting line condition data. Image acquisition units include infrared image acquisition units and visible light image acquisition units. The multimodal data collected by various terminal devices includes sensor data, visible light image data, infrared image data, lidar data, and other data from different modes.

[0087] The attribute information of the terminal device includes the collected data, the timestamp of the collected data, the device identifier, the device type, the location information, the geographical location, the performance indicators, the network segment to which it belongs, and the geographical location of the network segment. Among them, for the fixedly installed terminal device, the location information is the coordinates of the installation location, and for the drone, the location information is the coordinates of the collected data.

[0088] S102: Utilize multiple preset anomaly detection models to perform anomaly detection based on multimodal data, and obtain anomaly state results;

[0089] In this embodiment, multiple lightweight, low-latency anomaly detection models are configured. Each anomaly detection model is used to perform real-time preliminary anomaly detection on a corresponding type of modal data to obtain the corresponding anomaly state results.

[0090] In some implementations, multiple anomaly detection models include a sensor detection model for detecting sensor data, a visible light image detection model for detecting visible light image data, an infrared image detection model for detecting infrared image data, and a lidar detection model for detecting lidar data. Thus, the sensor detection model can be used to detect anomalies in sensor data to obtain sensor data anomaly status results; the visible light image detection model can be used to detect anomalies in visible light image data to obtain visible light image anomaly status results; the infrared image detection model can be used to detect anomalies in infrared image data to obtain infrared image anomaly status results; and the lidar detection model can be used to detect anomalies in lidar data to obtain lidar data anomaly status results.

[0091] In some approaches, sensor detection models are implemented based on the Transformer model, enabling anomaly detection (e.g., whether line temperature is within the normal range) for sensor data acquired sequentially over time. Visible light image detection models are implemented based on the YOLO target detection model, enabling anomaly detection (e.g., tower corrosion defects) for specific targets in visible light images. Infrared image detection models are implemented based on the PatchCore detection model, enabling anomaly detection (e.g., line defects) for specific targets in infrared images. LiDAR detection models are implemented based on the PointNet target detection model, enabling detection of radar point cloud data and identification of abnormal target states (e.g., transmission tower tilt). These are merely illustrative examples; the structure and specific implementation of each anomaly detection model are not specifically limited.

[0092] S103: Using a pre-defined multimodal semantic model, construct a multi-relationship topology diagram based on abnormal state results and terminal device attribute information, and generate a natural language description of the multi-relationship topology diagram in the transmission line scenario;

[0093] In this embodiment, after multiple anomaly detection models detect existing abnormal states in real time, the relevant data of the detected abnormal state results and the attribute information of the terminal devices related to the anomaly are input into a multimodal semantic model. Based on the relevant data of the abnormal state results and the attribute information of the terminal devices, the multimodal semantic model analyzes the possible multiple relationships between the terminal devices, constructs a multi-relationship topology diagram between the terminal devices, and generates a natural language description of the multi-relationship topology diagram in the transmission line. The terminal devices related to the anomaly can be those that are related to the terminal device that detected the abnormal state result in terms of physical connection, functional dependency, spatiotemporal correlation, etc., or they can be terminal devices within a certain distance range from the terminal device that detected the abnormal state result.

[0094] like Figure 4As shown, in some implementations, the multimodal semantic model includes two parts: a topology graph construction module and a natural language description generation module. The topology graph construction module is implemented based on a Hybrid Graph Neural Network (HGNN) architecture. HGNNs have the ability to construct multi-relationship graphs, enabling them to capture the complex and heterogeneous relationships between terminal devices in transmission lines. However, conventional HGNNs have certain limitations when applied to constructing the relationship topology graph of transmission lines. These limitations mainly include: firstly, the dependencies between multimodal data in transmission lines are constantly changing dynamically. For example, as the ice thickness on the conductor increases from 5mm to 15mm, the tension data collected by the tension sensor and the tilt data collected by the tower tilt sensor increase significantly, and the relationship weights between the tension sensor and the tilt sensor increase significantly. Traditional HGNNs cannot dynamically adjust the weights of these relationships based on the dynamically monitored data, and therefore cannot truly reflect the overall state change process of the line. Secondly, due to the structural influence of towers and transmission lines, there is a certain response delay between meteorological factors and the data collected by sensors affected by these factors. For example, 30 minutes after a strong wind, the tilt sensor on the tower will detect the maximum offset. Traditional hybrid graph neural networks do not analyze the delay between meteorological factors and sensor data and lack the ability to model the corresponding correlation. Thirdly, in the transmission line scenario, the spatial distance between terminal devices is related to the correlation of data changes, i.e., there is a spatial attenuation effect. For example, if the temperature sensor and the tension sensor are close, the correlation between the collected temperature data and tension data is greater (e.g., increased icing leads to a decrease in temperature and an increase in tension). If the temperature sensor and the tension sensor are far apart, the correlation between the collected temperature data and tension data is smaller, and the relationship between temperature changes and tension changes is not significant. The relational topology graph constructed by traditional hybrid graph neural networks does not consider the spatial attenuation effect.

[0095] Therefore, in order to accurately construct a multi-relationship topology diagram that conforms to the multimodal data in the transmission line scenario, it is necessary to make corresponding improvements to the graph neural network based on the multiple relationships and influencing factors of the transmission line, and to construct a multi-relationship topology diagram suitable for the transmission line scenario based on the improved graph neural network.

[0096] like Figure 5 As shown, the methods for constructing a multi-relationship topology graph include:

[0097] The physical connection relationships between terminal devices are determined based on the location information of each terminal device.

[0098] Based on the correlation of data changes among various terminal devices, determine the functional dependencies between terminal devices;

[0099] Based on the location information and data change correlation of environmental sensors and line sensors, the spatiotemporal correlation between environmental sensors and line sensors is determined;

[0100] Using terminal devices as nodes, construct a multi-relationship topology diagram that includes physical connections, functional dependencies, and spatiotemporal relationships.

[0101] In this embodiment, the relationships between terminal devices are divided into three categories: physical connection relationships, functional dependency relationships, and spatiotemporal correlation relationships. Specifically, based on the location information of the terminal devices, the physical connection relationships between the terminal devices are determined. Terminal devices fixedly installed on the same equipment have physical connection relationships. For example, the infrared image acquisition unit and the visible light image acquisition unit installed on the same drone have a fixed physical connection relationship, and the tilt sensor and the temperature sensor installed on the same tower have a fixed physical connection relationship. The data collected by terminal devices with physical connection relationships have correlation relationships. Based on the correlation of data changes of the terminal devices, the functional dependency relationships between the terminal devices are determined. The functional dependency relationships reflect the implicit functional correlations and data synchronization relationships between the data collected by the terminal devices. For example, there is a correlation of data changes between icing thickness and conductor tension and tilt angle. Based on the location information and data change correlations of environmental sensors and line sensors, the spatiotemporal correlation relationships between environmental sensors and line sensors are determined. The spatiotemporal correlation relationships reflect the delay and spatial attenuation effects between the environmental data collected by the environmental sensors and the line status data collected by the line sensors. For example, the wind speed collected by the wind speed sensor is a strong wind, and 30 minutes after the strong wind, the tilt angle collected by the tilt sensor of the tower reaches its maximum value. After analyzing and determining the relationships between various terminal devices, the terminal devices are used as nodes to construct a multi-relationship topology diagram that includes physical connections, functional dependencies, and spatiotemporal relationships between nodes.

[0102] In some implementations, a multi-relationship topology graph is constructed using terminal devices as nodes, including physical connections, functional dependencies, and spatiotemporal associations, including:

[0103] Based on the physical connection relationships, generate physical connection edges between each node and calculate their weights;

[0104] Connect any two nodes that do not have a physical connection edge with a functional dependency edge, calculate the weight of each functional dependency edge, and retain functional dependency edges whose weight is greater than or equal to a preset functional dependency threshold.

[0105] Based on the spatiotemporal correlation, spatiotemporal correlation edges are generated between the nodes corresponding to environmental sensors and line sensors, and the weight of each spatiotemporal correlation edge is calculated. Spatiotemporal correlation edges with weights greater than or equal to the preset spatiotemporal correlation threshold are retained.

[0106] Construct a multi-relationship topology graph based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal association edges.

[0107] In this embodiment, after determining the physical connection relationship between terminal devices, physical connection edges are generated between the nodes of the corresponding terminal devices. Since physical connection edges can reflect the data synchronization between two nodes, there is no need to generate additional functional dependency edges. Therefore, physical connection edges and functional dependency edges are mutually exclusive, and the two types of edges do not need to exist at the same time. To solve this problem, all physical connection edges are retained, and any two nodes without physical connection edges are connected through functional dependency edges. The weight of each functional dependency edge is calculated, and only functional dependency edges with a weight greater than or equal to a preset functional dependency threshold are retained. Spatiotemporal correlation edges are generated between the nodes corresponding to environmental sensors and the nodes corresponding to line sensors, and the weight of each spatiotemporal correlation edge is calculated. Only spatiotemporal correlation edges with a weight greater than or equal to a preset spatiotemporal correlation threshold are retained. Finally, each node and the physical connection edges, functional dependency edges, and spatiotemporal correlation edges between nodes constitute a complete multi-relationship topology graph.

[0108] In some methods, the physical connection edge E between nodes physical It can be represented as:

[0109] E physical ={(u,v)|u and v are physically connected} (1)

[0110] The method for calculating the weight of physical connection edges is as follows:

[0111]

[0112] in, Let be the weight of the physical connection edge between nodes u and v. The data collected by nodes u and v at time t are respectively, and W is the data collected by nodes u and v at time t. p and b p Here are the trainable weights and bias parameters, and σ() is the activation function, such as the sigmoid activation function.

[0113] Connect any two nodes that do not have a physical connection edge with a functional dependency edge to construct a potential functional dependency relationship. The functional dependency edge E between the nodes is... func It can be represented as:

[0114]

[0115] For all functionally dependent edges, based on a time-varying relational attention mechanism, the weights of each functionally dependent edge are calculated in real time according to dynamically collected data. The method is as follows:

[0116]

[0117] in, and Let be the projection matrix of the r-th trainable attention head, used to map the data features of the node to the query space and the key space; The query vector represents the ability of node u to influence other nodes at time t; Let be the key vector, representing the sensitivity of node v to the responses of other nodes at time t; H is the number of attention heads, which capture the dependency patterns of different subspaces through parallel computation. Let be the weight of the functional dependency edge between nodes u and v. The normalized attention weights represent the dependence strength of node u on node v in the r-th attention head, d k Let be the dimension of the key vector. The normalized attention weights W for the i-th attention head r,i Let be the trainable weight matrix for the i-th attention head, used to aggregate information from multiple attention heads.

[0118] After calculating the weights of all functional dependency edges, we filter out functional dependency edges whose weights are greater than or equal to the functional dependency threshold, and delete functional dependency edges whose weights are less than the functional dependency threshold. Only functional dependency edges with weights greater than or equal to the functional dependency threshold are retained, as shown below:

[0119]

[0120] Where, ∈ func This is an adjustable function dependency threshold.

[0121] All nodes corresponding to environmental sensors and nodes corresponding to line sensors are connected by spatiotemporal correlation edges, and the spatiotemporal correlation edges E between nodes are... st It can be represented as:

[0122] E st ={(u,v)|u∈{environmental sensors},v∈{circuit sensors},||p u -p v ||≤d max} (9)

[0124] Where, p u For the location information of node u, p v For the location information of node v, d max This is the distance threshold.

[0125] The weights of each spatiotemporal related edge are calculated as follows:

[0126]

[0127] Where, γ uv (τ) represents the time decay factor for nodes u and v, used to assign reasonable weights to the delay effect; τ∈[t-Δt,t] is the time sliding window; σ is a parameter used to control the time decay rate; β uv Let |p be the spatial decay factor between nodes u and v, used to indicate that the closer the terminal devices are, the stronger their correlation. u -p v || represents the Euclidean distance between nodes u and v, d0 is the distance scale parameter used to determine the spatial correlation decay rate, and is a hyperparameter; ρ uv (τ) represents the temporal covariance between nodes u and v, indicating the correlation between the data of the two nodes. The higher the covariance, the more likely a temporal correlation edge is to be established. τ For the data collected by node u at time τ, v t For the data collected by node v at time t, σ u σ v Let the standard deviation of the data for node u and node v be denoted as . The weight of the spatiotemporal connection edge between node u and node v.

[0128] After calculating the weights of all spatiotemporally related edges, edges with weights greater than or equal to the spatiotemporal association threshold are selected, and edges with weights less than the threshold are deleted. Only edges with weights greater than or equal to the threshold are retained, as shown below:

[0129]

[0130] Where, ∈ st This is an adjustable spatiotemporal correlation threshold.

[0131] After determining all physical connection edges, retained functional dependency edges, and spatiotemporal correlation edges, the topology graph construction module outputs E, which includes all nodes and the physical connection edges between nodes. physical Preserved functional dependency edges Spacetime related edges The multi-relationship topology diagram can comprehensively and accurately describe the complex interaction relationships between various terminal devices in the transmission line, including physical association, functional association, and spatiotemporal association, thereby enabling a comprehensive and accurate perception of the overall state changes of the transmission line.

[0132] In some embodiments, the natural language description generation module is based on a graph neural network encoder-decoder architecture. After aggregating node information through a graph attention mechanism, it uses a recurrent neural network to generate natural language descriptions. However, conventional natural language generation methods applied to transmission lines have the following problems: First, due to the lack of modeling capabilities for the dynamic relationships between terminal devices, the fixed attention weight mechanism cannot adapt to the time-varying coupling relationships of multiple physical field parameters such as conductor icing and tower tilting, thus failing to describe the relevant relationships. Second, the generally generated natural language descriptions only contain vague expressions such as "high load" and "abnormal temperature," failing to accurately reference industry standards or quantify the correlation strength, making it difficult to quickly determine the fault level. Third, the generally generated text content usually describes the status of each terminal device in isolation, failing to explicitly reflect the spatial topological relationships, functional relationships, and spatiotemporal relationships between terminal devices, severely hindering the collaborative diagnosis of complex faults.

[0133] Therefore, to accurately generate natural language descriptions for transmission line scenarios, the natural language description generation module uses a graph attention mechanism based on a multi-relation topology graph to generate the corresponding natural language descriptions. Specifically, generating natural language descriptions of the multi-relation topology graph in the transmission line scenario includes:

[0134] The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph.

[0135] Retrieve prior knowledge related to each node from a pre-defined knowledge graph of the power sector;

[0136] Based on the prior knowledge of each node and the node feature representation of neighboring nodes, the aggregated node features are obtained.

[0137] Generate natural language descriptions based on the features of aggregated nodes.

[0138] In this embodiment, the relevant information of the multi-relationship topology graph is... The input graph attention mechanism's natural language description generation module, where V is the set of node information for all nodes, and the node information is represented as v. i =[x i ,p i ,t i ,d i ], where x i For the data collected at node i, p i For the location information of node i, t i The timestamp for collecting data at node i, d i Encode the device type for node i.

[0139] The natural language description generation module aggregates node features based on knowledge from the power industry, fuses and encodes the node information of each node, and generates a node feature representation that includes topology graph information and industry prior knowledge, represented as follows:

[0140]

[0141] Where GELU is the Gaussian linear unit activation function, W enc and b enc For learnable weights and bias parameters, onehot(d) u ) represents the device type encoding d for node u. u Perform one-hot encoding, x u p u t u These are the data collected by node u, its location information, and the timestamp of the collected data, respectively. Let be the initial node feature representation of node u.

[0142] Based on the data collected from each node and the equipment type codes, from the power sector knowledge graph G... kg The system retrieves prior knowledge related to each node, obtaining equipment parameter thresholds, historical failure cases, industry standard clauses, etc. It then determines the set N(u) of neighboring nodes connected to node u via any type of edge. Based on the prior knowledge of each node and the graph attention network, it aggregates the node features of the neighboring nodes to obtain the aggregated node features, represented as:

[0143]

[0144] α uv =softmax(e uv (17)

[0145]

[0146] Among them, e uv For attention energy, W q and W k W v For learnable weight parameters, d k α is the scaling factor, typically taken as the feature dimension; uv This refers to the attention weight, which is the weight of the influence of neighbor node v on node u. LayerNorm() is the layer normalization operation, which represents the aggregated node features of node u after L layers of iteration, containing topological relationships and domain knowledge. Let k be the node feature representations of nodes v and u after training through layer l. v It is the prior knowledge of node v.

[0147] Based on the features of aggregated nodes, natural language description text conforming to power industry standards is generated. First, a context vector is constructed, represented as:

[0148]

[0149] Among them, W ctx Let c be a trainable weight matrix. t It is a context vector.

[0150] Calculate the hierarchical probability, i.e., the word generation probability at each time step t, and select the word with the highest probability to determine the word of the natural language description text at the current time step. The method for calculating the word generation probability is as follows:

[0151]

[0152] p LM (w t |h t-1 ,c t = softmax(W0·FFN([h t-1 ||c t ])) (twenty two)

[0153] p(w t )=β·p template (w t |T,R)+(1-β)·p LM (w t |h t-1 ,c t ) (twenty three)

[0154] Where β is the dynamic equilibrium coefficient, W β p is a trainable parameter matrix; template p is the template probability; LM For language model probability; T is the power industry basic template selected according to node type, such as "<device ID> detected <parameter> = <value>, which meets the <standard> level <clause> warning"; R is the physical rule set; p(w t Let p(w) be the probability distribution of the words in the text. t The calculation process must conform to the basic template T for electrical engineering and the physical rule R; h t-1 For the state at time step t-1, w t Let W0 be the candidate words generated at time step t, W0 be the trainable weight matrix used to map the output of the feedforward neural network to the vocabulary space, and FFN() represent the feedforward neural network processing.

[0155] The word probability p(w) is calculated through multiple iterations. tEach time, the word with the highest probability is selected, and the content of the natural language is determined word by word until the terminator is output, thus generating a complete natural language description.

[0156] In one specific embodiment, the relevant data of the abnormal state results detected by multiple anomaly detection models include:

[0157] Terminal device T-3621 is a tension sensor. It collected a vertical load of 12kN, which exceeds the normal threshold of 10kN. The attribute information of this terminal device includes: installed on the insulator string of tower #36, belonging to the network segment A, with spatial coordinates of [116.321, 39.987, 256.4], and sampling frequency of 1Hz.

[0158] Terminal device V-3605 is a visible light image acquisition unit. It observed uneven icing on the insulator skirts, with an icing area of ​​35%, a maximum thickness of 8 mm, and an icing growth rate of 0.8 mm / h. The terminal device's attribute information includes: deployed on the side of tower #36, belonging to line A, with spatial coordinates [116.321, 39.987, 248.1], an image resolution of 4000x3000, and a frame rate of 30fps.

[0159] Terminal device W-3608 is an environmental sensor (also known as a micro-weather station). It records the ambient temperature as -5℃, humidity as 92%, wind speed as 5m / s, and wind direction as northeast. The attribute information of this terminal device includes: deployed 8 meters above the top of tower #36, belonging to the network segment of line A, with spatial coordinates of [116.321, 39.987, 264.4], and a sampling frequency of 1Hz.

[0160] Terminal device A-3507 is a tilt sensor. It collects the tilt angle of the conductor, which is 2.3° higher than the reference value, with a change rate of 0.1° / min. The attribute information of this terminal device includes: adjacent to tower #35, belonging to the network segment A, with spatial coordinates [116.325, 39.982, 250.0], and a horizontal distance of 180 meters from tower #36.

[0161] Terminal device D-3610 is a visible light image acquisition unit. It captured images showing that the icing area on the insulator string of tower #36 has expanded, with the icing area increasing from 30% to 45%, and the edges are jagged. The conductor sag has also increased significantly. The terminal device's attribute information includes: UAV number DJI-004, network segment A, spatial coordinates [116.320, 39.986, 280.0], and flight altitude of 80 meters.

[0162] The terminal device IR-3612 is an infrared thermal imager. It detected an abnormal local temperature in the insulator string, with a minimum temperature of -8℃, a maximum temperature of -2℃, and a temperature difference exceeding the threshold of 5℃. The attribute information of the terminal device includes: UAV number DJI-004, network segment A, and spatial coordinates [116.320, 39.986, 280.0].

[0163] The topology graph construction module constructs a multi-relationship topology graph based on the abnormal state results and the attribute information of the terminal devices, such as... Figure 6 As shown, purple lines represent physical connection edges, red lines represent spatiotemporal connection edges, and blue lines represent functional dependency edges. Each edge is marked with its corresponding weight value. The set functional dependency threshold is 0.75, and the spatiotemporal connection threshold is 0.75.

[0164] Based on the generated multi-relationship topology graph, construct relevant information for the graph. The Natural Language Description (NLP) generation module generates corresponding NLP descriptions based on relevant information from a multi-relationship topology graph. Specific content includes: Monitoring data shows that the insulator of tower #36 exhibits abnormal icing. The tension sensor T-3621 located at the top of the tower detected a vertical load of 12kN, exceeding the threshold of 10kN. This node is physically connected to the visible light camera V-3605 installed on the same tower, which observed that the insulator's icing area reached 45%, with a thickness of 8mm and an increase in icing rate of 0.8mm / h. The infrared node IR-3612 of the UAV D-3610 and the ground camera V-3605 form a functionally complementary relationship, detecting a local temperature difference of 6℃ (-8℃ to -2℃) in the insulator string. The environmental data (temperature -5℃, humidity 92%, wind speed 5m / s northeasterly) recorded by the micro-weather station W-3608 at the top of the tower influences the tension sensor readings through spatiotemporal correlation edges, and simultaneously forms a mechanical response relationship with the tilt angle change of the adjacent tower A-3507 (currently +2.3°, increasing by 0.1° / min). The data collected from each node collectively indicate that the current operating conditions meet the Level II icing warning standard in DL / T 1242-2023. It is evident that the natural language description generated based on relevant information from the multi-relationship topology diagram not only reflects the time-varying coupling relationships of multi-physics parameters and the multiple correlations between terminal devices, but also provides a complete, detailed, and accurate description, facilitating rapid identification of potential single-type or compound-type faults and improving the maintenance efficiency of transmission lines.

[0165] S104: Using a pre-defined large-scale power industry model based on natural language description, determine the fault diagnosis results of transmission lines.

[0166] like Figure 7As shown in this embodiment, the large-scale power industry model is a specialized model based on a general-purpose large-scale language model. It utilizes a massive power industry knowledge graph and industry standards for deep knowledge distillation and alignment, enabling high-level reasoning and comprehensive decision-making regarding transmission line status, equipment operating characteristics, and complex power grid topologies. The large-scale power industry model performs multi-physics coupled reasoning and state decision-making based on the input natural language description. Specifically, it conducts comprehensive analysis of the data described in the text, environmental factors, and topological structure to perform in-depth fault mode identification and fault cause tracing, outputting fault diagnosis results for the transmission line status. This embodiment does not provide a detailed explanation of the structure and training process of the large-scale power industry model.

[0167] like Figure 8 As shown, the method of this application is compared with the methods that use only small models and only multimodal large models for transmission line detection. For several fault types, the detection accuracy of the method of this application is significantly better than the other two methods, which can effectively improve the accuracy of transmission line condition monitoring.

[0168] The transmission line status monitoring method provided in this application utilizes multiple anomaly detection models to perform real-time detection of corresponding modal data, initially determining the abnormal status results. Based on the abnormal status results and the attribute information of terminal devices, a multimodal semantic model is used to comprehensively analyze the dynamic correlation relationships of multimodal data in the transmission line scenario, constructing a multi-relationship topology diagram among terminal devices. Based on this topology diagram, a corresponding natural language description is generated. In the process of constructing the topology diagram and generating natural language, the characteristics of the transmission line monitoring scenario are effectively combined, fully integrating the multiple correlation relationships between data to achieve topological semanticization of the transmission line scenario. Then, a large-scale power industry model is used to determine the fault diagnosis results of the power industry scenario based on the natural language description. Through the collaborative processing of anomaly detection models, multimodal semantic models, and large-scale power industry models, the topological semanticization of the scenario is achieved for potential fault areas of the transmission line. The line status is detected through the semantic content of the scenario, ensuring real-time response capability and enabling multi-physics coupling analysis of complex scenarios through high-order reasoning. Finally, a complete diagnostic report containing anomaly judgment, cause analysis, and handling suggestions is generated, improving the accuracy and completeness of line monitoring results and increasing the operation and maintenance efficiency of transmission lines.

[0169] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0170] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] like Figure 9 As shown in the figure, this application provides a power transmission line condition monitoring device, including:

[0172] The acquisition module is used to acquire multimodal data collected by the terminal device;

[0173] An anomaly detection module is used to perform anomaly detection on the multimodal data using multiple preset anomaly detection models to obtain anomaly state results;

[0174] The semantic generation module is used to construct a multi-relationship topology diagram based on abnormal state results and terminal device attribute information using a preset multimodal semantic model, and generate a natural language description of the multi-relationship topology diagram in the transmission line scenario.

[0175] The fault diagnosis module is used to determine the fault diagnosis results of transmission lines based on natural language descriptions using a pre-set large-scale power industry model.

[0176] In some embodiments, the attribute information includes location information, and the terminal device includes environmental sensors and line sensors;

[0177] The semantic generation module is used to determine the physical connection relationship between terminal devices based on the location information of each terminal device; determine the functional dependency relationship between terminal devices based on the correlation of data changes of each terminal device; determine the spatiotemporal correlation relationship between environmental sensors and line sensors based on the location information and correlation of data changes of environmental sensors and line sensors; and construct a multi-relationship topology diagram including physical connection relationship, functional dependency relationship and spatiotemporal correlation relationship with terminal devices as nodes.

[0178] In some embodiments, the semantic generation module is used to generate physical connection edges between nodes based on physical connection relationships; connect any two nodes that do not have physical connection edges through functional dependency edges, and calculate the weight of each functional dependency edge, retaining functional dependency edges with weights greater than or equal to a preset functional dependency threshold; generate spatiotemporal association edges between nodes corresponding to environmental sensors and line sensors based on the spatiotemporal association relationships, and calculate the weight of each spatiotemporal association edge, retaining spatiotemporal association edges with weights greater than or equal to a preset spatiotemporal association threshold; and construct a multi-relationship topology graph based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal association edges.

[0179] In some embodiments, the weights of each functional dependency edge are calculated as follows:

[0180]

[0181] in, Let be the weight of the functional dependency edge between nodes u and v. and Let be the projection matrix of the r-th trainable attention head. For query vector, For key vectors, Data collected at time t for nodes u and v are shown below, respectively, and H represents the number of attention heads. For the normalized attention weights, d k Let be the dimension of the key vector. The normalized attention weights W for the i-th attention head r,i Let be the trainable weight matrix for the i-th attention head.

[0182] In some embodiments, the weights of each spatiotemporally related edge are calculated using the following method:

[0183]

[0184] in, γ represents the weight of the spatiotemporal association edge between node u and node v; uv (τ) represents the time decay factor for nodes u and v, τ∈[t-Δt,t] is the time sliding window, and σ is a parameter used to control the time decay rate; β uv p is the spatial decay factor for nodes u and v. u p v These represent the location information of nodes u and v, respectively, and d0 is the distance scale parameter; ρ uv (τ) represents the temporal covariance between node u and node v, u τ For the data collected by node u at time τ, v t For the data collected by node v at time t, σ uσ v Let be the standard deviation of the data for node u and node v.

[0185] In some embodiments, the semantic generation module is used to encode the node information of each node in the multi-relation topology graph to obtain the node feature representation of each node; retrieve prior knowledge related to each node from a preset power field knowledge graph; obtain aggregated node features based on the prior knowledge of each node and the node feature representation of neighboring nodes; wherein, neighboring nodes are nodes connected to the node through any kind of edge; and generate natural language descriptions based on the aggregated node features.

[0186] In some embodiments, node information includes data collected by the node, timestamps of the collected data, node location information, and node device type;

[0187] The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph. The method is as follows:

[0188]

[0189] Where GELU is the Gaussian linear unit activation function, W enc and b enc For learnable weights and bias parameters, onehot(d) u ) represents the device type encoding d for node u. u Perform one-hot encoding, x u p u t u These are, respectively, the data collected by node u, the location information of node u, and the timestamp of the data collected by node u. Let be the initial node feature representation of node u.

[0190] In some embodiments, the multimodal data includes sensor data, visible light image data, infrared image data, and lidar data; the multiple anomaly detection models include sensor detection models, visible light image detection models, infrared image detection models, and lidar detection models.

[0191] The anomaly detection module is used to perform anomaly detection on the sensor data using a sensor detection model to obtain anomaly status results for the sensor data; to perform anomaly detection on the visible light image data using a visible light image detection model to obtain anomaly status results for the visible light image; to perform anomaly detection on the infrared image data using an infrared image detection model to obtain anomaly status results for the infrared image; and to perform anomaly detection on the lidar data using a lidar detection model to obtain anomaly status results for the lidar data.

[0192] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware.

[0193] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0194] like Figure 10 As shown in the illustration, this application also provides a transmission line condition monitoring system for implementing a transmission line condition monitoring method. The system includes:

[0195] Terminal devices deployed at the edge; and multiple anomaly detection models, multimodal semantic models, and large-scale power industry models deployed in the cloud.

[0196] In this embodiment, the transmission line condition monitoring system includes multiple terminal devices deployed at the edge and various models deployed on a cloud server. A data acquisition system composed of multiple terminal devices collects multimodal data, which is preprocessed by a multimodal data processing module before being sent to the intelligent model hub and visualization platform on the cloud server. The preprocessing of the multimodal data includes noise removal, outlier filtering, standardization, and data screening, resulting in multimodal data suitable for model processing. The intelligent model hub includes a multimodal small model group composed of multiple anomaly detection models, a multimodal semantic large model, and a power industry large model. Multiple anomaly detection models perform real-time anomaly detection on the preprocessed multimodal data. If an anomaly is detected, the multimodal semantic large model is activated to process the detected anomaly state results and the attribute information of the terminal devices, constructing a multi-relationship topology diagram and generating corresponding natural language descriptions. The power industry large model then uses these natural language descriptions to generate detailed decision results, such as transmission line fault diagnosis results. The visualization platform is a comprehensive display platform for the status of power transmission lines built on WebGL and a 3D geographic information engine. It can display the 3D model of the power transmission line in real time, multimodal data collected in real time from the edge side, detailed decision results generated by the cloud server, fault alarm information, historical data backtracking and other information, so as to facilitate quick and intuitive viewing of status changes and improve the efficiency and reliability of operation and maintenance.

[0197] like Figure 11As shown, in some models, the transmission line condition monitoring system mainly includes five functional modules: data management, model training, anomaly detection, visualization interaction, and system management. The data management module is primarily responsible for storing data, parameters, logs, and abnormal events. The model training module is mainly responsible for training, evaluating, and managing the versions of various models. The anomaly detection module mainly uses various models to capture anomalies from multimodal data and make fault decisions. The visualization interaction module mainly realizes visualization display, monitoring, and operation. The system management function is mainly used for setting permissions, managing logs, and configuring the system.

[0198] Figure 12 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0199] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0200] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0201] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0202] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0203] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0204] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0205] The electronic devices described above are used to implement the corresponding methods in the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0206] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0207] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0208] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this disclosure, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0209] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0210] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this disclosure.

Claims

1. A method for monitoring the condition of a power transmission line, characterized in that, include: Acquire multimodal data collected by terminal devices; Anomalies are detected based on the multimodal data using multiple preset anomaly detection models to obtain anomaly state results; Based on the abnormal state results and the attribute information of the terminal devices, a multi-relationship topology diagram is constructed using a preset multimodal semantic model, and a natural language description of the multi-relationship topology diagram in the transmission line scenario is generated; wherein, the attribute information includes location information, and the terminal devices include environmental sensors and line sensors; the construction of the multi-relationship topology diagram includes: The physical connection relationships between terminal devices are determined based on the location information of each terminal device. Based on the correlation of data changes among various terminal devices, determine the functional dependencies between terminal devices; Based on the location information and data change correlation of the environmental sensors and line sensors, the spatiotemporal correlation between the environmental sensors and line sensors is determined; Based on the physical connection relationship, physical connection edges are generated between each node; Connect any two nodes that do not have a physical connection edge with a functional dependency edge, calculate the weight of each functional dependency edge, and retain functional dependency edges whose weight is greater than or equal to a preset functional dependency threshold. Based on the spatiotemporal correlation, spatiotemporal correlation edges are generated between the nodes corresponding to the environmental sensors and the line sensors, and the weight of each spatiotemporal correlation edge is calculated. Spatiotemporal correlation edges with weights greater than or equal to a preset spatiotemporal correlation threshold are retained. Construct a multi-relation topology graph based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal association edges; The method for calculating the weights of each spatiotemporal related edge is as follows: （10） （11） （12） （13） in, For nodes With nodes The weights of the spatiotemporal connections between them; For nodes With nodes Time decay factor, This belongs to the time sliding window At that moment, These are parameters used to control the rate of time decay; For nodes With nodes Space decay factor, They are nodes , Location information, For distance scale parameters; For nodes With nodes The temporal covariance, For nodes At any moment The collected data, For nodes Data collected at time t For nodes With nodes The standard deviation of the data; Based on the natural language description, the fault diagnosis results of the transmission line are determined using a pre-defined large-scale power industry model.

2. The method according to claim 1, characterized in that, The method for calculating the weights of each functional dependency edge is as follows: （4） （5） （6） （7） in, For nodes and The weights of the functionally dependent edges between them. and For trainable first r The projection matrix of each attention head. For query vector, For key vectors, , They are nodes , In time The collected data, For the number of attention heads, For normalized attention weights, Let be the dimension of the key vector. For the first i Normalized attention weights for each attention head For the first i A trainable weight matrix for each attention head.

3. The method according to claim 1, characterized in that, Generating a natural language description of the multi-relationship topology diagram in a transmission line scenario includes: The node information of each node in the multi-relation topology graph is encoded to obtain the node feature representation of each node; Retrieve prior knowledge related to each node from a pre-defined knowledge graph of the power sector; Based on the prior knowledge of each node and the node feature representation of neighboring nodes, aggregated node features are obtained; wherein, the neighboring node is a node connected to the node through any kind of edge. A natural language description is generated based on the features of the aggregated nodes.

4. The method according to claim 3, characterized in that, The node information includes the data collected by the node, the timestamp of the collected data, the node's location information, and the node's device type; The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph. The method is as follows: （15） in, The activation function is a Gaussian linear unit. and For learnable weights and bias parameters, Indicates a node Device type coding Perform one-hot encoding. , , They are nodes Collected data, nodes Location information, nodes The timestamp of the collected data For nodes The initial node feature representation.

5. The method according to claim 1, characterized in that, The multimodal data includes sensor data, visible light image data, infrared image data, and lidar data; the multiple anomaly detection models include sensor detection models, visible light image detection models, infrared image detection models, and lidar detection models. The step of using multiple preset anomaly detection models to perform anomaly detection on the multimodal data to obtain anomaly state results includes: The sensor detection model is used to detect anomalies in the sensor data to obtain the sensor data anomaly status results. The visible light image detection model is used to detect anomalies in the visible light image data to obtain the results of the visible light image anomaly status. The infrared image detection model is used to detect anomalies in the infrared image data to obtain the results of abnormal infrared image states. The lidar detection model is used to detect anomalies in the lidar data, and the abnormal state results of the lidar data are obtained.

6. A power transmission line condition monitoring device, characterized in that, include: The acquisition module is used to acquire multimodal data collected by the terminal device; An anomaly detection module is used to perform anomaly detection based on the multimodal data using multiple preset anomaly detection models to obtain anomaly state results; A semantic generation module is used to construct a multi-relationship topology diagram based on the abnormal state results and the attribute information of the terminal devices using a preset multimodal semantic model, and to generate a natural language description of the multi-relationship topology diagram in a transmission line scenario; wherein, the attribute information includes location information, and the terminal devices include environmental sensors and line sensors; constructing the multi-relationship topology diagram includes: determining the physical connection relationship between terminal devices based on the location information of each terminal device; determining the functional dependency relationship between terminal devices based on the data change correlation of each terminal device; and determining the functional dependency relationship between the environmental sensors and line sensors based on the location information and data change correlation of the environmental sensors and line sensors. The spatiotemporal relationships between environmental sensors and line sensors are described; physical connection edges are generated between nodes based on the physical connection relationships; any two nodes without physical connection edges are connected by functional dependency edges, and the weight of each functional dependency edge is calculated, retaining functional dependency edges with weights greater than or equal to a preset functional dependency threshold; spatiotemporal connection edges are generated between the nodes corresponding to the environmental sensors and line sensors based on the spatiotemporal relationships, and the weight of each spatiotemporal connection edge is calculated, retaining spatiotemporal connection edges with weights greater than or equal to a preset spatiotemporal connection threshold; a multi-relationship topology graph is constructed based on all nodes, all physical connection edges, retained functional dependency edges, and spatiotemporal connection edges. The method for calculating the weights of each spatiotemporal related edge is as follows: （10） （11） （12） （13） in, For nodes With nodes The weights of the spatiotemporal connections between them; For nodes With nodes Time decay factor, This belongs to the time sliding window Inside the moment, These are parameters used to control the rate of time decay; For nodes With nodes Space decay factor, They are nodes , Location information, For distance scale parameters; For nodes With nodes The temporal covariance, For nodes At any moment The collected data, For nodes Data collected at time t For nodes With nodes The standard deviation of the data; The fault diagnosis module is used to determine the fault diagnosis results of transmission lines based on the natural language description using a preset large power industry model.

7. The apparatus according to claim 6, characterized in that, The method for calculating the weights of each functional dependency edge is as follows: （4） （5） （6） （7） in, For nodes and The weights of the functionally dependent edges between them. and For trainable first r The projection matrix of each attention head. For query vector, For key vectors, , They are nodes , In time The collected data, For the number of attention heads, For normalized attention weights, Let be the dimension of the key vector. For the first i Normalized attention weights for each attention head For the first i A trainable weight matrix for each attention head.

8. The apparatus according to claim 7, characterized in that, The semantic generation module is used to encode the node information of each node in the multi-relation topology graph to obtain the node feature representation of each node. Prior knowledge related to each node is retrieved from a pre-defined knowledge graph of the power sector; aggregated node features are obtained based on the prior knowledge of each node and the node feature representations of neighboring nodes; wherein, the neighboring nodes are nodes connected to the node through any kind of edge; and natural language descriptions are generated based on the aggregated node features.

9. The apparatus according to claim 8, characterized in that, The node information includes the data collected by the node, the timestamp of the collected data, the node's location information, and the node's device type; The node feature representation of each node is obtained by encoding the node information of each node in the multi-relation topology graph. The method is as follows: （15） in, The activation function is a Gaussian linear unit. and For learnable weights and bias parameters, Indicates a node Device type coding Perform one-hot encoding. , , They are nodes Collected data, nodes Location information, nodes The timestamp of the collected data For nodes The initial node feature representation.

10. The apparatus according to claim 6, characterized in that, The multimodal data includes sensor data, visible light image data, infrared image data, and lidar data; the multiple anomaly detection models include sensor detection models, visible light image detection models, infrared image detection models, and lidar detection models. The anomaly detection module is used to perform anomaly detection on the sensor data using the sensor detection model, and obtain the sensor data anomaly status result; The visible light image detection model is used to detect anomalies in the visible light image data to obtain the results of the visible light image anomaly status. The infrared image detection model is used to detect anomalies in the infrared image data to obtain anomaly status results; the lidar detection model is used to detect anomalies in the lidar data to obtain anomaly status results.

11. A transmission line condition monitoring system, characterized in that, The system for implementing the method as described in any one of claims 1 to 5 comprises: Terminal devices deployed at the edge, and multiple anomaly detection models, multimodal semantic models, and large-scale power industry models deployed in the cloud.

12. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing the computer to perform the method described in any one of claims 1 to 5.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

Images