Multifunctional Online Video Monitoring Method and Device for Transmission Lines

Abstract

This application provides a multifunctional online video monitoring method and device for transmission lines, belonging to the field of power transmission monitoring technology. The method includes: determining the scene type based on meteorological monitoring data of the transmission line; determining a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene types include icing scenes, galloping scenes, and ordinary scenes; extracting image features from video frames of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; extracting features from sensor monitoring data based on the feature weight matrix of sensor monitoring data to obtain a sensor monitoring feature set; fusing the sensor monitoring feature set and the video monitoring feature set using multimodal features to obtain a fused image feature set; and determining the status information of the transmission line based on the fused image feature set. This application can improve the monitoring accuracy of abnormal conditions of transmission lines under complex meteorological scenarios.

Description

Technical Field

[0001] This application belongs to the field of power transmission monitoring technology, and more specifically, relates to a multifunctional online video monitoring method and device for power transmission lines. Background Technology

[0002] The safe and stable operation of transmission lines is crucial to the power system, and real-time monitoring of line status is a key means of preventing faults. Traditional transmission line monitoring methods mainly rely on fixed video surveillance systems to collect data or images for status analysis. However, the environment in which transmission lines operate is complex and variable, and they are susceptible to complex weather conditions. Existing video monitoring methods have low accuracy in identifying line fault risks under complex weather scenarios. Summary of the Invention

[0003] The purpose of this application is to provide a multifunctional online video monitoring method and device for transmission lines, so as to improve the accuracy of monitoring abnormal conditions of transmission lines under complex meteorological scenarios.

[0004] A first aspect of this application provides a multifunctional online video monitoring method for power transmission lines, comprising:

[0005] Scene types are determined based on meteorological monitoring data of power transmission lines, and video feature extraction strategies and feature weight matrices of sensor monitoring data are determined based on the scene types; the scene types include icing scenes, dancing scenes, and ordinary scenes;

[0006] Based on the video feature extraction strategy, image features are extracted from the video frames of the transmission line to obtain a video monitoring feature set; based on the feature weight matrix of the sensor monitoring data, features are extracted from the sensor monitoring data to obtain a sensor monitoring feature set.

[0007] The sensor monitoring feature set and the video monitoring feature set are fused using multimodal features to obtain a fused image feature set; the status information of the transmission line is determined based on the fused image feature set.

[0008] A second aspect of this application provides a multifunctional online video monitoring device for power transmission lines, comprising:

[0009] The scene recognition module is used to determine the scene type based on meteorological monitoring data of power transmission lines, and to determine the video feature extraction strategy and the feature weight matrix of sensor monitoring data based on the scene type; the scene types include icing scene, dancing scene and ordinary scene;

[0010] The feature extraction module is used to extract image features from the video frames of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; and to extract features from the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set.

[0011] The status analysis module is used to perform multimodal feature fusion of the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and to determine the status information of the transmission line based on the fused image feature set.

[0012] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described multifunctional online video monitoring method for transmission lines.

[0013] In a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described multifunctional online video monitoring method for transmission lines.

[0014] The beneficial effects of the multifunctional online video monitoring method and device for transmission lines provided in this application are as follows: On the one hand, this application embodiment determines the scene type through meteorological monitoring data and formulates targeted video feature extraction strategies and sensor feature weight matrices, thereby realizing adaptive monitoring of complex meteorological scenes and solving the problem of low recognition accuracy of traditional methods under adverse conditions.

[0015] On the other hand, the embodiments of this application employ multimodal fusion technology to combine differentiated extracted video image features with sensor data features, thereby overcoming the limitations of single video monitoring in complex environments. Through dynamic weight allocation, the extraction of key features is enhanced, significantly improving the comprehensiveness and accuracy of abnormal state identification.

[0016] In summary, the embodiments of this application, through the collaborative mechanism of scene perception and feature fusion, can accurately capture subtle anomalies of transmission lines under different weather conditions, providing a more reliable real-time monitoring method for the safe operation of the power system, effectively reducing the risk of faults, and improving the intelligence level of transmission line status assessment. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a multifunctional online video monitoring method for transmission lines provided in an embodiment of this application;

[0019] Figure 2 A structural block diagram of a multifunctional online video monitoring device for power transmission lines provided in an embodiment of this application;

[0020] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0023] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a multifunctional online video monitoring method for transmission lines provided in an embodiment of this application. The method can be executed by an electronic device, and specifically, the method may include steps S101 to S103.

[0024] S101: Determine the scene type based on meteorological monitoring data of power transmission lines, and determine the video feature extraction strategy and feature weight matrix of sensor monitoring data based on the scene type; the scene types include icing scene, dancing scene and ordinary scene.

[0025] In this embodiment, meteorological monitoring data for transmission lines refers to environmental meteorological parameters collected in real time by sensors deployed around the transmission lines, used to determine the type of impact of current meteorological conditions on the transmission lines. For example, meteorological monitoring data can be used to determine whether the environment is low temperature, high humidity, or strong wind. Low temperature and high humidity are key conditions for icing formation, while strong wind can cause conductor galloping or accelerate the icing process.

[0026] In this embodiment, the icing scenario refers to the formation of ice layers on the surfaces of conductors and insulators under low temperature and high humidity conditions, threatening the safety of the line load. The galloping scenario refers to the low-frequency, high-amplitude vibration of conductors under strong winds, which can easily lead to hardware damage or phase-to-phase short circuits. The normal scenario refers to stable weather conditions with no significant risk of icing or galloping, allowing for the monitoring of defects in conventional components.

[0027] In this embodiment, the video feature extraction strategy may include differentiated video feature extraction methods corresponding to icy scenes, dancing scenes, and normal scenes, respectively. The feature weight matrix is ​​a matrix formed by assigning weights to sensor data using an attention mechanism. For example, the feature weight matrix may include weight coefficients corresponding to the data monitored by vibration sensors, tension sensors, and tilt sensors, respectively.

[0028] In this embodiment, due to the complex environment of transmission lines, the risks posed by different weather conditions vary. Under different scenarios, the risk triggers and key monitoring objects of transmission lines differ, resulting in significant differences in the correlation between sensor data and line status. By using scenario-adaptive weight allocation, sensor features strongly correlated with the current scenario are dynamically selected, reducing redundant data interference, thereby improving the accuracy of transmission line status assessment after multimodal fusion.

[0029] This embodiment simplifies complex problems by classifying meteorological data into different scenario types and customizing monitoring strategies for each scenario, thus achieving a rational allocation of computing resources. For example, icing scenarios can focus on monitoring icing conditions, galloping scenarios can focus on monitoring conductor dynamics, and ordinary scenarios can focus on monitoring component defects. The feature weight matrix can adjust the importance of data according to the scenario, avoiding interference from irrelevant data, improving monitoring accuracy while ensuring the rational use of computing resources.

[0030] For example, in this embodiment, temperature and humidity sensors and anemometers can be installed along the transmission line to acquire meteorological data; tension sensors, tower tilt angle sensors, and vibration sensors can be deployed to collect line operation data; and high-definition cameras can be installed to collect video data at a fixed frame rate. This embodiment determines the scene type through logical judgment based on the established criteria for temperature, humidity, and wind speed.

[0031] Based on the identified scene type, this embodiment can retrieve the corresponding video feature extraction strategy from the strategy library. For example, in an icy scene, the region segmentation and texture analysis strategy can be enabled. At the same time, this embodiment uses an attention mechanism to assign weights to sensor data according to scene type and generate a feature weight matrix. For example, in a dancing scene, the weight of the conductor vibration signal can be increased.

[0032] S102: Based on the video feature extraction strategy, image features are extracted from the video frames of the transmission line to obtain the video monitoring feature set; based on the feature weight matrix of the sensor monitoring data, features are extracted from the sensor monitoring data to obtain the sensor monitoring feature set.

[0033] In this embodiment, a video frame refers to a continuous sequence of images captured by a camera on the transmission line. Image features refer to status monitoring information extracted from the video frames, such as the percentage of icing area in an icing scenario and the amplitude of conductor swaying in a galloping scenario. Sensor monitoring data may include data such as conductor tension, tower tilt angle, and conductor vibration signals, including parameters such as numerical values ​​and sampling frequency.

[0034] Considering that potential faults in transmission lines manifest differently in video images and sensor data, a single data source cannot comprehensively reflect the line's condition. This embodiment employs a differentiated video feature extraction strategy and feature weight matrix to selectively extract key information from different data sources. By leveraging the complementary advantages of multi-source data, the accuracy of monitoring abnormal line conditions is improved.

[0035] For example, in this embodiment, it is first confirmed that an appropriate video feature extraction strategy and a feature weight matrix of sensor monitoring data have been selected according to the scene type; at the same time, video frame data of the transmission line, as well as sensor monitoring data such as conductor tension, tower tilt angle, and conductor vibration signal are acquired.

[0036] In terms of video feature extraction, this embodiment processes video frames according to a selected video feature extraction strategy. For example, this embodiment can first segment the power transmission line video, decomposing it into multiple single-frame images (i.e., video frames), and then perform preprocessing operations such as grayscale conversion, filtering and noise reduction, and size normalization to improve image quality. This embodiment can use classic algorithms such as SIFT and SURF to extract local key points and descriptors from the image, and use HOG and LBP algorithms to obtain global contour and texture features of the image; this embodiment can also utilize a pre-trained convolutional neural network model to automatically extract the depth features of the image through forward propagation. Finally, this embodiment can concatenate and fuse the features extracted by different methods to form a complete video monitoring feature set.

[0037] For sensor monitoring data, this embodiment first performs outlier removal, missing value imputation, and standardization on the raw data. Then, it uses algorithms such as principal component analysis and mutual information to calculate the importance of each sensor data feature and construct a feature weight matrix. This embodiment can then weight the data according to the constructed feature weight matrix to highlight key information, and simultaneously use the sliding window technique to extract time-domain features. The weighted features are then combined to obtain the sensor monitoring feature set.

[0038] After obtaining the processed feature set, this embodiment can perform integrity and accuracy checks on the generated video monitoring feature set and sensor monitoring feature set to ensure that no important features are omitted or data is abnormal, thus providing reliable data for subsequent multimodal fusion and line status assessment.

[0039] S103: Perform multimodal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; determine the status information of the transmission line based on the fused image feature set.

[0040] In this embodiment, the sensor monitoring feature set and the video monitoring feature set are fused using multimodal features to obtain a fused image feature set, specifically including:

[0041] Linear interpolation is performed on the sensor monitoring feature set to generate a sensor data interpolation sequence aligned with the timestamps of the video monitoring feature set;

[0042] The video monitoring feature set is dimensionality reduced to obtain a video monitoring feature set with the same dimension as the sensor monitoring feature set.

[0043] The features in the video monitoring feature set are matched with the target feature template. Based on the matching results, multiple target locations are determined. The features in the sensor data interpolation sequence are then associated and aligned with the multiple target locations.

[0044] In this embodiment, the sensor monitoring feature set refers to the set of feature data collected and processed from various sensors, which may include parameters such as conductor tension, tower tilt angle, and conductor vibration signal. The video monitoring feature set refers to the set of features extracted through video analysis of transmission lines, which may include visual features such as conductor appearance, insulator condition, and surrounding environment, and exist in the form of image feature vectors.

[0045] Multimodal feature fusion refers to the organic combination of different modal features from sensor monitoring and video monitoring to form a more representative fused feature set. Timestamp alignment means that the sensor monitoring feature set and the video monitoring feature set correspond in time, which facilitates fusion analysis.

[0046] Target feature templates are pre-established sample sets containing typical features of various targets, used for matching with video monitoring feature sets. For example, they may include conductor feature templates, tower feature templates, etc. The purpose of establishing target feature templates is to more accurately and efficiently identify different target objects from the video monitoring feature set. In practice, a large amount of image data of various targets in transmission lines under different environments and angles can be collected. Stable and representative features can be extracted using image processing and machine learning techniques to construct a target feature template library. During real-time monitoring, this can be used to quickly match the video monitoring feature set with the features in the template library, achieving accurate identification and positioning of various parts of the transmission line.

[0047] In this embodiment, different modal data differ in time and dimension, and direct fusion would affect accuracy and efficiency. This embodiment uses linear interpolation to synchronize sensor data and video data in time, and dimensionality reduction processing to match their dimensions, placing them in the same spatial dimension. Feature template matching and association alignment can achieve accurate correspondence of different modal features, comprehensively reflecting the status of the transmission line.

[0048] For example, this embodiment can obtain the timestamp information of the sensor monitoring feature set and the video monitoring feature set to determine the differences between them in the time series. This embodiment selects a suitable linear interpolation method based on the acquisition frequency and time interval of the sensor data. For example, if the data changes relatively smoothly, simple linear interpolation can be used; if the data fluctuates significantly, a higher-order method such as Lagrange interpolation is used. This embodiment uses the timestamp of the video monitoring feature set as a reference, performs interpolation calculations on the sensor monitoring feature set according to the selected interpolation method, fills data gaps or adjusts data time points, and generates an interpolation sequence aligned with the timestamp of the video monitoring feature set.

[0049] For the video monitoring feature set, this embodiment can select a suitable dimensionality reduction algorithm, such as principal component analysis (PCA) or linear discriminant analysis (LDA). If PCA is used, the covariance matrix of the video monitoring feature set can be calculated to obtain eigenvalues ​​and eigenvectors. Principal components are then selected based on their cumulative contribution rate, achieving dimensionality reduction. If LDA is used, the class information of the data can be considered to find projection directions that maximize inter-class distance and minimize intra-class distance, thereby reducing dimensionality. This embodiment, through dimensionality reduction, makes the dimensionality of the video monitoring feature set consistent with that of the sensor monitoring feature set, reducing computational complexity while retaining key feature information for subsequent fusion.

[0050] This embodiment matches the dimensionality-reduced video monitoring feature set with a pre-constructed target feature template. This embodiment can use distance-based methods, such as Euclidean distance or cosine similarity, to calculate the similarity between video features and template features. Based on the matching results, this embodiment determines the positions of multiple targets in the video. For example, it identifies the position coordinates of targets such as conductors and insulators in the video frame. Based on the obtained target positions, this embodiment associates and aligns the features in the sensor monitoring feature set with the corresponding target positions, completing multimodal feature fusion to obtain a fused image feature set, providing comprehensive data support for determining the status of transmission lines. For example, it associates and aligns the conductor vibration features in the sensor monitoring feature set with the corresponding target conductor positions.

[0051] For example, in practical applications, determining the status information of transmission lines based on fused image feature sets can specifically include:

[0052] The fused image feature set is input into a pre-trained state recognition model. This model can employ a deep learning approach suitable for multimodal data, such as a multimodal convolutional neural network or a Transformer architecture. The state recognition model judges the line status based on the learned patterns. For example, if the fused image feature set shows abnormal conductor vibration frequency and the video captures the conductor contacting a foreign object, the model quickly identifies it as external force damage and issues a warning. If the model determines that the line is abnormal, it can accurately locate the fault point based on the location information in the fused image feature set and output information such as the fault type and location.

[0053] As can be seen from the above, this embodiment divides scene types by meteorological data, customizes video feature extraction strategies and sensor data weights for different scenes, and integrates multimodal features to comprehensively capture line status information, thereby improving the monitoring accuracy of abnormal conditions such as icing and galloping as well as component defects, and enabling timely detection of potential risks.

[0054] This embodiment also enhances the system's adaptability and reliability. By segmenting scenarios and implementing differentiated monitoring, it can flexibly adjust monitoring strategies based on real-time weather conditions, allocate computing resources for different scenarios, avoid interference from irrelevant data, reduce unnecessary computation, and improve resource utilization efficiency. Simultaneously, the multi-source data fusion and integrity / accuracy checking mechanisms reduce the impact of single data failures, improve the fault tolerance of the entire monitoring system, and ensure the stable operation of transmission lines.

[0055] In one embodiment of this application, the meteorological monitoring data for the transmission line includes temperature, humidity, and wind speed;

[0056] Scene types are determined based on meteorological monitoring data of power transmission lines, including:

[0057] If the relationship between temperature and humidity satisfies the first condition, and the wind speed satisfies the second condition, then the scene type is determined to be an icing scene.

[0058] If the wind speed meets the second condition, and the relationship between temperature and humidity does not meet the first condition, then the scene type is determined to be a dancing scene.

[0059] If the relationship between temperature and humidity does not meet the first condition, and the wind speed does not meet the second condition, then the scene type is determined to be a normal scene.

[0060] In this embodiment, the first condition is: the temperature is less than a first threshold and the humidity is greater than a second threshold. The second condition is: the wind speed is greater than a third threshold. The first, second, and third thresholds are preset values.

[0061] For example, this embodiment can collect temperature, humidity, and wind speed data using temperature sensors, humidity sensors, and wind speed sensors installed along the transmission line. This embodiment makes judgments based on pre-set first and second conditions. For example, the first condition is set to a temperature below 0°C and humidity above 85%, and the second condition is set to a wind speed greater than 10 m / s. This embodiment compares whether the temperature and humidity data meet the first condition and whether the wind speed meets the second condition. If the temperature and humidity meet the first condition and the wind speed meets the second condition, it is determined to be an icing scenario; if only the wind speed meets the second condition, but the temperature and humidity do not meet the first condition, it is determined to be a galloping scenario; if neither condition is met, it is determined to be a normal scenario. If both conditions are met, they are processed in parallel. This embodiment activates corresponding monitoring strategies based on the scenario type, such as focusing on monitoring icing-related parameters in icing scenarios, monitoring the galloping state of conductors in galloping scenarios, and performing routine component checks in normal scenarios.

[0062] This embodiment provides a method for determining the scenario type and activating the corresponding monitoring strategy based on meteorological monitoring data, which can accurately identify different operating scenarios of transmission lines.

[0063] In one embodiment of this application, the sensor monitoring data includes conductor tension, tower tilt angle, and conductor vibration signal;

[0064] The feature weight matrix of sensor monitoring data is determined based on scene type, including:

[0065] Based on the scenario type, an attention mechanism is used to assign feature attention weights to conductor tension, tower tilt angle and conductor vibration frequency.

[0066] A feature weight matrix is ​​generated based on the feature attention weights of conductor tension, tower tilt angle, and conductor vibration frequency.

[0067] In this embodiment, feature extraction is performed on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set, specifically including:

[0068] Calculate tension fluctuation entropy based on conductor tension;

[0069] Extracting the frequency domain characteristics of conductor vibration signals;

[0070] Calculate the rate of change of tilt angle based on the tower tilt angle;

[0071] The weighted sensor monitoring feature set is obtained by multiplying the feature weight matrix with the tension fluctuation entropy, the frequency domain features of the conductor vibration signal, and the tilt angle change rate.

[0072] In this embodiment, conductor tension refers to the tensile force borne by the conductor; tower tilt angle refers to the angle by which the tower deviates from the vertical direction; conductor vibration signal is the signal formed by the vibration of the conductor due to factors such as wind force and current, including parameters such as vibration frequency and amplitude. Tension fluctuation entropy is obtained by calculating the probability distribution of tension data and is used to measure the complexity of conductor tension fluctuations, reflecting the uncertainty of tension changes, and is an indicator for judging the stability of the conductor under stress. Frequency domain characteristics refer to the features extracted after converting the conductor vibration signal from the time domain to the frequency domain, such as dominant frequency and frequency amplitude, used to analyze the frequency components and energy distribution of vibration. Tilting angle change rate refers to the amount of change in the tower tilt angle per unit time, reflecting the speed of tower tilt change.

[0073] In this embodiment, the risk points of transmission lines differ under different scenarios, and the importance of each sensor's data in determining the line's condition also varies. This embodiment utilizes an attention mechanism to generate a feature weight matrix, which can highlight key data features and weaken secondary features according to the scenario. By calculating tension fluctuation entropy, extracting frequency domain features, and measuring the rate of change of tilt angle, this embodiment can uncover deeper features of the data. Multiplying the extracted features by the weight matrix yields a more representative set of sensor monitoring features, improving the accuracy of line condition assessment.

[0074] For example, this embodiment can acquire relevant data through tension sensors, tower tilt angle sensors, and conductor vibration sensors installed on the transmission line. For instance, the tension sensor is installed at a fixed point on the conductor, the tower tilt angle sensor is installed at a key support point at the bottom of the tower, and the conductor vibration sensor is installed at a location on the conductor prone to vibration.

[0075] This embodiment acquires conductor tension data over a period of time, statistically analyzes the probability distribution of the tension data, and calculates the tension fluctuation entropy using the information entropy formula. This embodiment processes the acquired conductor vibration signal, using Fourier transform to convert the time-domain vibration signal to the frequency domain, obtaining a spectrum. This embodiment extracts frequency domain features such as dominant frequency and frequency amplitude from the spectrum; these features reflect the frequency components and energy distribution of the conductor vibration. This embodiment records tower tilt angle data at multiple consecutive moments and calculates the rate of change of tilt angle.

[0076] This embodiment uses the calculated tension fluctuation entropy, the frequency domain characteristics of the conductor vibration signal, and the tilt angle change rate to form a feature vector. Based on the currently determined scene type, this embodiment utilizes an attention mechanism to determine the feature attention weights for conductor tension, tower tilt angle, and conductor vibration frequency. For example, in a galloping scene, the weights of tower tilt angle and conductor vibration frequency are higher than the weight of conductor tension, while in an icing scene, the weight of conductor tension is higher than the weights of tower tilt angle and conductor vibration frequency.

[0077] In this embodiment, the feature weight matrix and the feature vector are multiplied by a dot product to obtain a weighted sensor monitoring feature set. This feature set comprehensively considers the importance of each feature under different scenarios and can more accurately reflect the actual state of the transmission line.

[0078] This embodiment dynamically allocates sensor data weights for different scenarios, enabling precise focus on key risk parameters in scenarios such as icing and galloping, while avoiding interference from secondary information. Simultaneously, it mines deep data features, and the resulting weighted and fused feature set more comprehensively and accurately reflects the line's operating status, improving anomaly identification accuracy and achieving efficient and precise assessment of transmission line conditions, thus providing strong support for the safe and stable operation of the lines.

[0079] In one embodiment of this application, determining a video feature extraction strategy based on scene type includes:

[0080] If the scene type is an icing scene, then the video feature extraction strategy is determined to be the first extraction strategy; the first extraction strategy includes: performing region segmentation on the video frames of the transmission line, and taking the segmented conductor region and insulator string region as the target region;

[0081] Texture features of the target region are extracted using the gray-level co-occurrence matrix;

[0082] The target area is binarized using a threshold segmentation algorithm to obtain icing and non-icing areas, and the proportion of icing area is calculated based on the icing and non-icing areas.

[0083] Edge detection is performed on the insulator string area to obtain edge icing characteristics.

[0084] If the scene type is a dancing scene, then the video feature extraction strategy is determined to be the second extraction strategy; the first extraction strategy includes: using a deep learning optical flow algorithm to calculate the motion vector of the wire pixels between adjacent video frames;

[0085] A motion trajectory model of the conductor is established based on motion vectors, and the dancing pattern of the conductor is determined based on the motion trajectory model;

[0086] The amplitude and frequency features of conductor swing are extracted based on motion vectors.

[0087] If the scene type is a normal scene, then the video feature extraction strategy is determined to be the third extraction strategy; the third extraction strategy includes: using a multi-target detection algorithm to identify multiple component regions in the video frame, including insulator regions, hardware regions, conductor regions and tower regions;

[0088] Defect features are extracted from multiple component regions to obtain a subset of defect features corresponding to each component region.

[0089] For each component region: establish a feature value time series based on the defect feature subset corresponding to that component region.

[0090] In this embodiment, the threshold segmentation algorithm refers to a method that classifies image pixels into different categories based on a comparison of their grayscale values ​​with a set threshold. The threshold setting is a key parameter and can be determined through methods such as fixed thresholds or adaptive thresholds. The feature value time series refers to a sequence formed by arranging the defect feature values ​​of a component region in chronological order, used to analyze the development trend of defects. Parameters include time intervals and feature value types.

[0091] In this embodiment, the risks and characteristics of transmission lines differ significantly under different scenarios. In the icing scenario, ice formation mainly affects the appearance and texture of conductors and insulators; therefore, icing is monitored through region segmentation, texture analysis, and ice area calculation. In the galloping scenario, conductor movement is the main monitoring point; optical flow algorithms can be used to track pixel movement and analyze galloping patterns and amplitude frequencies. In ordinary scenarios, it is necessary to focus on monitoring long-term defect changes in components; multi-target detection can be used to locate components, extract defect features, and establish time series for trend analysis. The strategy of this embodiment can extract effective information in a targeted manner according to the characteristics of the scenario, improving the accuracy and efficiency of monitoring.

[0092] For example, for video feature extraction in icing scenes, this embodiment can employ a deep learning-based semantic segmentation model, such as U-Net, to segment transmission line video frames into regions. For instance, the model can be trained to identify conductor and insulator string regions, which are then used as target regions. For each target region, this embodiment first determines parameters such as pixel spacing and grayscale levels, then calculates the grayscale co-occurrence matrix, and subsequently extracts texture features such as contrast and correlation of the target region to reflect the surface texture of the ice layer.

[0093] This embodiment can use a threshold segmentation algorithm, such as the Otsu algorithm, to binarize the target region, dividing the image into iced and uniced areas, calculating the area ratio of the iced region, and assessing the degree of icing. This embodiment can also employ Canny edge detection to detect edges in the insulator string region, obtaining edge icing features to determine the icing status of the insulator string.

[0094] For example, for feature extraction from a dancing scene video, this embodiment can utilize a deep learning optical flow algorithm, such as FlowNet, to calculate the motion vectors of conductor pixels between adjacent video frames, capturing the conductor's motion information. Based on the obtained motion vectors, this embodiment establishes a trajectory model of the conductor. By analyzing features such as the shape and period of the trajectory, the dancing pattern of the conductor is determined, such as elliptical dancing or vertical dancing. This embodiment extracts the conductor's swing amplitude and frequency features from the motion vectors to evaluate the intensity and frequency of the dancing, thereby determining the degree of impact of the dancing on the line.

[0095] For example, in extracting video features from ordinary scenes, this embodiment can employ a multi-target detection algorithm, such as YOLOv5, to identify insulator regions, hardware regions, conductor regions, and tower regions in the video frame, locating each component. For each component region, this embodiment can adopt a corresponding defect feature extraction method based on its characteristics. For example, for the insulator region, features such as cracks and damage are detected; for the hardware region, defects such as corrosion and deformation are identified. This embodiment can establish a feature value time series for each component region, recording defect feature values ​​at certain time intervals, and predicting the development of component defects by analyzing the changing trends of the time series.

[0096] This embodiment addresses icing, galloping, and normal scenarios, focusing on core risk characteristics such as appearance texture, dynamic movement, and component defects, thus avoiding information redundancy or omissions caused by fixed, single monitoring methods. This embodiment employs advanced technologies such as deep learning optical flow algorithms and multi-target detection to accurately extract key information and combines it with time series analysis to predict defect development trends. This strategy can not only quickly identify current line anomalies but also predict potential risks in advance, providing strong support for operation and maintenance decisions, reducing the failure rate, and ensuring the safe and stable operation of transmission lines.

[0097] Corresponding to the multifunctional online video monitoring method for transmission lines in the above embodiments, Figure 2 This is a structural block diagram of a multifunctional online video monitoring device for transmission lines provided in one embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. (See references) Figure 2 The multi-functional online video monitoring device 20 for power transmission lines includes: a scene recognition module 21, a feature extraction module 22, and a status analysis module 23.

[0098] The scene recognition module 21 is used to determine the scene type based on meteorological monitoring data of the power transmission line, and to determine the video feature extraction strategy and the feature weight matrix of the sensor monitoring data based on the scene type; the scene types include icing scene, dancing scene and ordinary scene;

[0099] Feature extraction module 22 is used to extract image features from video frames of transmission lines based on video feature extraction strategies to obtain a video monitoring feature set; and to extract features from sensor monitoring data based on the feature weight matrix of sensor monitoring data to obtain a sensor monitoring feature set.

[0100] The status analysis module 23 is used to perform multimodal feature fusion of sensor monitoring feature set and video monitoring feature set to obtain fused image feature set; and to determine the status information of transmission line based on fused image feature set.

[0101] In one embodiment of this application, the meteorological monitoring data of the transmission line includes temperature, humidity and wind speed; the scene recognition module 21 is specifically used to determine the scene type as an icing scene if the relationship between temperature and humidity meets a first condition and the wind speed meets a second condition.

[0102] If the wind speed meets the second condition, and the relationship between temperature and humidity does not meet the first condition, then the scene type is determined to be a dancing scene.

[0103] If the relationship between temperature and humidity does not meet the first condition, and the wind speed does not meet the second condition, then the scene type is determined to be a normal scene.

[0104] In one embodiment of this application, the sensor monitoring data includes conductor tension, tower tilt angle, and conductor vibration signal; the scene recognition module 21 is further configured to assign feature attention weights to conductor tension, tower tilt angle, and conductor vibration frequency based on scene type using an attention mechanism;

[0105] A feature weight matrix is ​​generated based on the feature attention weights of conductor tension, tower tilt angle, and conductor vibration frequency.

[0106] In one embodiment of this application, the feature extraction module 22 is specifically used to calculate the tension fluctuation entropy based on the conductor tension;

[0107] Extracting the frequency domain characteristics of conductor vibration signals;

[0108] Calculate the rate of change of tilt angle based on the tower tilt angle;

[0109] The weighted sensor monitoring feature set is obtained by multiplying the feature weight matrix with the tension fluctuation entropy, the frequency domain features of the conductor vibration signal, and the tilt angle change rate.

[0110] In one embodiment of this application, the scene recognition module 21 is specifically used to determine the video feature extraction strategy as follows if the scene type is an icing scene: perform region segmentation on the video frame of the transmission line, and take the segmented conductor region and insulator string region as the target region.

[0111] Texture features of the target region are extracted using the gray-level co-occurrence matrix;

[0112] The target area is binarized using a threshold segmentation algorithm to obtain icing and non-icing areas, and the proportion of icing area is calculated based on the icing and non-icing areas.

[0113] Edge detection is performed on the insulator string area to obtain edge icing characteristics.

[0114] In one embodiment of this application, the scene recognition module 21 is further configured to determine the video feature extraction strategy as follows if the scene type is a dancing scene: calculate the motion vector of the wire pixels between adjacent video frames using a deep learning optical flow algorithm.

[0115] A motion trajectory model of the conductor is established based on motion vectors, and the dancing pattern of the conductor is determined based on the motion trajectory model;

[0116] The amplitude and frequency features of conductor swing are extracted based on motion vectors.

[0117] In one embodiment of this application, the scene recognition module 21 is further configured to determine the video feature extraction strategy as follows if the scene type is a normal scene: using a multi-target detection algorithm to identify multiple component regions in the video frame, including insulator regions, hardware regions, conductor regions and tower regions.

[0118] Defect features are extracted from multiple component regions to obtain a subset of defect features corresponding to each component region.

[0119] For each component region: establish a feature value time series based on the defect feature subset corresponding to that component region.

[0120] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of the scene recognition module 21, feature extraction module 22, and state analysis module 23 are shown.

[0121] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), but it may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0122] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0123] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store sensor-type information.

[0124] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in the embodiments of the multifunctional transmission line video online monitoring method provided in the embodiments of this application, or they can execute the implementation methods of the electronic device 300 described in the embodiments of this application, which will not be repeated here.

[0125] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0126] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0127] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0128] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0129] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0130] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0131] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0132] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A multifunctional online video monitoring method for transmission lines, characterized in that, include: The scene type is determined based on meteorological monitoring data of power transmission lines, and the video feature extraction strategy and feature weight matrix of sensor monitoring data are determined based on the scene type. Based on the video feature extraction strategy, image features are extracted from the video frames of the transmission line to obtain a video monitoring feature set; based on the feature weight matrix of the sensor monitoring data, features are extracted from the sensor monitoring data to obtain a sensor monitoring feature set. The sensor monitoring feature set and the video monitoring feature set are fused using multimodal features to obtain a fused image feature set; the status information of the transmission line is determined based on the fused image feature set. The meteorological monitoring data for the transmission lines include temperature, humidity, and wind speed; The method of determining the scene type based on meteorological monitoring data of power transmission lines includes: if the relationship between the temperature and the humidity meets a first condition and the wind speed meets a second condition, then the scene type is determined to be an icing scene; if the wind speed meets the second condition and the relationship between the temperature and the humidity does not meet the first condition, then the scene type is determined to be a dancing scene; if the relationship between the temperature and the humidity does not meet the first condition and the wind speed does not meet the second condition, then the scene type is determined to be a normal scene. The sensor monitoring data includes conductor tension, tower tilt angle, and conductor vibration signal; Determining the feature weight matrix of sensor monitoring data based on the scenario type includes: assigning feature attention weights to the conductor tension, the tower tilt angle, and the conductor vibration frequency using an attention mechanism based on the scenario type; and generating a feature weight matrix based on the feature attention weights of the conductor tension, the tower tilt angle, and the conductor vibration frequency.

2. The multifunctional online video monitoring method for transmission lines as described in claim 1, characterized in that, The feature extraction of the sensor monitoring data based on the feature weight matrix of the sensor monitoring data yields a sensor monitoring feature set, including: Calculate the tension fluctuation entropy based on the conductor tension; Extract the frequency domain features of the conductor vibration signal; Calculate the rate of change of tilt angle based on the tilt angle of the tower; The weighted feature matrix is ​​multiplied by the tension fluctuation entropy, the frequency domain features of the conductor vibration signal, and the tilt angle change rate to obtain the weighted sensor monitoring feature set.

3. The multifunctional online video monitoring method for transmission lines as described in claim 1, characterized in that, Based on the scene type, a video feature extraction strategy is determined, including: If the scene type is the icing scene, then the video feature extraction strategy is determined to be the first extraction strategy; the first extraction strategy includes: The video frames of the transmission line are segmented into regions, and the segmented conductor region and insulator string region are used as target regions. Texture features of the target region are extracted using the gray-level co-occurrence matrix; The target region is binarized using a threshold segmentation algorithm to obtain icing and non-icing regions, and the icing area ratio is calculated based on the icing and non-icing regions. Edge detection is performed on the insulator string region to obtain edge icing characteristics.

4. The multifunctional online video monitoring method for transmission lines as described in claim 3, characterized in that, Based on the scene type, a video feature extraction strategy is determined, including: If the scene type is the dancing scene, then the video feature extraction strategy is determined to be the second extraction strategy; the second extraction strategy includes: The motion vectors of wire pixels between adjacent video frames are calculated using a deep learning optical flow algorithm. A motion trajectory model of the conductor is established based on the motion vector, and the dancing pattern of the conductor is determined based on the motion trajectory model. Based on the motion vector, the characteristics of the conductor swing amplitude and the conductor swing frequency are extracted.

5. The multifunctional online video monitoring method for transmission lines as described in claim 4, characterized in that, Based on the scene type, a video feature extraction strategy is determined, including: If the scene type is the ordinary scene, then the video feature extraction strategy is determined to be the third extraction strategy; the third extraction strategy includes: A multi-target detection algorithm is used to identify multiple component regions in a video frame, including insulator regions, hardware regions, conductor regions, and tower regions. Defect features are extracted from the multiple component regions to obtain a subset of defect features corresponding to each component region; For each component region: establish a feature value time series based on the defect feature subset corresponding to that component region.

6. A multifunctional online video monitoring device for power transmission lines, characterized in that, include: The scene recognition module is used to determine the scene type based on meteorological monitoring data of the power transmission line, and to determine the video feature extraction strategy and the feature weight matrix of the sensor monitoring data based on the scene type; the meteorological monitoring data of the power transmission line includes temperature, humidity and wind speed; The scene recognition module is specifically used to determine the scene type as an icing scene if the relationship between the temperature and humidity satisfies a first condition and the wind speed satisfies a second condition; if the wind speed satisfies the second condition but the relationship between the temperature and humidity does not satisfy the first condition, the scene type is determined as a dancing scene; if the relationship between the temperature and humidity does not satisfy the first condition and the wind speed does not satisfy the second condition, the scene type is determined as a normal scene; the sensor monitoring data includes conductor tension, tower tilt angle, and conductor vibration signal. The scene recognition module is specifically used to assign feature attention weights to the conductor tension, the tower tilt angle, and the conductor vibration frequency based on the scene type using an attention mechanism; A feature weight matrix is ​​generated based on the feature attention weights of the conductor tension, the tower tilt angle, and the conductor vibration frequency. The feature extraction module is used to extract image features from the video frames of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; and to extract features from the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set. The status analysis module is used to perform multimodal feature fusion of the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and to determine the status information of the transmission line based on the fused image feature set.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images