Multifunctional power transmission line video on-line monitoring method and device

Abstract

The invention provides a multifunctional power transmission line video online monitoring method and device, and belongs to the technical field of power transmission monitoring, and the method comprises the steps: determining a scene type based on the meteorological monitoring data of a power transmission line, and determining a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene type comprises an icing scene, a galloping scene and a common scene; performing image feature extraction on the video frame of the power transmission line based on a video feature extraction strategy to obtain a video monitoring feature set; performing feature extraction on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; performing multi-modal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and determining state information of the power transmission line based on the fused image feature set. According to the invention, the monitoring accuracy of the abnormal condition of the power transmission line in a complex meteorological scene can be improved.

Description

Technical Field

[0001] The present application belongs to the field of power transmission monitoring technology, and more specifically, relates to a multifunctional power transmission line video online monitoring method and device. Background Art

[0002] The safe and stable operation of transmission lines is crucial to power systems, and real-time monitoring of line status is a key means of preventing failures. Traditional transmission line monitoring methods rely primarily on fixed video surveillance systems, which collect data or images for status analysis. However, the complex and ever-changing environments in which transmission lines operate are susceptible to complex meteorological conditions. Existing video monitoring methods have low accuracy in identifying line fault risks in complex weather scenarios. Summary of the Invention

[0003] The purpose of this application is to provide a multifunctional online video monitoring method and device for power transmission lines, so as to improve the monitoring accuracy of abnormal conditions of power transmission lines in complex meteorological scenarios.

[0004] A first aspect of an embodiment of the present application provides a multifunctional transmission line video online monitoring method, comprising: Determining a scene type based on meteorological monitoring data of the power transmission line, and determining a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene types include ice-covered scenes, dancing scenes, and ordinary scenes; Performing image feature extraction on the video frame of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; performing feature extraction on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; The sensor monitoring feature set and the video monitoring feature set are subjected to multimodal feature fusion to obtain a fused image feature set; and the state information of the transmission line is determined based on the fused image feature set.

[0005] A second aspect of the embodiments of the present application provides a multifunctional online video monitoring device for a power transmission line, comprising: A scene recognition module is configured to determine a scene type based on meteorological monitoring data of the transmission line, and to determine a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene types include ice-covered scenes, dancing scenes, and ordinary scenes; a feature extraction module configured to extract image features from the video frames of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; and to extract features from the sensor monitoring data based on a feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; The state analysis module is used to perform multimodal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and determine the state information of the transmission line based on the fused image feature set.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned multifunctional transmission line video online monitoring method when executing the computer program.

[0007] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned multifunctional transmission line video online monitoring method are implemented.

[0008] The beneficial effects of the multifunctional transmission line video online monitoring method and device provided in the embodiments of the present application are: on the one hand, the embodiments of the present application determine the scene type through meteorological monitoring data, and formulate targeted video feature extraction strategies and sensor feature weight matrices, thereby realizing adaptive monitoring of complex meteorological scenes and solving the problem of low recognition accuracy of traditional methods under harsh conditions.

[0009] On the other hand, the embodiment of the present application adopts multimodal fusion technology to combine the differentially extracted video image features with sensor data features, which makes up for the limitations of single video monitoring in complex environments, and realizes enhanced extraction of key features through dynamic weight allocation, significantly improving the comprehensiveness and accuracy of abnormal state recognition.

[0010] In summary, the embodiments of the present application can accurately capture subtle anomalies of transmission lines under different meteorological conditions through the collaborative mechanism of scene perception and feature fusion, providing a more reliable real-time monitoring means for the safe operation of the power system, effectively reducing the risk of failure, and improving the intelligence level of transmission line status assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0012] Figure 1 A flowchart of a multifunctional online video monitoring method for power transmission lines provided in one embodiment of the present application; Figure 2A structural block diagram of a multifunctional transmission line video online monitoring device provided in one embodiment of the present application; Figure 3 A schematic block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0013] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0014] In order to make the purpose, technical solutions and advantages of this application clearer, specific embodiments will be described below with reference to the accompanying drawings.

[0015] Please refer to Figure 1 , Figure 1 This is a flow chart of a multifunctional transmission line video online monitoring method provided in one embodiment of the present application. The method can be executed by an electronic device. Specifically, the method can include S101 to S103.

[0016] S101: Determine a scene type based on meteorological monitoring data of the transmission line, and determine a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene types include ice-covered scenes, dancing scenes, and ordinary scenes.

[0017] In this embodiment, transmission line meteorological monitoring data refers to environmental meteorological parameters collected in real time by sensors deployed around the transmission line. These data are used to determine the type of impact the current meteorological conditions are having on the transmission line. For example, meteorological monitoring data can be used to determine whether the transmission line is experiencing low temperatures, high humidity, or strong winds. Low temperatures and high humidity are key conditions for icing, while strong winds can cause conductor oscillation or accelerate icing.

[0018] In this embodiment, icing scenarios refer to low-temperature, high-humidity environments where ice forms on the surfaces of conductors and insulators, threatening line load safety. Galloping scenarios refer to strong winds causing low-frequency, high-amplitude vibrations in conductors, which can easily lead to hardware damage or phase short circuits. Normal scenarios refer to stable weather conditions with no significant icing or galloping risks, allowing for monitoring of common component defects.

[0019] In this embodiment, the video feature extraction strategy may include differentiated video feature extraction methods for ice-covered scenes, dancing scenes, and normal scenes. A feature weight matrix is a matrix formed by assigning weights to sensor data using an attention mechanism. For example, the feature weight matrix may include weight coefficients corresponding to data monitored by a vibration sensor, a tension sensor, and an inclination sensor.

[0020] In this embodiment, due to the complex environment of power transmission lines, different meteorological conditions trigger varying risks. The risk factors and key monitoring targets for power transmission lines vary across different scenarios, leading to significant differences in the correlation between sensor data and line status. By dynamically selecting sensor features that are strongly relevant to the current scenario and reducing redundant data interference, the accuracy of transmission line status judgment after multimodal fusion can be improved.

[0021] This embodiment simplifies complex problems by classifying scenarios using meteorological data, customizing monitoring strategies for each scenario and ensuring optimal allocation of computing resources. For example, icing scenarios can prioritize monitoring ice conditions, dancing scenarios can prioritize monitoring conductor dynamics, and general scenarios can prioritize monitoring component defects. A feature weight matrix adjusts data importance based on the scenario, avoiding interference from irrelevant data and ensuring optimal use of computing resources while improving monitoring accuracy.

[0022] For example, this embodiment can install temperature and humidity sensors and anemometers along the transmission line to obtain meteorological data; deploy tension sensors, tower tilt sensors, and vibration sensors to collect line operation data; and deploy high-definition cameras to capture video data at a fixed frame rate. This embodiment determines the scene type through logical judgment based on the judgment conditions set for temperature, humidity, and wind speed.

[0023] This embodiment can call the corresponding video feature extraction strategy from the strategy library according to the identified scene type, such as enabling the region segmentation and texture analysis strategy for the ice-covered scene; at the same time, this embodiment uses the attention mechanism to assign weights to the sensor data according to the scene type and generate a feature weight matrix. For example, the weight of the wire vibration signal can be increased in the dancing scene.

[0024] S102: performing image feature extraction on the video frame of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; performing feature extraction on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set.

[0025] In this embodiment, a video frame refers to a continuous sequence of images captured by a transmission line camera. Image features refer to condition monitoring-related information extracted from the video frames, such as the ice area percentage in an icing scenario or the conductor swing amplitude in a dancing scenario. Sensor monitoring data can include data such as conductor tension, tower inclination angle, and conductor vibration signals, including parameters such as numerical values and sampling frequency.

[0026] Given that transmission line fault hazards manifest differently in video images and sensor data, a single data point cannot fully reflect the line status. This embodiment employs a differentiated video feature extraction strategy and feature weight matrix to specifically extract key information from different data sources, leveraging the complementary advantages of multi-source data to improve the accuracy of monitoring line abnormalities.

[0027] Exemplarily, this embodiment first confirms that an adapted video feature extraction strategy and a feature weight matrix of sensor monitoring data have been selected according to the scene type; and simultaneously obtains transmission line video frame data, as well as sensor monitoring data such as conductor tension, tower inclination angle, and conductor vibration signal.

[0028] In terms of video feature extraction, this embodiment processes video frames according to a selected video feature extraction strategy. For example, this embodiment can first segment the power transmission line video, breaking it down into multiple single-frame images (i.e., video frames) and then perform preprocessing operations such as grayscale conversion, filtering and noise reduction, and size normalization to improve image quality. This embodiment can use classic algorithms such as SIFT and SURF to extract local key points and descriptors from the image, and HOG and LBP algorithms to obtain global image contour and texture features. This embodiment can also utilize a pretrained convolutional neural network model to automatically extract deep image features through forward propagation. Finally, this embodiment can cascade and fuse features extracted by different methods to form a complete video monitoring feature set.

[0029] For sensor monitoring data, this embodiment can first remove outliers, interpolate missing values, and normalize the raw data. It then uses algorithms such as principal component analysis and mutual information to calculate the importance of each sensor data feature and construct a feature weight matrix. Based on this constructed feature weight matrix, this embodiment can weight the data to highlight key information. It also uses a sliding window technique to extract time-domain features and combines these weighted features to generate a sensor monitoring feature set.

[0030] After obtaining the processed feature set, this embodiment can check the integrity and accuracy of the generated video monitoring feature set and sensor monitoring feature set to ensure that no important features are missed or the data is abnormal, thereby providing reliable data for subsequent multimodal fusion and line status assessment.

[0031] S103: Perform multimodal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and determine state information of the transmission line based on the fused image feature set.

[0032] In this embodiment, the sensor monitoring feature set and the video monitoring feature set are subjected to multimodal feature fusion to obtain a fused image feature set, which specifically includes: Perform linear interpolation on the sensor monitoring feature set to generate a sensor data interpolation sequence aligned with the time stamp of the video monitoring feature set; Perform dimensionality reduction on the video monitoring feature set to obtain a video monitoring feature set with the same dimension as the sensor monitoring feature set; The features in the video monitoring feature set are matched with the target feature template, multiple target positions are determined based on the matching results, and the features in the sensor data interpolation sequence are associated and aligned with the multiple target positions respectively.

[0033] In this embodiment, the sensor monitoring feature set refers to a collection of feature data collected and processed from various sensors. For example, it may include parameters such as conductor tension, tower inclination, and conductor vibration signals. The video monitoring feature set refers to a collection of features extracted through video analysis of transmission line videos. For example, it may include visual features such as conductor appearance, insulator status, and surrounding environment, and exists in the form of image feature vectors.

[0034] Multimodal feature fusion combines features from different modalities, including sensor and video monitoring, to form a more representative fused feature set. Timestamp alignment ensures that the sensor and video monitoring feature sets are aligned in time, facilitating fusion analysis.

[0035] A target feature template is a pre-established sample set containing typical features of various targets, used for matching with a video monitoring feature set. For example, it can include conductor feature templates, tower feature templates, and so on. The purpose of establishing a target feature template is to more accurately and efficiently identify different target objects from the video monitoring feature set. In practice, a large amount of image data of various targets on transmission lines in different environments and at different angles can be collected. Image processing and machine learning techniques can be used to extract stable and representative features to construct a target feature template library. During real-time monitoring, it can be used to quickly match the video monitoring feature set with the features in the template library to achieve accurate identification and positioning of various parts of the transmission line.

[0036] In this embodiment, data from different modalities differ in time and dimension, so direct fusion would compromise accuracy and efficiency. This embodiment uses linear interpolation to synchronize the time of sensor data with video data. Dimensionality reduction allows the two to align their dimensions, placing them in the same spatial dimension. Feature template matching and correlation alignment enable precise alignment of features from different modalities, comprehensively reflecting the status of the transmission line.

[0037] Exemplarily, this embodiment can obtain the timestamp information of the sensor monitoring feature set and the video monitoring feature set, and determine the difference between the two in the time series. This embodiment selects a suitable linear interpolation method based on the acquisition frequency and time interval of the sensor data. For example, if the data changes relatively smoothly, simple linear interpolation can be used; if the data fluctuates greatly, a higher-order method such as Lagrange interpolation is used. This embodiment uses the timestamp of the video monitoring feature set as a reference, and performs interpolation calculations on the sensor monitoring feature set according to the selected interpolation method to fill in data gaps or adjust data time points to generate an interpolation sequence aligned with the timestamp of the video monitoring feature set.

[0038] For the video monitoring feature set, this embodiment can select a suitable dimensionality reduction algorithm, such as principal component analysis or linear discriminant analysis. If principal component analysis is used, the covariance matrix of the video monitoring feature set can be calculated, the eigenvalues and eigenvectors can be obtained, and the main components can be selected according to the cumulative contribution rate to achieve dimensionality reduction. If linear discriminant analysis is used, the category information of the data can be considered to find the projection direction that can maximize the inter-class distance and minimize the intra-class distance, thereby reducing the dimension. This embodiment uses dimensionality reduction processing to make the dimension of the video monitoring feature set consistent with the sensor monitoring feature set, reduce computational complexity, and retain key feature information to facilitate subsequent fusion.

[0039] This embodiment matches the video monitoring feature set after dimensionality reduction with a pre-constructed target feature template. This embodiment can use a distance metric-based method, such as Euclidean distance, cosine similarity, etc., to calculate the similarity between the video features and the template features. This embodiment determines the positions of multiple targets in the video based on the matching results. For example, the position coordinates of targets such as conductors and insulators in the video image are identified. Based on the obtained target position, this embodiment associates and aligns the features in the sensor monitoring feature set with the corresponding target position, completes multimodal feature fusion, and obtains a fused image feature set to provide comprehensive data support for determining the status of the transmission line. For example, the conductor vibration features in the sensor monitoring feature set are associated and aligned with the corresponding target conductor position.

[0040] For example, in actual application, determining the state information of the power transmission line based on the fused image feature set may specifically include: The fused image feature set is fed into a trained state recognition model. This model can utilize a deep learning model suitable for multimodal data, such as a multimodal convolutional neural network or Transformer architecture. The state recognition model determines the line status based on the learned patterns. For example, if the fused image feature set indicates an abnormal conductor vibration frequency and the video captures contact between the conductor and a foreign object, the model quickly identifies external force damage and issues an alert. If the model determines a line anomaly, it can accurately locate the fault point based on the location information in the fused image feature set and output information such as the fault type and location.

[0041] From the above, it can be concluded that this embodiment divides scene types through meteorological data, customizes video feature extraction strategies and sensor data weights for different scenes, and also integrates multimodal features to comprehensively capture line status information, thereby improving the monitoring accuracy of abnormal conditions such as icing and dancing, as well as component defects, and can promptly detect potential risks.

[0042] This embodiment also enhances the system's adaptability and reliability. By segmenting scenarios and performing differentiated monitoring, it can flexibly adjust monitoring strategies based on real-time meteorological conditions, allocate computing resources to different scenarios, avoid interference from irrelevant data, reduce unnecessary computations, and improve resource utilization efficiency. Furthermore, multi-source data fusion and integrity and accuracy checking mechanisms mitigate the impact of single data failures, improve the fault tolerance of the entire monitoring system, and ensure stable operation of the transmission line.

[0043] In one embodiment of the present application, the environmental monitoring data of the power transmission line includes temperature, humidity and wind speed; Determine the scenario type based on the environmental monitoring data of the transmission line, including: If the relationship between temperature and humidity meets the first condition, and the wind speed meets the second condition, then the scene type is determined to be an ice-covered scene; If the wind speed meets the second condition, and the relationship between temperature and humidity does not meet the first condition, then the scene type is determined to be a dancing scene; If the magnitude relationship between the temperature and the humidity does not satisfy the first condition, and the wind speed does not satisfy the second condition, then the scene type is determined to be a normal scene.

[0044] In this embodiment, the first condition is that the temperature is less than a first threshold and the humidity is greater than a second threshold. The second condition is that the wind speed is greater than a third threshold. The first, second, and third thresholds are preset values.

[0045] Exemplarily, this embodiment can collect temperature, humidity and wind speed data through temperature sensors, humidity sensors and wind speed sensors installed along the transmission line. This embodiment makes judgments based on the pre-set first and second conditions. For example, the first condition is set to a temperature below 0°C and a humidity above 85%, and the second condition is set to a wind speed greater than 10m / s. This embodiment compares whether the temperature and humidity data meet the first condition and whether the wind speed meets the second condition. If the temperature and humidity meet the first condition and the wind speed meets the second condition, it is determined to be an icing scene; if only the wind speed meets the second condition and the temperature and humidity do not meet the first condition, it is determined to be a dancing scene; if neither of them is satisfied, it is determined to be a normal scene. If both are satisfied, they are processed in parallel. This embodiment starts the corresponding monitoring strategy according to the scene type. For example, the icing scene focuses on monitoring icing-related parameters, the dancing scene pays attention to the dancing state of the wires, and the normal scene performs routine component inspections.

[0046] This embodiment provides a method for determining scenario types based on environmental monitoring data and initiating corresponding monitoring strategies, which can accurately identify different operating scenarios of transmission lines.

[0047] In one embodiment of the present application, the sensor monitoring data includes conductor tension, tower inclination angle, and conductor vibration signal; Determine the feature weight matrix of sensor monitoring data based on the scene type, including: Based on the scene type, the attention mechanism is used to assign feature attention weights to the conductor tension, tower tilt angle, and conductor vibration frequency; A feature weight matrix is generated based on the feature attention weight of conductor tension, the feature attention weight of tower inclination angle and the feature attention weight of conductor vibration frequency.

[0048] In this embodiment, feature extraction is performed on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set, which specifically includes: Calculate the tension fluctuation entropy based on the wire tension; Extract the frequency domain characteristics of the conductor vibration signal; Calculate the rate of change of the tilt angle based on the tilt angle of the tower; The feature weight matrix is multiplied by the tension fluctuation entropy, the frequency domain characteristics of the conductor vibration signal and the tilt angle change rate to obtain the weighted sensor monitoring feature set.

[0049] In this embodiment, conductor tension refers to the pulling force borne by the conductor; the tower inclination angle refers to the angle at which the tower deviates from the vertical direction; the conductor vibration signal is a signal generated by the vibration of the conductor due to factors such as wind and current, and includes parameters such as vibration frequency and amplitude. The tension fluctuation entropy is obtained by calculating the probability distribution of tension data. It is used to measure the complexity of conductor tension fluctuations, reflects the uncertainty of tension changes, and is an indicator for judging the stability of conductor force. Frequency domain features refer to features extracted after converting the conductor vibration signal from the time domain to the frequency domain, such as the dominant frequency, frequency amplitude, etc., which are used to analyze the frequency components and energy distribution of the vibration. The inclination angle change rate refers to the change in the tower inclination angle per unit time, which can reflect the changing speed of the tower inclination.

[0050] In this embodiment, the risk points of power transmission lines vary in different scenarios, and the importance of each sensor data to line status assessment also varies. This embodiment utilizes an attention mechanism to generate a feature weight matrix, highlighting key data features and deemphasizing secondary features based on the scenario. By calculating tension fluctuation entropy, extracting frequency domain features, and the rate of change of tilt angle, this embodiment can mine deep-level data features. Multiplying the extracted features by the weight matrix yields a more representative set of sensor monitoring features, improving the accuracy of line status assessment.

[0051] For example, this embodiment can obtain relevant data through tension sensors, tower tilt sensors, and conductor vibration sensors installed on the transmission line. For example, the tension sensor can be installed at a fixed point on the conductor, the tower tilt sensor can be installed at a key support point at the bottom of the tower, and the conductor vibration sensor can be installed at a location prone to conductor vibration.

[0052] This embodiment acquires conductor tension data over a period of time, calculates the probability distribution of this tension data, and calculates the tension fluctuation entropy using the information entropy formula. This embodiment processes the collected conductor vibration signals, converting them from the time domain to the frequency domain using a Fourier transform to produce a spectrum. This embodiment extracts frequency domain features such as the dominant frequency and frequency amplitude from the spectrum. These features reflect the frequency composition and energy distribution of conductor vibration. This embodiment also records tower inclination angle data at multiple consecutive moments and calculates the rate of change of the inclination angle.

[0053] This embodiment combines the calculated tension fluctuation entropy, the frequency domain characteristics of the conductor vibration signal, and the rate of change of the inclination angle into a feature vector. This embodiment utilizes an attention mechanism to determine the feature attention weights for conductor tension, tower inclination angle, and conductor vibration frequency based on the currently determined scenario type. For example, in a dancing scene, the tower inclination angle and conductor vibration frequency are weighted higher than the conductor tension, while in an ice-covered scene, the conductor tension is weighted higher than the tower inclination angle and conductor vibration frequency.

[0054] This embodiment performs a dot product operation on the feature weight matrix and the feature vector to obtain a weighted sensor monitoring feature set. This feature set comprehensively considers the importance of each feature in different scenarios and can more accurately reflect the actual status of the transmission line.

[0055] This embodiment dynamically assigns sensor data weights based on different scenarios, accurately focusing on key risk parameters in scenarios like icing and swaying, while avoiding interference from secondary information. It also mines deep data features, and the resulting weighted fusion creates a feature set that more comprehensively and accurately reflects the line's operating status, improving the accuracy of anomaly identification and enabling efficient and accurate assessment of transmission line status, providing a strong guarantee for safe and stable line operation.

[0056] In one embodiment of the present application, determining a video feature extraction strategy based on a scene type includes: If the scene type is an ice-covered scene, the video feature extraction strategy is determined to be the first extraction strategy; the first extraction strategy includes: performing region segmentation on the video frame of the transmission line, and using the segmented conductor area and insulator string area as the target area; The texture features of the target area are extracted using the gray level co-occurrence matrix; The target area is binarized using a threshold segmentation algorithm to obtain iced and un-iced areas, and the ice area ratio is calculated based on the iced and un-iced areas. Edge detection is performed on the insulator string area to obtain edge ice coverage features.

[0057] If the scene type is a dancing scene, the video feature extraction strategy is determined to be the second extraction strategy; the first extraction strategy includes: using a deep learning optical flow algorithm to calculate the motion vector of the wire pixels between adjacent video frames; A motion trajectory model of the wire is established based on the motion vector, and a dancing pattern of the wire is determined based on the motion trajectory model; The wire swing amplitude characteristics and wire swing frequency characteristics are extracted based on the motion vector.

[0058] If the scene type is a normal scene, the video feature extraction strategy is determined to be the third extraction strategy; the third extraction strategy includes: using a multi-target detection algorithm to identify multiple component areas in the video frame, the multiple component areas including the insulator area, the hardware area, the conductor area, and the tower area; Extract defect features from multiple component regions to obtain defect feature subsets corresponding to each component region; For each component region: a feature value time series is established based on the defect feature subset corresponding to the component region.

[0059] In this embodiment, the threshold segmentation algorithm classifies image pixels into different categories based on the comparison of their grayscale values with a set threshold. The threshold setting is a key parameter and can be determined using methods such as a fixed threshold or an adaptive threshold. The eigenvalue time series is a sequence of defect eigenvalues of a component region arranged in chronological order. It is used to analyze defect development trends. Parameters include time intervals and eigenvalue types.

[0060] In this embodiment, the risks and characteristics of transmission lines in different scenarios vary significantly. In the icing scenario, the formation of ice mainly affects the appearance and texture of the conductors and insulators. Therefore, the icing situation is monitored through regional segmentation, texture analysis, and ice area calculation; in the dancing scenario, the movement of the conductor is the main monitoring point. The optical flow algorithm can be used to track the pixel movement and then analyze the dancing pattern and amplitude frequency; in ordinary scenarios, it is necessary to focus on monitoring the long-term defect changes of components. Multi-target detection can be used to locate the components, extract defect features, and establish a time series for trend analysis. The strategy of this embodiment can extract effective information in a targeted manner according to the characteristics of the scenario, thereby improving the accuracy and efficiency of monitoring.

[0061] For example, to extract video features from ice-covered scenes, this embodiment can use a deep learning-based semantic segmentation model, such as U-Net, to segment transmission line video frames. For example, the model can be trained to identify conductors and insulator strings as target areas. For these target areas, this embodiment first determines parameters such as pixel spacing and grayscale levels, then calculates the gray-level co-occurrence matrix. This allows the extraction of texture features such as contrast and correlation in the target area to reflect the surface texture of the ice.

[0062] This embodiment can use a threshold segmentation algorithm, such as the Otsu algorithm, to binarize the target area, dividing the image into iced and uniced areas, calculating the area ratio of the iced area, and assessing the degree of icing. This embodiment can also use Canny edge detection to detect edges in the insulator string area and obtain edge icing features to determine the icing status of the insulator string.

[0063] For example, to extract features from dancing scene videos, this embodiment can utilize a deep learning optical flow algorithm, such as FlowNet, to calculate the motion vectors of wire pixels between adjacent video frames, capturing the wire's motion information. This embodiment establishes a wire motion trajectory model based on the resulting motion vectors. By analyzing the trajectory's shape, period, and other characteristics, it determines the wire's dancing pattern, such as elliptical or vertical. This embodiment extracts wire swing amplitude and frequency features from the motion vectors to assess the intensity and frequency of the swing, thereby determining the extent of the swing's impact on the line.

[0064] For example, for video feature extraction of common scenes, this embodiment can employ a multi-target detection algorithm, such as YOLOv5, to identify insulators, hardware, conductors, and towers within the video frame and locate each component. For each component region, this embodiment can employ a corresponding defect feature extraction method based on its characteristics. For example, for insulators, features such as cracks and breakage can be detected; for hardware, defects such as rust and deformation can be identified. This embodiment can establish a feature value time series for each component region, recording defect feature values at regular intervals. By analyzing the changing trends of the time series, the development of component defects can be predicted.

[0065] This embodiment focuses on core risk features such as appearance texture, dynamic motion, and component defects for ice-covered, dancing, and ordinary scenes, respectively, to avoid information redundancy or omissions caused by a fixed single monitoring method. This embodiment uses advanced technologies such as deep learning optical flow algorithms and multi-target detection to accurately extract key information and combine time series analysis to predict defect development trends. This strategy can not only quickly identify current line anomalies, but also predict potential risks in advance, providing strong support for operation and maintenance decisions, reducing the occurrence of failures, and ensuring the safe and stable operation of transmission lines.

[0066] Corresponding to the multifunctional transmission line video online monitoring method of the above embodiment, Figure 2 This is a structural block diagram of a multifunctional power transmission line video online monitoring device provided by an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 2 The multifunctional transmission line video online monitoring device 20 includes: a scene recognition module 21, a feature extraction module 22 and a state analysis module 23.

[0067] The scene recognition module 21 is used to determine the scene type based on the meteorological monitoring data of the transmission line, and determine the video feature extraction strategy and the feature weight matrix of the sensor monitoring data based on the scene type; the scene types include ice-covered scene, dancing scene and ordinary scene; A feature extraction module 22 is configured to extract image features from the video frames of the transmission line based on a video feature extraction strategy to obtain a video monitoring feature set; and extract features from the sensor monitoring data based on a feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; The state analysis module 23 is used to perform multimodal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and determine the state information of the transmission line based on the fused image feature set.

[0068] In one embodiment of the present application, the environmental monitoring data of the power transmission line includes temperature, humidity, and wind speed; the scene recognition module 21 is specifically configured to determine that the scene type is an icing scene if the relationship between the temperature and humidity satisfies a first condition and the wind speed satisfies a second condition; If the wind speed meets the second condition, and the relationship between temperature and humidity does not meet the first condition, then the scene type is determined to be a dancing scene; If the magnitude relationship between the temperature and the humidity does not satisfy the first condition, and the wind speed does not satisfy the second condition, then the scene type is determined to be a normal scene.

[0069] In one embodiment of the present application, the sensor monitoring data includes conductor tension, tower inclination angle, and conductor vibration signal; the scene recognition module 21 is further configured to assign feature attention weights to the conductor tension, tower inclination angle, and conductor vibration frequency using an attention mechanism based on the scene type; A feature weight matrix is generated based on the feature attention weight of conductor tension, the feature attention weight of tower inclination angle and the feature attention weight of conductor vibration frequency.

[0070] In one embodiment of the present application, the feature extraction module 22 is specifically configured to calculate the tension fluctuation entropy based on the wire tension; Extract the frequency domain characteristics of the conductor vibration signal; Calculate the rate of change of the tilt angle based on the tilt angle of the tower; The feature weight matrix is multiplied by the tension fluctuation entropy, the frequency domain characteristics of the conductor vibration signal and the tilt angle change rate to obtain the weighted sensor monitoring feature set.

[0071] In one embodiment of the present application, the scene recognition module 21 is specifically configured to, if the scene type is an ice-covered scene, determine a video feature extraction strategy as follows: performing region segmentation on a video frame of the transmission line, and using the segmented conductor region and insulator string region as target regions; The texture features of the target area are extracted using the gray level co-occurrence matrix; The target area is binarized using a threshold segmentation algorithm to obtain iced and un-iced areas, and the ice area ratio is calculated based on the iced and un-iced areas. Edge detection is performed on the insulator string area to obtain edge ice coverage features.

[0072] In one embodiment of the present application, the scene recognition module 21 is further configured to, if the scene type is a dancing scene, determine a video feature extraction strategy as follows: using a deep learning optical flow algorithm to calculate motion vectors of wire pixels between adjacent video frames; A motion trajectory model of the wire is established based on the motion vector, and a dancing pattern of the wire is determined based on the motion trajectory model; The wire swing amplitude characteristics and wire swing frequency characteristics are extracted based on the motion vector.

[0073] In one embodiment of the present application, the scene recognition module 21 is further configured to, if the scene type is a normal scene, determine a video feature extraction strategy as follows: using a multi-target detection algorithm to identify multiple component regions in a video frame, where the multiple component regions include an insulator region, a hardware region, a conductor region, and a tower region; Extract defect features from multiple component regions to obtain defect feature subsets corresponding to each component region; For each component region: a feature value time series is established based on the defect feature subset corresponding to the component region.

[0074] See also Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided in one embodiment of the present application. Figure 3 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned device embodiments, such as Figure 2 The functions of the scene recognition module 21, the feature extraction module 22 and the state analysis module 23 are shown.

[0075] It should be understood that in the embodiment of the present application, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0076] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0077] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the sensor type.

[0078] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiment of the present application can execute the implementation method described in the embodiment of the multi-functional transmission line video online monitoring method provided in the embodiment of the present application, and can also execute the implementation method of the electronic device 300 described in the embodiment of the present application, which will not be repeated here.

[0079] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0080] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0081] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0082] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0084] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0085] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0086] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A multifunctional transmission line video online monitoring method, characterized in that: include: Determining a scene type based on meteorological monitoring data of the power transmission line, and determining a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; The scene types include ice-covered scenes, dancing scenes and ordinary scenes; Performing image feature extraction on the video frame of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; performing feature extraction on the sensor monitoring data based on the feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; The sensor monitoring feature set and the video monitoring feature set are subjected to multimodal feature fusion to obtain a fused image feature set; and the state information of the transmission line is determined based on the fused image feature set.

2. The multifunctional transmission line video online monitoring method according to claim 1, characterized in that: The environmental monitoring data of the transmission line includes temperature, humidity and wind speed; The determining of the scene type based on the environmental monitoring data of the transmission line includes: If the magnitude relationship between the temperature and the humidity satisfies a first condition, and the wind speed satisfies a second condition, determining that the scene type is an ice-covered scene; If the wind speed satisfies the second condition, and the relationship between the temperature and the humidity does not satisfy the first condition, determining that the scene type is a dancing scene; If the magnitude relationship between the temperature and the humidity does not satisfy the first condition, and the wind speed does not satisfy the second condition, then the scene type is determined to be a normal scene.

3. The multifunctional transmission line video online monitoring method according to claim 1, characterized in that: The sensor monitoring data includes conductor tension, tower inclination angle and conductor vibration signal; Determining a feature weight matrix of sensor monitoring data based on the scene type includes: Based on the scene type, using an attention mechanism to assign feature attention weights to the conductor tension, the tower inclination angle, and the conductor vibration frequency; A feature weight matrix is generated based on the feature attention weight of the conductor tension, the feature attention weight of the tower inclination angle, and the feature attention weight of the conductor vibration frequency.

4. The multifunctional transmission line video online monitoring method according to claim 3, characterized in that: The feature weight matrix based on the sensor monitoring data is used to extract features from the sensor monitoring data to obtain a sensor monitoring feature set, including: calculating tension fluctuation entropy based on the wire tension; Extracting frequency domain characteristics of the wire vibration signal; Calculating the rate of change of the inclination angle based on the inclination angle of the tower; The feature weight matrix is multiplied by the tension fluctuation entropy, the frequency domain feature of the wire vibration signal, and the tilt angle change rate to obtain a weighted sensor monitoring feature set.

5. The multifunctional transmission line video online monitoring method according to claim 1, characterized in that: Determining a video feature extraction strategy based on the scene type includes: If the scene type is the ice-covered scene, the video feature extraction strategy is determined to be a first extraction strategy; the first extraction strategy includes: Performing region segmentation on the video frame of the transmission line, and using the segmented conductor region and insulator string region as target regions; Extracting texture features of the target area using a gray level co-occurrence matrix; Binarizing the target area using a threshold segmentation algorithm to obtain an iced area and an un-iced area, and calculating an iced area ratio based on the iced area and the un-iced area; Edge detection is performed on the insulator string area to obtain edge ice coverage features.

6. The multifunctional transmission line video online monitoring method according to claim 5, characterized in that: Determining a video feature extraction strategy based on the scene type includes: If the scene type is the dancing scene, the video feature extraction strategy is determined to be the second extraction strategy; the second extraction strategy includes: Use deep learning optical flow algorithm to calculate the motion vector of wire pixels between adjacent video frames; establishing a motion trajectory model of the wire based on the motion vector, and determining a dancing pattern of the wire based on the motion trajectory model; The wire swing amplitude feature and the wire swing frequency feature are extracted based on the motion vector.

7. The multifunctional transmission line video online monitoring method according to claim 6, characterized in that: Determining a video feature extraction strategy based on the scene type includes: If the scene type is the common scene, the video feature extraction strategy is determined to be a third extraction strategy; the third extraction strategy includes: Identifying multiple component regions in a video frame using a multi-target detection algorithm, wherein the multiple component regions include an insulator region, a hardware region, a conductor region, and a tower region; Extracting defect features from the plurality of component regions to obtain a defect feature subset corresponding to each component region; For each component region: a feature value time series is established based on the defect feature subset corresponding to the component region.

8. A multifunctional transmission line video online monitoring device, characterized in that: include: A scene recognition module is configured to determine a scene type based on meteorological monitoring data of the transmission line, and to determine a video feature extraction strategy and a feature weight matrix of sensor monitoring data based on the scene type; the scene types include ice-covered scenes, dancing scenes, and ordinary scenes; a feature extraction module configured to extract image features from the video frames of the transmission line based on the video feature extraction strategy to obtain a video monitoring feature set; and to extract features from the sensor monitoring data based on a feature weight matrix of the sensor monitoring data to obtain a sensor monitoring feature set; The state analysis module is used to perform multimodal feature fusion on the sensor monitoring feature set and the video monitoring feature set to obtain a fused image feature set; and determine the state information of the transmission line based on the fused image feature set.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images