Video monitoring method and device for overhead transmission line

Abstract

The invention provides a video monitoring method and device for an overhead transmission line, and the method comprises the steps: monitoring the overhead transmission line through a high-definition camera, and obtaining a video segment sequence; inputting the video clip sequence into a preset semantic segmentation model, wherein the preset semantic segmentation model comprises a double-branch connected network, a context sensing embedding module, a clustering guide head and a global guide module; extracting a facility segmentation mask and an environment segmentation mask from each frame of image of the video clip sequence through a double-branch connected network and a context sensing embedding module; performing damage feature analysis on the facility segmentation mask through a clustering guide head to generate a damage classification result; and in the global guide module, determining a damage correction result according to the damage classification result and the environment segmentation mask.

Description

Technical Field

[0001] This application relates to the field of video monitoring technology, and in particular to a video monitoring method and device for overhead power transmission lines. Background Technology

[0002] Currently, video-based monitoring methods for overhead transmission lines are often hampered by environmental complexity and the diversity of equipment. Traditional image analysis methods have limited accuracy in extracting line facilities and surrounding environmental areas, making it difficult to accurately distinguish facility details from background noise. This results in a lack of comprehensive utilization of contextual semantic information during the damage identification stage, making damage assessment susceptible to external conditions such as occlusion and changes in lighting. Overall, the reliability and adaptability of the monitoring results remain significantly insufficient. Summary of the Invention

[0003] This application provides a video monitoring method and apparatus for overhead transmission lines, which can improve the automation level and recognition accuracy of condition monitoring of overhead transmission lines.

[0004] In a first aspect, embodiments of this application provide a video monitoring method for overhead transmission lines, the method comprising: A sequence of video clips was obtained by monitoring overhead power transmission lines using high-definition cameras; The video segment sequence is input into a preset semantic segmentation model, which includes: a dual-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module. The facility segmentation mask and environment segmentation mask are extracted from each frame of the video clip sequence using the dual-branch conjoint network and the context-aware embedding module. The clustering guide analyzes the damage characteristics of the facility segmentation mask to generate damage classification results. In the global guidance module, the damage correction result is determined based on the damage classification result and the environment segmentation mask.

[0005] In some embodiments, the step of monitoring overhead transmission lines with a high-definition camera to obtain a sequence of video clips includes: The foreground motion segment is separated from the raw video data captured by the high-definition camera, and the foreground motion segment includes a moving target; The foreground motion segment is subjected to image enhancement processing to obtain an enhanced motion segment; Add timestamps and camera location identifiers to the enhanced motion segments to generate a video segment sequence.

[0006] In some embodiments, extracting facility segmentation masks and environment segmentation masks from each frame of the video clip sequence via the dual-branch conjoint network and the context-aware embedding module includes: Each frame of the video segment sequence is input into the dual-branch conjoint network to generate a first feature map and a second feature map. The context-aware embedding module enhances and fuses the first feature map and the second feature map to obtain a multidimensional feature representation. The multidimensional feature representation is subjected to pyramid semantic parsing to generate a first pre-segmentation mask and a second pre-segmentation mask, respectively. Clustering and allocation optimization are performed on the pixels of the first pre-segmentation mask and the pixels of the second pre-segmentation mask respectively, and the facility segmentation mask and the environment segmentation mask are output.

[0007] In some embodiments, the step of performing damage feature analysis on the facility segmentation mask using the clustering guide to generate damage classification results includes: Pixel feature vectors are extracted from the facility segmentation mask, and the similarity between the pixel feature vectors and preset cluster centers is calculated. Based on the similarity, a damage category probability is assigned to each pixel of the facility segmentation mask to generate a damage probability map; The damage probability map is aggregated to obtain a proposed damage area for the facility component; Calculate the semantic matching result between the proposed damage region and the preset cluster center, and output the damage classification result based on the semantic matching result.

[0008] In some embodiments, determining the damage correction result based on the damage classification result and the environment segmentation mask in the global guidance module includes: Environmental context features are extracted from the environmental segmentation mask, and damage classification weights are calculated based on the environmental context features; The damage classification results are context-aware adjusted using the damage classification weights to generate a damage correction probability map. Based on pyramid semantic parsing, the damage correction probability map is verified across scales to obtain a damage verification probability map. The damage verification probability map is mapped according to the preset cluster centers, and the damage correction result is output.

[0009] In some embodiments, performing pyramid semantic parsing on the multidimensional feature representation to generate a first pre-segmentation mask and a second pre-segmentation mask includes: The multidimensional feature representation is divided into sub-region feature maps according to multiple scales. Global average pooling is performed on each sub-region feature map to generate a sub-region feature vector. The sub-region feature vector is used to represent multiple sub-regions. Based on the semantic similarity between the feature vector of each sub-region and the preset cluster center, a mask label is assigned to each sub-region; Based on the mask labels and the positions of the sub-regions in the sub-region feature map, a first pre-segmentation mask and a second pre-segmentation mask are fused to generate the first pre-segmentation mask and the second pre-segmentation mask.

[0010] In some embodiments, the step of performing clustering allocation optimization on the pixels of the first pre-segmentation mask and the pixels of the second pre-segmentation mask respectively, and outputting a facility segmentation mask and an environment segmentation mask, includes: Cluster all pixel features of the first pre-segmentation mask to obtain a first cluster label group and a first label cluster center; Cluster all pixel features of the second pre-segmentation mask to obtain the second cluster label group and the second label cluster center; Calculate the semantic consistency between the first label cluster center and the second label cluster center; Based on the semantic consistency, the first cluster label group and the second cluster label group are optimized and aligned to obtain the third cluster label group and the fourth cluster label group; The pixel categories are reassigned using the third and fourth clustering label groups, and the facility segmentation mask and environment segmentation mask are output.

[0011] In some embodiments, the first feature map focuses on the spatial structure features of the image, and the second feature map focuses on the color texture features.

[0012] In some embodiments, the step of obtaining the preset cluster centers includes: Collect historical damage image samples, including various known types of facility damage; The historical damaged image samples are clustered unsupervised using the K-means clustering algorithm to obtain preset cluster centers.

[0013] Secondly, embodiments of this application provide a video monitoring device for overhead transmission lines, the video monitoring device for overhead transmission lines comprising: The video capture module is used to monitor overhead power transmission lines using a high-definition camera and obtain video clip sequences. The model invocation module is used to input the video segment sequence into a preset semantic segmentation model, which includes: a dual-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module. A mask generation module is used to extract facility segmentation masks and environment segmentation masks from each frame of the video clip sequence through the dual-branch conjoint network and the context-aware embedding module. The damage analysis module is used to perform damage feature analysis on the facility segmentation mask through the clustering guide head and generate damage classification results; The result output module is used in the global guidance module to determine the damage correction result based on the damage classification result and the environment segmentation mask.

[0014] This application provides a video monitoring method for overhead transmission lines. The method includes: monitoring the overhead transmission line using a high-definition camera to obtain a video segment sequence; inputting the video segment sequence into a preset semantic segmentation model, which includes a dual-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module; extracting facility segmentation masks and environment segmentation masks from each frame of the video segment sequence using the dual-branch conjoined network and the context-aware embedding module; performing damage feature analysis on the facility segmentation masks using the clustering guide head to generate damage classification results; and determining damage correction results based on the damage classification results and the environment segmentation masks in the global guide module. In this method, the introduction of the dual-branch conjoined network and the context-aware embedding module significantly improves the accuracy and robustness of facility and environment mask segmentation. The clustering guide head enhances the discriminative ability of damage features through unsupervised clustering and damage analysis of pixel-level features. The global guide module integrates environmental semantics with preliminary damage classification to achieve context-aware correction of the damage results, thereby comprehensively improving the automation level and recognition accuracy of overhead transmission line condition monitoring. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic flowchart illustrating a video monitoring method for overhead transmission lines provided in this application embodiment; Figure 2 This is a schematic block diagram of a video monitoring device for an overhead transmission line provided in an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described below with reference to the accompanying drawings.

[0018] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0019] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0020] It should be understood that in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0021] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a video monitoring method for overhead transmission lines provided in an embodiment of this application. Figure 1 As shown, the specific steps of the video monitoring method for this overhead transmission line include: S101-S105.

[0022] S101. High-definition cameras are used to monitor overhead power transmission lines to obtain video clip sequences.

[0023] For example, by continuously capturing video of overhead transmission lines, a raw video stream is obtained, and this video stream is preprocessed to generate a sequence of video segments suitable for subsequent analysis. This process includes, but is not limited to, separating foreground motion segments containing significant moving targets from consecutive frames, then performing image enhancement operations on these segments to improve contrast and clarity, and finally adding a corresponding timestamp and camera location identifier to each enhanced motion segment, thereby constructing a sequence of video segments with spatiotemporal context, which serves as the input source for a semantic segmentation model.

[0024] S102. Input the video segment sequence into the preset semantic segmentation model. The preset semantic segmentation model includes: a two-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module.

[0025] For example, the obtained video segment sequence is input frame by frame into a pre-built semantic segmentation model, which integrates a two-branch conjoined network, a context-aware embedding module, a clustering guide, and a global guide module. The two-branch conjoined network is responsible for applying different perspective transformations to the same image to obtain complementary feature representations. The context-aware embedding module uses a self-attention mechanism to fuse multi-level semantic and spatial features. The clustering guide achieves pixel-level semantic grouping based on an unsupervised clustering mechanism, while the global guide module is responsible for integrating image-level contextual information to optimize semantic consistency.

[0026] S103. Extract facility segmentation mask and environment segmentation mask from each frame of the video clip sequence using a dual-branch conjoint network and a context-aware embedding module.

[0027] For example, a dual-branch conjoined network is used to extract features from two viewpoints of the input image, and a context-aware embedding module is used to further enhance the feature representation capability. This module establishes inter-pixel relationships by fusing color, spatial coordinates, and high-level semantic features, thereby outputting a feature map with rich contextual information. Based on this, pseudo-labels are assigned to multi-scale regions using a pyramid semantic parsing strategy, and pixels are grouped and optimized using a clustering algorithm to extract facility segmentation masks representing the transmission line facility area and environment segmentation masks representing the surrounding background environment.

[0028] S104. The damage characteristics of the facility segmentation mask are analyzed by the clustering guide, and the damage classification results are generated.

[0029] For example, damage feature analysis is performed on the extracted facility segmentation mask using a clustering guide. This process first extracts depth feature vectors from the mask pixels and calculates their similarity to predefined cluster centers, and then assigns a damage category probability to each pixel to generate a damage probability map.

[0030] Candidate damage regions are formed by performing region aggregation on the damage probability map. Then, the category is determined based on the semantic matching degree between the candidate damage regions and the cluster centers, and the damage classification result with pixel-level semantic information is output.

[0031] S105. In the global guidance module, the damage correction result is determined based on the damage classification result and the environment segmentation mask.

[0032] For example, in the global guidance module, the initial damage classification result is corrected based on the contextual information contained in the environment segmentation mask. The global guidance module first extracts global scene features from the environment mask and calculates their weight mapping for damage discrimination. Then, it adjusts the damage probability distribution using the weight mapping to generate a corrected probability map. Furthermore, a cross-scale semantic verification mechanism enhances the consistency of the output, resulting in a damage correction result that integrates environmental contextual information, thereby improving the robustness and accuracy of transmission line damage identification.

[0033] This application provides a video monitoring method for overhead transmission lines. The method includes: monitoring the overhead transmission line using a high-definition camera to obtain a video segment sequence; inputting the video segment sequence into a preset semantic segmentation model, which includes a dual-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module; extracting facility segmentation masks and environment segmentation masks from each frame of the video segment sequence using the dual-branch conjoined network and the context-aware embedding module; performing damage feature analysis on the facility segmentation masks using the clustering guide head to generate damage classification results; and determining damage correction results based on the damage classification results and the environment segmentation masks in the global guide module. In this method, the introduction of the dual-branch conjoined network and the context-aware embedding module significantly improves the accuracy and robustness of facility and environment mask segmentation. The clustering guide head enhances the discriminative ability of damage features through unsupervised clustering and damage analysis of pixel-level features. The global guide module integrates environmental semantics with preliminary damage classification to achieve context-aware correction of the damage results, thereby comprehensively improving the automation level and recognition accuracy of overhead transmission line condition monitoring.

[0034] To more clearly illustrate the technical solution of this application, the technical solution of this application will be described below through specific embodiments. It should be noted that the specific embodiments are used to expand the description of the technical solution of this application, and are not intended to limit this application.

[0035] In one embodiment, monitoring an overhead power transmission line using a high-definition camera to obtain a video segment sequence includes: separating a foreground motion segment from the raw video data captured by the high-definition camera, the foreground motion segment including a moving target; performing image enhancement processing on the foreground motion segment to obtain an enhanced motion segment; adding a timestamp and camera location identifier to the enhanced motion segment to generate a video segment sequence.

[0036] For example, a motion detection algorithm is used to separate foreground motion segments containing moving targets from raw video data continuously captured by a high-definition camera. These moving targets may include flocks of birds, floating objects, or swaying lines. Image enhancement processing, such as histogram equalization or adaptive contrast enhancement, is then applied to the foreground motion segments to improve image quality and visual detail, resulting in enhanced motion segments. Furthermore, a timestamp accurate to milliseconds and the geographic coordinates of the camera's installation location are added to each enhanced motion segment to generate a video segment sequence.

[0037] In one embodiment, facility segmentation masks and environment segmentation masks are extracted from each frame of a video segment sequence using a dual-branch conjoined network and a context-aware embedding module. This includes: inputting each frame of the video segment sequence into the dual-branch conjoined network to generate a first feature map and a second feature map; performing feature enhancement and fusion on the first and second feature maps using the context-aware embedding module to obtain a multi-dimensional feature representation; performing pyramid semantic parsing on the multi-dimensional feature representation to generate a first pre-segmentation mask and a second pre-segmentation mask, respectively; and performing clustering allocation optimization on the pixels of the first pre-segmentation mask and the pixels of the second pre-segmentation mask, respectively, to output the facility segmentation mask and the environment segmentation mask.

[0038] For example, each frame of image is input into a dual-branch connected network to generate a first feature map and a second feature map from two different perspectives. The first feature map focuses on the spatial structural features of the image, while the second feature map focuses on color and texture features. By utilizing spatial structural information (such as edges and shapes), the contours and positions of facilities (such as conductors and insulators) are accurately located, overcoming interference from complex backgrounds. Simultaneously, color and texture information effectively characterizes the physical state of the facility's surface (such as rust and dirt) and the materials of the environment.

[0039] A context-aware embedding module performs cross-feature fusion and semantic enhancement on the first and second feature maps. This module utilizes spatial self-attention and color consistency constraints to construct inter-pixel associations, outputting a multi-dimensional feature representation with rich contextual information. Pyramid semantic parsing is then applied to the multi-dimensional feature representation, dividing it into sub-regions at multiple scales and extracting global feature vectors for each. Pseudo-labels are assigned based on their semantic similarity to pre-trained cluster centers, generating a first and second pre-segmentation mask. Clustering assignment optimization is then performed on the pixel features in the first and second pre-segmentation masks, respectively. Pixel grouping optimization is achieved by minimizing intra-class distance and maximizing inter-class difference clustering criteria, outputting a facility segmentation mask and an environment segmentation mask.

[0040] In one embodiment, damage feature analysis of a facility segmentation mask is performed using a clustering guidance head to generate damage classification results. This includes: extracting pixel feature vectors from the facility segmentation mask and calculating the similarity between the pixel feature vectors and preset cluster centers; assigning damage category probabilities to each pixel of the facility segmentation mask based on the similarity to generate a damage probability map; performing region aggregation on the damage probability map to obtain damage region proposals for facility components; calculating the semantic matching results between the damage region proposals and preset cluster centers, and outputting damage classification results based on the semantic matching results.

[0041] For example, high-dimensional pixel feature vectors are extracted pixel-by-pixel from the facility segmentation mask. These feature vectors can capture subtle texture and structural changes. Then, the cosine similarity between these feature vectors and preset cluster centers, derived from unsupervised clustering of historical undamaged and typical damage samples, is calculated. Based on the obtained similarity metric, a softmax function or a related probability allocation mechanism is used to assign a probability value belonging to different damage categories (such as corrosion, deformation, cracks, etc.) to each pixel in the facility segmentation mask, thereby generating a damage probability map with the same resolution as the input, where each pixel location contains multiple damage probability distributions. A region aggregation operation based on connected component analysis or region growing algorithms is performed on this damage probability map to merge spatially adjacent and semantically similar pixel regions, forming candidate damage region proposals. Each region proposal includes attributes such as spatial extent, average probability, and main damage category. Finally, the semantic matching result between the overall features (such as the average feature vector of the region) of each damage region proposal and the preset cluster centers is calculated. The most reliable damage category is determined by maximizing the similarity or minimizing the distance metric, and a structured damage classification result is output accordingly.

[0042] In some embodiments, the step of obtaining the preset cluster centers includes: collecting historical damage image samples, including: various known types of facility damage; and performing unsupervised clustering on the historical damage image samples using the K-means clustering algorithm to obtain the preset cluster centers.

[0043] To construct a highly discriminative damage feature benchmark, the pre-defined cluster centers need to be obtained through offline pre-training. Specifically, historical damage image samples need to be systematically collected, covering various known types of damage to common transmission line facilities (such as insulators, conductors, and fittings), ensuring sufficient data volume and diversity (e.g., different lighting conditions, angles, and severity) for each type of damage. Subsequently, the K-means clustering algorithm is used to perform unsupervised clustering on the historical damage image samples. It is important to note that unsupervised clustering does not directly cluster the original pixels. Instead, a pre-trained feature extraction network (such as a CNN) is first used to map each damage image or its specific region into a high-dimensional feature vector. Then, cosine distance or Euclidean distance is used as a similarity metric, and K-means clustering is performed on all vectors in this high-dimensional feature space. The number of clusters K can be determined based on prior knowledge or the elbow rule. After the algorithm converges, the center point vector of each cluster is used as the pre-defined cluster center.

[0044] In one embodiment, in the global guidance module, determining the damage correction result based on the damage classification result and the environment segmentation mask includes: extracting environmental context features from the environment segmentation mask and calculating damage classification weights based on the environmental context features; using the damage classification weights to perform context-aware adjustment on the damage classification result to generate a damage correction probability map; performing cross-scale verification on the damage correction probability map based on pyramid semantic parsing to obtain a damage verification probability map; mapping the damage verification probability map according to preset cluster centers and outputting the damage correction result.

[0045] For example, various environmental context features are extracted from the environmental segmentation mask, such as background vegetation type, weather condition indicators (e.g., rain, fog, snow), lighting conditions, and spatial occlusion relationships. These features can be used to infer the potential impact of the environment on damage visibility and judgment confidence. Based on the extracted environmental context features, damage classification weights are calculated through a learnable weight network or heuristic rules. These weights reflect the modulating effect of environmental factors on the initial damage classification results. The initial damage classification results are then adjusted using the damage classification weights in a context-aware manner. For example, the confidence of pixels classified as "cracks" in foggy conditions is reduced, or the damage probability of occluded areas is adaptively attenuated, thereby generating a damage correction probability map modulated by environmental factors. Furthermore, this damage correction probability map is validated across scales using a pyramid semantic parsing strategy, i.e., checking the semantic consistency of the damage region at multiple image resolutions to eliminate misjudgments caused by scale changes, resulting in a more consistent damage verification probability map. Finally, based on the semantic category of a preset cluster center, each pixel in the damage verification probability map is mapped to the closest damage category, and the damage correction result is output.

[0046] In one embodiment, pyramid semantic parsing is performed on the multidimensional feature representation to generate a first pre-segmentation mask and a second pre-segmentation mask, respectively. This includes: dividing the multidimensional feature representation into sub-region feature maps according to multiple scales; performing global average pooling on each sub-region feature map to generate a sub-region feature vector, which is used to represent multiple sub-regions; assigning a mask label to each sub-region based on the semantic similarity between each sub-region feature vector and a preset cluster center; and fusing the mask label and the position of the sub-region in the sub-region feature map to generate the first pre-segmentation mask and the second pre-segmentation mask.

[0047] For example, the multidimensional feature representation is divided into several sub-region feature maps according to multiple predefined scales (such as 1 / 4, 1 / 8, and 1 / 16 of the original scale), with each sub-region corresponding to a local region in the original image. Global average pooling is performed on each sub-region feature map to aggregate its spatial information and generate a fixed-length sub-region feature vector, which effectively represents the overall semantic content of the corresponding sub-region. The semantic similarity between each sub-region feature vector and a set of predefined cluster centers (learned from a large amount of data in an unsupervised manner) is calculated, typically using cosine similarity, and the cluster with the highest similarity is selected as the mask label for that sub-region. Based on the assigned mask labels and the spatial location of each sub-region in the original image, label fusion and upsampling operations are used to map the coarse-grained sub-region labels back to the original image resolution, thereby generating a first pre-segmentation mask and a second pre-segmentation mask. These masks provide a preliminary estimate of pixel-level semantic segmentation.

[0048] In one embodiment, clustering allocation optimization is performed on the pixels of the first pre-segmentation mask and the pixels of the second pre-segmentation mask to output facility segmentation mask and environment segmentation mask, respectively. This includes: clustering all pixel features of the first pre-segmentation mask to obtain a first cluster label group and a first label cluster center; clustering all pixel features of the second pre-segmentation mask to obtain a second cluster label group and a second label cluster center; calculating the semantic consistency between the first label cluster center and the second label cluster center; optimizing and aligning the first cluster label group and the second label group according to the semantic consistency to obtain a third cluster label group and a fourth cluster label group; and reallocating pixel categories using the third cluster label group and the fourth cluster label group to output facility segmentation mask and environment segmentation mask.

[0049] For example, all pixel features in the first pre-segmentation mask are re-clustered using a clustering algorithm (such as K-Means or spectral clustering) to optimize intra-cluster consistency and inter-cluster discriminability. This process outputs a first cluster label group (i.e., a new label for each pixel) and a first label cluster center (i.e., the feature center of each class). Similarly, the same clustering operation is performed on all pixel features in the second pre-segmentation mask to obtain a second cluster label group and a second label cluster center. The semantic consistency between the first and second label cluster centers is calculated, for example, by calculating the average cosine distance between all corresponding class centers to measure the consistency of the clustering results of the two branches. Based on this semantic consistency metric, a label alignment algorithm (such as the Hungarian algorithm or optimal transport theory) is used to optimize the alignment of the first and second cluster label groups, aiming to eliminate possible label assignment ambiguities between the two branches, and output a semantically consistent and aligned third and fourth cluster label groups. By using the aligned third and fourth cluster label groups, a unified semantic category is reassigned to all pixels, thereby outputting optimized facility segmentation masks and environment segmentation masks, which have higher spatial consistency and semantic accuracy.

[0050] Please see Figure 2 , Figure 2 This is a schematic block diagram of a video monitoring device for an overhead transmission line provided in an embodiment of this application. The video monitoring device 200 is used to execute the aforementioned video monitoring method for overhead transmission lines. The video monitoring device 200 can be configured in a server.

[0051] The server can be a standalone server, a server cluster, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0052] like Figure 2 As shown, the video monitoring device 200 for overhead transmission lines includes: a video capture module 201, a model calling module 202, a mask generation module 203, a damage analysis module 204, and a result output module 205.

[0053] The video capture module 201 is used to monitor overhead power transmission lines through a high-definition camera and obtain a sequence of video clips.

[0054] The model invocation module 202 is used to input video segment sequences into a preset semantic segmentation model, which includes: a dual-branch conjoined network, a context-aware embedding module, a clustering guide head, and a global guide module.

[0055] The mask generation module 203 is used to extract facility segmentation masks and environment segmentation masks from each frame of the video clip sequence through a dual-branch conjoint network and a context-aware embedding module.

[0056] The damage analysis module 204 is used to perform damage feature analysis on the facility segmentation mask through the clustering guide head and generate damage classification results.

[0057] The result output module 205 is used to determine the damage correction result based on the damage classification result and the environment segmentation mask in the global guidance module.

[0058] This application provides an electronic device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and, when executing the computer program, implement the video monitoring method for overhead transmission lines as described in any of the embodiments of this application.

[0059] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it enables the processor to implement a video monitoring method for overhead transmission lines as described in any of the embodiments of this application.

[0060] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of video monitoring of an overhead power transmission line, characterized in that, The method comprises: monitoring an overhead transmission line through a high-definition camera to obtain a video clip sequence; inputting the video clip sequence into a preset semantic segmentation model, the preset semantic segmentation model comprising: a double-branch connected network, a context-aware embedding module, a clustering guide head, and a global guide module; extracting facility segmentation masks and environment segmentation masks from each frame of the video clip sequence through the double-branch connected network and the context-aware embedding module; performing damage feature analysis on the facility segmentation masks through the clustering guide head to generate a damage classification result; in the global guide module, determining a damage correction result according to the damage classification result and the environment segmentation masks.

2. The video monitoring method of overhead electric transmission lines according to claim 1, characterized in that, The method comprises: separating foreground motion clips from the original video data collected by the high-definition camera, the foreground motion clips comprising motion targets; performing image enhancement processing on the foreground motion clips to obtain enhanced motion clips; adding timestamps and camera position identifiers to the enhanced motion clips to generate a video clip sequence.

3. The video monitoring method of overhead electric transmission lines according to claim 1, characterized in that, The method comprises: inputting each frame of the video clip sequence into the double-branch connected network to generate first feature maps and second feature maps; performing feature enhancement and fusion on the first feature maps and the second feature maps through the context-aware embedding module to obtain multi-dimensional feature representations; performing pyramid semantic analysis on the multi-dimensional feature representations to generate first pre-segmentation masks and second pre-segmentation masks, respectively; performing clustering allocation optimization on the pixels of the first pre-segmentation masks and the pixels of the second pre-segmentation masks, respectively, to output facility segmentation masks and environment segmentation masks.

4. The video monitoring method of overhead electric transmission lines according to claim 1, characterized in that, The method comprises: extracting pixel feature vectors from the facility segmentation masks and calculating the similarity between the pixel feature vectors and preset cluster centers; based on the similarity, assigning damage class probabilities to each pixel of the facility segmentation masks to generate a damage probability map; performing regional aggregation on the damage probability map to obtain damage region proposals of facility components; calculating semantic matching results of the damage region proposals and the preset cluster centers, and outputting a damage classification result according to the semantic matching results.

5. The method of video monitoring of an overhead power line according to claim 4, characterized in that, The method comprises: extracting environment context features from the environment segmentation masks and calculating damage classification weights according to the environment context features; performing context-aware adjustment on the damage classification result using the damage classification weights to generate a damage correction probability map; performing cross-scale verification on the damage correction probability map based on pyramid semantic analysis to obtain a damage verification probability map; mapping the damage verification probability map according to the preset cluster centers to output a damage correction result.

6. The method of video monitoring of an overhead power line according to claim 2, wherein, The pyramid semantic parsing on the multi-dimensional feature representation generates a first pre-segmentation mask and a second pre-segmentation mask respectively, including: According to multiple scales, the multi-dimensional feature representation is divided into sub-region feature maps, and global average pooling is performed on each sub-region feature map to generate a sub-region feature vector, which is used to represent multiple sub-regions; According to the semantic similarity of each sub-region feature vector and a preset cluster center, a mask label is assigned to each sub-region; Based on the mask label and the position of the sub-region in the sub-region feature map, a first pre-segmentation mask and a second pre-segmentation mask are fused.

7. The method of video monitoring of an overhead power line according to claim 2, wherein, The clustering and optimization of pixels of the first pre-segmentation mask and pixels of the second pre-segmentation mask respectively output a facility segmentation mask and an environment segmentation mask, including: All pixel features of the first pre-segmentation mask are clustered to obtain a first cluster label group and a first label cluster center; All pixel features of the second pre-segmentation mask are clustered to obtain a second cluster label group and a second label cluster center; The semantic consistency between the first label cluster center and the second label cluster center is calculated; According to the semantic consistency, the first cluster label group and the first cluster label group are optimized and aligned to obtain a third cluster label group and a fourth cluster label group; The third cluster label group and the fourth cluster label group are used to reassign pixel categories to output a facility segmentation mask and an environment segmentation mask.

8. The method of video monitoring of an overhead power line according to claim 3, wherein, The first feature map focuses on the spatial structure features of the image, and the second feature map focuses on the color texture features.

9. The method for video monitoring of overhead electric transmission lines according to claim 4 or 5, characterized in that, The obtaining step of the preset cluster center includes: Collecting historical damage image samples, including: multiple known types of facility damage; Using a K-means clustering algorithm to perform unsupervised clustering on the historical damage image samples to obtain a preset cluster center.

10. A video monitoring device for an overhead power line, characterized in that The overhead transmission line video monitoring device is used to perform the overhead transmission line video monitoring method as claimed in any one of claims 1 to 9, and the overhead transmission line video monitoring device includes: A video shooting module for monitoring the overhead transmission line through a high-definition camera to obtain a video segment sequence; A model calling module for inputting the video segment sequence into a preset semantic segmentation model, and the preset semantic segmentation model includes: a double-branch connected network, a context-aware embedding module, a cluster-guided head, and a global guidance module; A mask generation module for extracting a facility segmentation mask and an environment segmentation mask from each frame of image of the video segment sequence through the double-branch connected network and the context-aware embedding module; A damage analysis module for performing damage feature analysis on the facility segmentation mask through the cluster-guided head to generate a damage classification result; A result output module for determining a damage correction result according to the damage classification result and the environment segmentation mask in the global guidance module.

Images