Power transmission line hidden danger identification method and device, computer equipment and storage medium

Abstract

The invention relates to a power transmission line hidden danger identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of power transmission line images of a target power transmission line, and inputting each frame of power transmission line image into a line defect identification model; performing feature extraction on the power transmission line image by using a feature extraction module in the line defect identification model to obtain a plurality of target feature images containing different defect features, and generating an anchor frame for each pixel point of each target feature image; performing defect identification based on each target feature image and the anchor frame by using a defect classification module in the line defect identification model to obtain a defect identification result of the power transmission line image; inputting the defect identification result into an alarm analysis model for conjoint analysis to obtain a hidden danger alarm result of the target power transmission line; the hidden danger alarm result is used for indicating whether the target power transmission line has to-be-repaired hidden dangers. The method can accurately and efficiently identify whether the power transmission line has hidden dangers.

Description

Technical Field

[0001] This application relates to the field of power grid technology, and in particular to a method, apparatus, computer equipment, and storage medium for identifying potential hazards in power transmission lines. Background Technology

[0002] With the continuous expansion of my country's power transmission line scale and the ongoing development of new technologies such as communication technology, internet technology, and the Internet of Things, the number of power transmission lines has begun to show an explosive growth trend, and the development of power transmission lines has gradually entered the era of big data. The normal status of power transmission lines plays a crucial role in the operation of the power grid.

[0003] Therefore, the problem of how to accurately and efficiently identify potential hazards in power transmission lines urgently needs to be solved. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, device, computer equipment, and storage medium for identifying potential hazards in power transmission lines, which can accurately and efficiently identify whether there are hidden dangers in power transmission lines, in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for identifying potential hazards in power transmission lines. The method includes:

[0006] Acquire multiple frames of transmission line images of the target transmission line and input each frame of transmission line image into the line defect identification model;

[0007] For each frame of transmission line image, the feature extraction module in the line defect identification model is used to extract features from the transmission line image to obtain multiple target feature images containing different defect features, and an anchor box is generated for each pixel of each target feature image; the defect classification module in the line defect identification model is used to identify defects based on each target feature image and the anchor box to obtain the defect identification result of the transmission line image; the defect identification result includes defect type and defect location.

[0008] The defect identification results of each frame of the transmission line image are input into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line; the hidden danger alarm results are used to indicate whether there are hidden dangers to be repaired in the target transmission line.

[0009] In one embodiment, the feature extraction module includes an initial convolutional layer, a residual module, and a feature fusion module; the feature extraction module in the line defect identification model is used to extract features from the transmission line image to obtain multiple target feature images containing different defect features, including:

[0010] The image of the power transmission line is input into the initial convolutional layer for global basic feature extraction to obtain the first feature image;

[0011] The first feature image is input into the residual module for different types of feature extraction processing to obtain multiple residual feature images;

[0012] Multiple residual feature images are input into the feature fusion module for feature fusion to obtain multiple target feature images containing different defect features.

[0013] In one embodiment, the residual module includes a first residual module, a second residual module, and a third residual module; the first feature image is input into the residual module for different types of feature extraction processing to obtain multiple residual feature images, including:

[0014] The first feature image is input into the first residual module, and the first convolutional layer and the second convolutional layer cascaded in the first residual module are used to extract the bottom edge features to obtain the first residual feature image; the first residual feature image is added to the first feature image to obtain the first candidate residual image, and the first candidate residual image is processed by the activation function to obtain the first residual feature image.

[0015] The first residual feature image is input into the second residual module. The third and fourth convolutional layers cascaded in the second residual module are used to extract the mid-layer shape features to obtain the second residual feature image. The second residual feature image is added to the first residual feature image to obtain the second candidate residual image. The second candidate residual image is then processed using an activation function to obtain the second residual feature image.

[0016] The second residual feature image is input into the third residual module, and the high-level defect features are extracted using the cascaded fifth and sixth convolutional layers in the third residual module to obtain the third residual feature image. The third residual feature image is added to the second residual feature image to obtain the third candidate residual image, and the third candidate residual image is processed using an activation function to obtain the third residual feature image.

[0017] The first residual feature image, the second residual feature image, and the third residual feature image are used as multiple residual feature images.

[0018] In one embodiment, multiple residual feature images are input into a feature fusion module for feature fusion to obtain multiple target feature images containing different defect features, including:

[0019] The third residual feature image is convolved to obtain the first candidate fused image; and the first candidate fused image is processed using an activation function to obtain the top fused feature image.

[0020] The second residual feature image is added to the top-level fused feature image to perform feature fusion, resulting in an intermediate fused feature image;

[0021] The first residual feature image is added to the intermediate fused feature image to perform feature fusion, resulting in a low-level fused feature image;

[0022] Convolution is performed on the top-level fused feature image to obtain a candidate top-level fused feature image, and the candidate top-level fused feature image is processed by an activation function to obtain the first reference fused feature image;

[0023] Convolve the first reference fusion feature image to obtain the second reference fusion feature image;

[0024] The top-level fused feature image, the middle-level fused feature image, the low-level fused feature image, the first reference fused feature image, and the second reference fused feature image are used as multiple target feature images.

[0025] In one embodiment, the defect classification module in the line defect identification model performs defect identification based on each target feature image and anchor frame to obtain the defect identification result of the transmission line image, including:

[0026] For each target feature image, defects are classified according to the target feature image to obtain the defect type corresponding to the target feature image; and the location is predicted according to the anchor box corresponding to the target feature image to obtain the defect location corresponding to the target feature image; and the result confidence score corresponding to the target feature image is generated; the result confidence score is used to characterize the confidence of the accuracy of defect type and defect location recognition corresponding to the target feature image.

[0027] By comparing the confidence levels of each target feature image, the defect type and defect location corresponding to the target feature image with the highest confidence level are output as the defect identification result of the transmission line image.

[0028] In one embodiment, the defect identification results of each frame of the transmission line image are input into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line, including:

[0029] If the defect identification results of multiple consecutive transmission line images exceeding a preset threshold show the same defect type and defect location, then the hidden danger alarm result of the target transmission line can be obtained based on the defect identification results of the multiple transmission line images.

[0030] In one embodiment, the loss function used in the training process of the line defect identification model is the cross-entropy loss function.

[0031] Secondly, this application also provides a device for identifying potential hazards in power transmission lines. The device includes:

[0032] The acquisition module is used to acquire multiple frames of transmission line images of the target transmission line and input each frame of the transmission line image into the line defect identification model;

[0033] The feature module is used to extract features from each frame of the transmission line image using the feature extraction module in the line defect identification model, obtaining multiple target feature images containing different defect features, and generating anchor boxes for each pixel of each target feature image; the defect classification module in the line defect identification model performs defect identification based on each target feature image and the anchor boxes, obtaining the defect identification result of the transmission line image; the defect identification result includes defect type and defect location;

[0034] The alarm module is used to input the defect identification results of each frame of the transmission line image into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line; the hidden danger alarm results are used to indicate whether there are hidden dangers to be repaired in the target transmission line.

[0035] Thirdly, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0036] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0037] The aforementioned method, apparatus, computer equipment, and storage medium for identifying potential hazards in transmission lines acquire multiple frames of images of the target transmission line and input each frame into a line defect identification model. For each frame, the feature extraction module in the line defect identification model extracts features from the transmission line image, obtaining multiple target feature images containing different defect features, and generating anchor boxes for each pixel of each target feature image. The defect classification module in the line defect identification model performs defect identification based on each target feature image and the anchor boxes, obtaining the defect identification result of the transmission line image. The defect identification result includes the defect type and defect location. The defect identification results of each frame are input into an alarm analysis model for joint analysis to obtain the hazard alarm result of the target transmission line. The hazard alarm result is used to indicate whether there are potential hazards to be repaired in the target transmission line. In this way, the line defect identification model can automatically identify the defect identification results of transmission line images, and the joint determination of the hazard alarm result of the target transmission line based on the defect identification results of multiple frames ensures the accuracy and efficiency of the obtained hazard alarm result of the target transmission line. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating a method for identifying potential hazards in transmission lines in one embodiment;

[0040] Figure 2 This is a schematic diagram of the line defect identification model structure in one embodiment;

[0041] Figure 3 This is a schematic diagram of the alarm analysis model flow in one embodiment;

[0042] Figure 4 This is a flowchart illustrating the process of acquiring multiple target feature images in one embodiment;

[0043] Figure 5 This is a schematic diagram of the residual module in one embodiment;

[0044] Figure 6 This is a flowchart illustrating the process of acquiring multiple target feature images in another embodiment;

[0045] Figure 7 This is a structural block diagram of a power transmission line hazard identification device in one embodiment;

[0046] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. It should be understood that many specific details are set forth in the following description in order to provide a full understanding of this application, but this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0049] With the continuous expansion of my country's power transmission line scale and the ongoing development of new technologies such as communication, internet, and the Internet of Things, the quantity and types of power transmission data have begun to show an explosive growth trend, and the development of power transmission lines has gradually entered the era of big data. Massive amounts of different operation and maintenance management data have been generated from data collected by numerous monitoring devices, inspection data, and various monitoring and analysis systems. This data comprehensively reflects the operating environment and status of power transmission lines and has become an extremely valuable resource. However, currently, the data suffers from multi-source heterogeneity, a lack of unified management standards, duplicate storage, inconsistencies, and distribution across different systems. In particular, the operation and maintenance management of power transmission line data is not perfect, data analysis functions are limited and relatively simple, and the massive amounts of power transmission operation and maintenance data cannot be used for collaborative, comprehensive analysis, and decision-making applications.

[0050] Currently, there are significant difficulties in communication and resource sharing among various departments in the operation and maintenance management of power transmission lines. The business functions of each system are limited, data sharing between systems is not smooth, and different data cannot be intersected and analyzed. This makes it impossible to achieve a closed loop of intelligent operation and maintenance for power transmission business, and also prevents the full realization of the intelligent analysis potential of power transmission big data. This limits the support of power transmission data for intelligent and digital power transmission, resulting in the research results of power transmission business not being able to provide intelligent auxiliary decision-making for operation and maintenance management personnel.

[0051] Existing operation and maintenance strategies are not widely applicable to digital transmission lines. At present, the pilot construction of digital transmission lines still uses existing operation and maintenance strategies as guidance, which are not applicable in some cases and fail to highlight the advantages of the transmission lines after digital transformation.

[0052] The adoption of digital power transmission infrastructure is limited, and the challenges of reducing manpower, increasing efficiency, and achieving high-quality development urgently need to be addressed. Currently, only a very small number of lines have completed digital transformation, while the average maintenance workload per frontline personnel remains high, making it impossible to complete power transmission equipment maintenance plans with high quality within differentiated inspection cycles.

[0053] In the aforementioned context, data on potential hazards in power transmission lines is a crucial element, playing a vital role in the normal operation of the power grid. Currently, the algorithms used for intelligent target detection can be broadly categorized into two types: the first is second-order target detection models. These algorithms first extract candidate boxes, then extract image features using convolutional neural networks, and finally perform target classification and regression. The second type is first-order target detection models. These algorithms directly transform target classification and localization into a regression problem without needing to extract candidate regions. Second-order target detection models can fully extract image features, achieving refined classification and localization, and their accuracy is higher than that of first-order models. However, first-order target detection models have simpler network structures, faster detection speeds, and higher real-time performance compared to second-order models. Given the high requirements for real-time detection of potential hazards in power transmission lines, the use of first-order target detection models is more reasonable. However, the dense detectors of first-order target detection models encounter the problem of extreme imbalance between positive and negative sample classes during training, thus affecting the accuracy of target detection.

[0054] In view of this, this application provides a method for identifying potential hazards in power transmission lines, thereby improving the accuracy of hazard detection in power transmission lines.

[0055] It should be noted that the transmission line hazard identification method provided in this application can be implemented by a transmission line hazard identification device, which can be implemented as part or all of a computer device through software, hardware, or a combination of software and hardware. The computer device can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. In the following method embodiments, the implementation subject is always described using a computer device as an example.

[0056] In one embodiment, such as Figure 1 As shown, a method for identifying potential hazards in power transmission lines is provided, the method comprising:

[0057] Step 101: Obtain multiple frames of transmission line images of the target transmission line and input each frame of transmission line image into the line defect identification model.

[0058] The images of the power transmission lines are collected using visual inspection equipment. These multi-frame images can be continuous or discontinuous. The target power transmission line is the one requiring hazard detection. The visual inspection equipment can be a drone or camera equipment set up at fixed monitoring points.

[0059] Optionally, the initial multi-frame transmission line images output by the acquired visualization inspection equipment are preprocessed to obtain the aforementioned multi-frame transmission line images. For example, the preprocessing operations mainly include: scaling the original images to a uniform size to achieve size standardization and adapt to the model input requirements; pixel normalization, mapping pixel values ​​to the [0,1] interval to achieve pixel normalization and reduce the interference of contrast differences, etc.

[0060] In an optional embodiment of this application, each frame of the transmission line image is input into the line defect identification model for identification.

[0061] Step 102: For each frame of transmission line image, the feature extraction module in the line defect identification model is used to extract features from the transmission line image to obtain multiple target feature images containing different defect features, and an anchor box is generated for each pixel of each target feature image; the defect classification module in the line defect identification model is used to identify defects based on each target feature image and the anchor box to obtain the defect identification result of the transmission line image; the defect identification result includes the defect type and defect location.

[0062] A line defect identification model can be constructed using a RetinaNet network structure. Specifically, the line defect identification model mainly includes a feature extraction module and a defect classification module.

[0063] For example, such as Figure 2 As shown, the line defect identification model mainly consists of a feature extraction network (ResNet), a feature pyramid network, and a classification sub-network. The feature extraction network and the feature pyramid network constitute the feature extraction module, and the classification sub-network constitutes the defect classification module.

[0064] Figure 1 The ResNet feature extraction network is responsible for extracting preliminary features of the image; Figure 1 The role of the Feature Pyramid Network (FPN) is to recombine the proposed preliminary image features with high-level semantic features to complete the refined extraction of image features and enrich the receptive field of each feature layer. Figure 1 The subnetworks in the main network perform convolutional object classification and bounding box regression tasks on the output of the main network, respectively.

[0065] The feature extraction network ResNet is responsible for extracting preliminary image features; the feature pyramid network FPN is responsible for recombining the proposed preliminary image features with high-level semantic feature information to complete the refined extraction of image features and enrich the receptive field of each feature layer; the classification subnetwork performs convolutional object classification and bounding box regression tasks on the output of the backbone network, respectively.

[0066] Step 103: Input the defect identification results of each frame of transmission line image into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line.

[0067] The hazard alarm results are used to indicate whether there are any hidden dangers that need to be repaired in the target transmission line.

[0068] Defect types include defects and hidden dangers in the equipment, components, and materials of the transmission line itself, such as hidden dangers in conductors and lightning protection wires, hidden dangers in hardware and connectors, hidden dangers in poles and foundations, etc., which are not fully exemplified here.

[0069] Optionally, if multiple frames of transmission line images have the same defect type and defect location, then the hazard alarm result is determined to be the presence of that type of hazard.

[0070] For example, the defect identification results of each frame of transmission line image are input into the alarm analysis model for joint analysis to obtain the hidden danger alarm result of the target transmission line, including: if the defect identification results of multiple frames of transmission line images with a number greater than a preset threshold represent the same defect type and defect location, then the hidden danger alarm result of the target transmission line is obtained based on the defect identification results of the multiple frames of transmission line images.

[0071] In other words, an alarm analysis model is used to judge the recognition results of multiple frames of images to determine whether a continuous alarm is triggered. If a continuous alarm is triggered and the recognition result is an image without hidden dangers, the image is marked as a suspected missed sample. If a continuous alarm is not triggered and the recognition result is an image with hidden dangers, the image is marked as a suspected missed sample. A continuous alarm is defined as multiple consecutive frames of transmission line images containing defects of the same type and location. Figure 3 The diagram shows the alarm analysis model flow.

[0072] The aforementioned method for identifying potential hazards in transmission lines involves acquiring multiple frames of images of the target transmission line and inputting each frame into a line defect identification model. For each frame, the feature extraction module within the model extracts features to obtain multiple target feature images containing different defect features, and generates anchor boxes for each pixel of each target feature image. The defect classification module in the model then performs defect identification based on the target feature images and anchor boxes, yielding the defect identification result for the transmission line image. This result includes the defect type and location. The defect identification results from each frame are then input into an alarm analysis model for joint analysis to obtain the hazard alarm result for the target transmission line. This alarm result indicates whether there are any potential hazards to be repaired in the target transmission line. This method allows for the automatic identification of defects in transmission line images using the line defect identification model. Furthermore, by jointly determining the hazard alarm result for the target transmission line based on the defect identification results from multiple frames, the accuracy and efficiency of the obtained hazard alarm result are ensured.

[0073] In one embodiment, the feature extraction module includes an initial convolutional layer, a residual module, and a feature fusion module. For example... Figure 4 A flowchart illustrating the acquisition of multiple target feature images is provided. The feature extraction module in the line defect identification model is used to extract features from the transmission line image, resulting in multiple target feature images containing different defect features, including:

[0074] Step 401: Input the transmission line image into the initial convolutional layer to extract global basic features and obtain the first feature image.

[0075] The initial convolutional layer serves as the starting layer for feature extraction, performing global basic feature extraction.

[0076] In an optional embodiment of this application, this layer uses a 7×7 convolutional kernel, 64 convolutional kernels, and a stride of 2. Using a transmission line image (original image or pre-processed image) as input, it achieves downsampling with the image size halved. Through the wide receptive field of the large-size convolutional kernels, it quickly captures global basic features such as the overall route of the line and the outline of the towers, as well as basic texture features such as image brightness distribution and background and line boundaries. This provides structured basic features for subsequent residual block assembly while reducing computational load.

[0077] Step 402: Input the first feature image into the residual module for different types of feature extraction processing to obtain multiple residual feature images.

[0078] The ResNet feature extraction network introduces residual blocks, which work by skipping several layers of data output from earlier layers and incorporating them into the input of later layers. This overcomes the problem of decreasing learning rate and ineffective accuracy improvement caused by increasing network depth. Figure 5 The diagram shows the structure of a residual module.

[0079] In this embodiment, based on the RetinaNet algorithm, ResNet retains only three core residual block groups (removing conv2_x), directly outputting C3, C4, and C5 feature maps through a simplified structure of "7×7 convolutional layer → conv3_x → conv4_x → conv5_x", avoiding redundant computation of high-resolution feature maps. The core introduces a residual block design, which skips several layers of output from previous layers and incorporates them into subsequent inputs, overcoming the problems of reduced learning rate and difficulty in improving accuracy caused by increasing network depth.

[0080] Step 403: Input multiple residual feature images into the feature fusion module for feature fusion to obtain multiple target feature images containing different defect features.

[0081] The feature fusion module consists of a feature pyramid network.

[0082] Based on the characteristics that low-level features have weak semantic information but clear target location, while high-level features have strong semantic information but ambiguous target location, the Feature Pyramid Network (FPN) fuses feature information from different layers through bottom-up connections, top-down connections, and lateral connections.

[0083] Furthermore, the feature fusion module is not entirely consistent with the structure in the aforementioned FPN. In this embodiment, the feature fusion module only utilizes feature maps C3, C4, and C5 to avoid generating anchor boxes in high-resolution feature maps, thereby reducing model detection time. P6 of the feature pyramid is obtained by convolution based on C5, and P7 is obtained by convolution and activation based on P6.

[0084] In this way, multiple target feature images are generated through multiple stages such as top-down propagation, lateral connection, feature pyramid expansion, and feature pyramid output, resulting in a five-level feature pyramid of P3-P7, which can meet the needs of full-scale defect detection.

[0085] In one embodiment, the residual module includes a first residual module, a second residual module, and a third residual module. A first feature image is input into the residual module for different types of feature extraction processing to obtain multiple residual feature images, including: inputting the first feature image into the first residual module, using a cascaded first convolutional layer and a second convolutional layer in the first residual module to extract low-level edge features, obtaining a first residual feature image; adding the first residual feature image to the first feature image to obtain a first candidate residual image, and processing the first candidate residual image using an activation function to obtain the first residual feature image. The first residual feature image is then input into the second residual module, using a cascaded third convolutional layer and a fourth convolutional layer in the second residual module to extract mid-level shape features, obtaining a second residual feature image; adding the second residual feature image to the first residual feature image to obtain a second candidate residual image, and processing the second candidate residual image using an activation function to obtain the second residual feature image. The second residual feature image is input into the third residual module. High-level defect features are extracted using the cascaded fifth and sixth convolutional layers in the third residual module to obtain the third residual feature image. The third residual feature image is then added to the second residual feature image to obtain the third candidate residual image. This third candidate residual image is then processed using an activation function to obtain the final third residual feature image. The first, second, and third residual feature images are used as multiple residual feature images.

[0086] It can be understood that the input of the second convolutional layer is the output of the first convolutional layer; the input of the fourth convolutional layer is the output of the third convolutional layer; and the input of the sixth convolutional layer is the output of the fifth convolutional layer.

[0087] The activation function is, for example, the ReLU activation function.

[0088] For example, the 300×400×64 feature map output by the 7×7 convolutional layer directly enters three core residual modules: the first residual module conv3_x, the second residual module conv4_x, and the third residual module conv5_x. Progressive downsampling is achieved through the convolutional layers within each residual module. Specifically, conv3_x focuses on extracting edge features of small-sized defects such as conductor texture and insulator contours, resulting in the first residual feature image C3. conv4_x focuses on the shape features of medium-sized defects such as tower details and the connection points between conductors and insulators, resulting in the second residual feature image C4. conv5_x extracts high-level features such as defect category differentiation and scene semantics, resulting in the third residual feature image C5.

[0089] The core mechanism of each residual module is the shortcut connection, which is divided into the main path and the shortcut path: the main path obtains refined features through the convolutional structure; when the input and output dimensions or sizes do not match, the shortcut path adjusts the fit through 1×1 convolution, and directly passes the input features when they match; after element-wise addition, the two are activated by ReLU to enhance the response, so as to realize gradient backpropagation through deep networks and avoid gradient vanishing or exploding.

[0090] Optionally, C3 is a 150×200×512 feature map output by conv3_x, with a detail feature retention rate of ≥80% and a positional error of ≤2 pixels, suitable for small-sized defect detection; C4 is a 75×100×1024 feature map output by conv4_x, with a balanced detail and semantic features and a positional error of ≤4 pixels, suitable for medium-sized defect detection; C5 is a 38×50×2048 feature map output by conv5_x, with a semantic feature strength of ≥80%, suitable for large-sized defect detection. The scaling ratios of the three are 1 / 4, 1 / 8, and 1 / 16 of the input image, respectively, maintaining multi-scale progressive characteristics without high-resolution redundancy.

[0091] In one embodiment, such as Figure 6 The flowchart illustrates the process of acquiring multiple target feature images. Multiple residual feature images are input into a feature fusion module for feature fusion to obtain multiple target feature images containing different defect features, including:

[0092] Step 601: Convolve the third residual feature image to obtain the first candidate fused image; and process the first candidate fused image using an activation function to obtain the top-level fused feature image.

[0093] For example, starting with the C5 high-level feature map, the number of channels is first compressed to 256 dimensions through 1×1 convolution to adapt to the fusion dimension, and then the features are refined through 3×3 convolution and ReLU activation to generate the top-level fusion feature image P5.

[0094] Step 602: Add the second residual feature image to the top-level fused feature image to perform feature fusion, and obtain the intermediate fused feature image.

[0095] For example, P5 is upsampled by 2x bilinear interpolation to align its size with C4, while C4 is compressed using a 1×1 convolutional channel. Then, the second residual feature image is added to the top-level fused feature image to complete the semantic and detail feature fusion, resulting in the intermediate fused feature image P4.

[0096] Step 603: Add the first residual feature image to the intermediate fused feature image to perform feature fusion, and obtain the low-level fused feature image.

[0097] For example, following the same logic, P4 is upsampled by 2 times and added to the channel-compressed C3 for feature fusion, finally generating a low-level fused feature image P3.

[0098] In this way, the entire process, through upsampling size alignment and lateral connection feature fusion, allows the defect category judgment information of high-level features to be passed down to the low-level features layer by layer, while preserving the defect localization details of low-level features, thus achieving a dual accurate expression of semantics and location.

[0099] Step 604: Convolve the top-level fused feature image to obtain a candidate top-level fused feature image, and process the candidate top-level fused feature image using an activation function to obtain the first reference fused feature image.

[0100] To cover large and ultra-large defects, a first reference fusion feature image P6 and a second reference fusion feature image P7 are generated based on P5.

[0101] For example, C5 is used as input and generated by convolution with a 3×3 kernel stride of 2 to enhance the response to large-size defects, thus obtaining a candidate top-level fusion feature image; then, ReLU activation is performed on the obtained candidate top-level fusion feature image to enhance effective features and suppress ineffective features, thus obtaining a first reference fusion feature image P6.

[0102] Step 605: Convolve the first reference fused feature image to obtain the second reference fused feature image.

[0103] For example, taking the first reference fusion feature image P6 as input, it is generated by convolution with a 3×3 kernel and a stride of 2, without activation function, and outputs a second reference fusion feature image P7 adapted for the detection of ultra-large defects.

[0104] Step 606: The top-level fused feature image, the intermediate fused feature image, the low-level fused feature image, the first reference fused feature image, and the second reference fused feature image are used as multiple target feature images.

[0105] That is, multiple target feature images include P3 to P7 mentioned above.

[0106] For example, the final generated five-level feature pyramid (P3-P7) all maintain 256 channels to ensure consistent input dimensions for subsequent subnets. Each layer's feature map adapts to defects of different sizes. Specifically, P3 (150×200×256) has a high proportion of detail features and excellent positional accuracy, adapting to small defects; P4 (75×100×256) balances detail and semantics, adapting to medium-sized defects; P5 (38×50×256) has a high proportion of semantic features and a wide coverage, adapting to medium-to-large defects; P6 (19×25×256) has a large receptive field, adapting to large defects; and P7 (10×13×256) possesses a global field of view, adapting to ultra-large defects. The receptive field of each layer increases by a factor of 2, achieving full-scale coverage from 1 / 4 to 1 / 64 of the original size, without high-resolution anchor frame redundancy.

[0107] Thus, in the feature fusion module, the core of this stage is to achieve accurate fusion of high-level semantic features and low-level detailed features, with a concise and clear overall process. Starting with the high-level feature map C5, the number of channels is first compressed to 256 dimensions through a 1×1 convolution to adapt to the fusion dimension. Then, the features are refined through a 3×3 convolution and ReLU activation to generate the top-level fused feature P5. Subsequently, P5 is upsampled by 2x bilinear interpolation to align its size with C4, while C4 is compressed by a 1×1 convolution. The elements of both are added together to complete the fusion of semantic and detailed features, and P4 is generated after smoothing by a 3×3 convolution. Following the same logic, P4 is upsampled by 2x, fused with the channel-compressed C3, and smoothed to finally generate P3. The entire process, through "upsampling size alignment + lateral connection feature fusion," allows the defect category judgment information of high-level features to be passed down layer by layer to the low-level features, while retaining the defect localization details of low-level features, achieving a dual accurate expression of "semantics + location".

[0108] In one embodiment, the defect classification module in the line defect identification model performs defect identification based on each target feature image and anchor frame to obtain the defect identification result of the transmission line image. This includes: classifying defects according to each target feature image to obtain the defect type corresponding to the target feature image; predicting the location of the defect corresponding to the target feature image based on the anchor frame corresponding to the target feature image; generating the result confidence score corresponding to the target feature image; the result confidence score is used to characterize the confidence score of the accuracy of the defect type and defect location identification corresponding to the target feature image; comparing the confidence scores corresponding to each target feature image, and outputting the defect type and defect location corresponding to the target feature image with the highest confidence score as the defect identification result of the transmission line image.

[0109] In this process, anchor boxes are generated and initially screened on the target feature images at layers P3-P7. For example, for each pixel in each feature map layer, anchor boxes of different scales and aspect ratios (1:2, 1:1, 2:1, etc.) are generated to cover various defect morphologies. Optionally, invalid anchor boxes can be filtered using preset conditions such as an IoU threshold of 0.5, retaining those with a certain degree of overlap with the actual labeled boxes, thus reducing subsequent computational load.

[0110] The defect classification module consists of a set of dual-parallel fully convolutional subnetworks (a classification subnetwork and a bounding box regression subnetwork). The target feature images and anchor boxes from layers P3-P7 are input into the dual-parallel fully convolutional subnetworks. The two subnetworks operate in parallel and are trained independently, respectively completing the classification and location correction of defects at all scales. The bounding box regression subnetwork is essentially an FCN network, predicting the offset between the anchor box and its corresponding ground truth (GT) bounding box.

[0111] For example, the internal structures of the two sub-networks are completely identical, both employing a 5-layer deep convolutional design (the first layer is a 256-dimensional convolution, and the subsequent 4 layers are 3×3 convolutions). By sharing parameters to adapt to the feature inputs of different layers P3-P7, deep feature extraction is achieved. The classification sub-network takes the P3-P7 feature maps and corresponding anchor boxes as input, and finally outputs a KA-dimensional vector (K is the number of defect categories, and A is the number of anchor boxes per pixel). After being converted into a probability distribution by the Softmax function, the defect category is determined. The input of the bounding box regression sub-network is completely identical to that of the classification sub-network, and outputs a 4A-dimensional vector, providing a four-dimensional offset (x, y, w, h) for each anchor box, which is combined with the original anchor boxes to obtain the precise location bounding box.

[0112] Furthermore, although the classification and regression subnets have similar structures, their parameters are not shared.

[0113] In this embodiment, the result confidence level corresponding to the target feature image is generated accordingly; by comparing the confidence levels corresponding to each target feature image, the defect type and defect location corresponding to the target feature image with the highest confidence level are output as the defect identification result of the transmission line image.

[0114] In one embodiment, the loss function used in the training process of the line defect identification model is the cross-entropy loss function.

[0115] To address the imbalance between positive and negative samples caused by the dense sampling random mechanism in the first-order object detection model, this embodiment employs the FocalLoss function, an improved cross-entropy loss function. By multiplying the original cross-entropy loss function by an exponential factor that weakens the contribution of easily detectable targets to model training, it successfully reduces the phenomenon that the object detection loss value is easily influenced by a large number of negative samples.

[0116] This loss function adds two factors, γ and αt, to the standard cross-entropy metric. Factor αt suppresses the imbalance between positive and negative samples, while factor γ controls the imbalance between easily distinguishable and difficult-to-distinguish samples. Furthermore, since the optimal values ​​of γ and αt are interdependent, they need to be combined and adjusted to obtain the optimal FocalLoss function when evaluating accuracy.

[0117] In summary, this application's embodiments enhance the feature extraction and hazard identification capabilities of inspection equipment images by constructing a RetinaNet target detection model, achieving real-time, accurate hazard identification and alarming of inspection images. Specifically, it conforms to the native RetinaNet design, eliminating the conv2_x residual block group and retaining only the C3, C4, and C5 feature maps, avoiding anchor box redundancy in high-resolution feature maps, significantly reducing computation and detection time, and aligning with real-time detection targets. ResNet residual connections address deep gradient issues, FPN multi-scale fusion adapts to defects of different sizes, and FocalLoss alleviates sample imbalance, improving accuracy in multiple dimensions. Through an alarm analysis model and a suspicious sample labeling mechanism, false positives and false negatives are effectively filtered out, reducing invalid alarms; it supports various inspection equipment images, with independently designed subnet parameters that can expand defect categories, and alarm rules that can be dynamically adjusted to adapt to different inspection scenarios.

[0118] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0119] Based on the same inventive concept, this application also provides a transmission line hazard identification device for implementing the aforementioned transmission line hazard identification method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the transmission line hazard identification device provided below can be found in the limitations of the transmission line hazard identification method described above, and will not be repeated here.

[0120] In one embodiment, such as Figure 7 As shown, a power transmission line hazard identification device is provided, comprising: an acquisition module, a feature module, and an alarm module, wherein:

[0121] The acquisition module is used to acquire multiple frames of transmission line images of the target transmission line and input each frame of transmission line image into the line defect recognition model;

[0122] The feature module is used to extract features from each frame of transmission line images using the feature extraction module in the line defect identification model, obtaining multiple target feature images containing different defect features, and generating anchor boxes for each pixel of each target feature image; the defect classification module in the line defect identification model performs defect identification based on each target feature image and anchor boxes, and obtains the defect identification result of the transmission line image; the defect identification result includes defect type and defect location;

[0123] The alarm module is used to input the defect identification results of each frame of transmission line image into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line; the hidden danger alarm results are used to indicate whether there are hidden dangers to be repaired in the target transmission line.

[0124] In one embodiment, the feature extraction module includes an initial convolutional layer, a residual module, and a feature fusion module. The feature module is specifically used for: inputting the transmission line image into the initial convolutional layer for global basic feature extraction to obtain a first feature image; inputting the first feature image into the residual module for different types of feature extraction processing to obtain multiple residual feature images; and inputting the multiple residual feature images into the feature fusion module for feature fusion to obtain multiple target feature images containing different defect features.

[0125] In one embodiment, the residual module includes a first residual module, a second residual module, and a third residual module; the feature module is specifically used for: inputting a first feature image into the first residual module, extracting low-level edge features using a cascaded first convolutional layer and a second convolutional layer in the first residual module to obtain a first residual feature image; adding the first residual feature image to the first feature image to obtain a first candidate residual image, and processing the first candidate residual image using an activation function to obtain a first residual feature image; inputting the first residual feature image into the second residual module, extracting mid-level shape features using a cascaded third convolutional layer and a fourth convolutional layer in the second residual module to obtain a second residual feature image. The image is processed as follows: The second residual feature image is added to the first residual feature image to obtain a second candidate residual image, and then processed using an activation function to obtain the second residual feature image. The second residual feature image is input into the third residual module, and high-level defect features are extracted using the cascaded fifth and sixth convolutional layers in the third residual module to obtain the third residual feature image. The third residual feature image is added to the second residual feature image to obtain a third candidate residual image, and then processed using an activation function to obtain the third residual feature image. The first, second, and third residual feature images are used as multiple residual feature images.

[0126] In one embodiment, the feature module is specifically configured to: convolve the third residual feature image to obtain a first candidate fused image; process the first candidate fused image using an activation function to obtain a top-level fused feature image; add the second residual feature image to the top-level fused feature image for feature fusion to obtain an intermediate fused feature image; add the first residual feature image to the intermediate fused feature image for feature fusion to obtain a low-level fused feature image; convolve the top-level fused feature image to obtain a candidate top-level fused feature image; process the candidate top-level fused feature image using an activation function to obtain a first reference fused feature image; convolve the first reference fused feature image to obtain a second reference fused feature image; and use the top-level fused feature image, intermediate fused feature image, low-level fused feature image, first reference fused feature image, and second reference fused feature image as multiple target feature images.

[0127] In one embodiment, the feature module is specifically used for: classifying defects based on each target feature image to obtain the defect type corresponding to the target feature image; predicting the location based on the anchor box corresponding to the target feature image to obtain the defect location corresponding to the target feature image; and generating the result confidence score corresponding to the target feature image; the result confidence score is used to characterize the confidence score of the accuracy of the defect type and defect location identification corresponding to the target feature image; comparing the confidence scores corresponding to each target feature image, and outputting the defect type and defect location corresponding to the target feature image with the highest confidence score as the defect identification result of the transmission line image.

[0128] In one embodiment, the alarm module is specifically used to: if the defect identification results of multiple frames of transmission line images exceeding a preset threshold show that the defect type and defect location are consistent, then obtain the hidden danger alarm result of the target transmission line based on the defect identification results of the multiple frames of transmission line images.

[0129] In one embodiment, the loss function used in the training process of the line defect identification model is the cross-entropy loss function.

[0130] Each module in the aforementioned power transmission line hazard identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0131] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores transmission line hazard identification data. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for identifying transmission line hazards.

[0132] Those skilled in the art will understand that Figure 8The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0133] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0134] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0135] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for identifying potential hazards in power transmission lines, characterized in that, The method includes: Acquire multiple frames of transmission line images of the target transmission line, and input each frame of the transmission line image into the line defect identification model; For each frame of transmission line image, the feature extraction module in the line defect identification model is used to extract features from the transmission line image to obtain multiple target feature images containing different defect features, and an anchor box is generated for each pixel of each target feature image; the defect classification module in the line defect identification model is used to identify defects based on each target feature image and the anchor box to obtain the defect identification result of the transmission line image; the defect identification result includes defect type and defect location; The defect identification results of each frame of the transmission line image are input into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line; the hidden danger alarm results are used to indicate whether there are hidden dangers to be repaired in the target transmission line.

2. The method according to claim 1, characterized in that, The feature extraction module includes an initial convolutional layer, a residual module, and a feature fusion module; the feature extraction module in the line defect identification model is used to extract features from the transmission line image to obtain multiple target feature images containing different defect features, including: The image of the power transmission line is input into the initial convolutional layer for global basic feature extraction to obtain the first feature image; The first feature image is input into the residual module for different types of feature extraction processing to obtain multiple residual feature images; The multiple residual feature images are input into the feature fusion module for feature fusion to obtain multiple target feature images containing different defect features.

3. The method according to claim 2, characterized in that, The residual module includes a first residual module, a second residual module, and a third residual module; the first feature image is input into the residual module for different types of feature extraction processing to obtain multiple residual feature images, including: The first feature image is input into the first residual module, and the first convolutional layer and the second convolutional layer cascaded in the first residual module are used to extract the bottom edge features to obtain the first residual feature image; the first residual feature image is added to the first feature image to obtain the first candidate residual image, and the first candidate residual image is processed by the activation function to obtain the first residual feature image. The first residual feature image is input into the second residual module, and the middle layer shape features are extracted using the cascaded third and fourth convolutional layers in the second residual module to obtain the second residual feature image; the second residual feature image is added to the first residual feature image to obtain the second candidate residual image, and the second candidate residual image is processed using an activation function to obtain the second residual feature image. The second residual feature image is input into the third residual module, and the high-level defect features are extracted using the cascaded fifth and sixth convolutional layers in the third residual module to obtain the third residual feature image; the third residual feature image is added to the second residual feature image to obtain the third candidate residual image, and the third candidate residual image is processed using an activation function to obtain the third residual feature image. The first residual feature image, the second residual feature image, and the third residual feature image are used as the plurality of residual feature images.

4. The method according to claim 3, characterized in that, The step of inputting the multiple residual feature images into the feature fusion module for feature fusion to obtain multiple target feature images containing different defect features includes: The third residual feature image is convolved to obtain the first candidate fused image; and the first candidate fused image is processed using an activation function to obtain the top-level fused feature image. The second residual feature image is added to the top-level fused feature image to perform feature fusion, resulting in an intermediate fused feature image; The first residual feature image is added to the intermediate fused feature image to perform feature fusion, resulting in a low-level fused feature image; The top-level fused feature image is convolved to obtain a candidate top-level fused feature image, and the candidate top-level fused feature image is processed using an activation function to obtain a first reference fused feature image; The first reference fusion feature image is convolved to obtain the second reference fusion feature image; The top-level fused feature image, the intermediate fused feature image, the low-level fused feature image, the first reference fused feature image, and the second reference fused feature image are used as the plurality of target feature images.

5. The method according to any one of claims 1 to 4, characterized in that, The defect classification module in the line defect identification model performs defect identification based on each target feature image and anchor frame to obtain the defect identification result of the transmission line image, including: For each of the target feature images, defect classification is performed based on the target feature images to obtain the defect type corresponding to the target feature image; and position prediction is performed based on the anchor boxes corresponding to the target feature images to obtain the defect position corresponding to the target feature images; and a result confidence score corresponding to the target feature image is generated; the result confidence score is used to characterize the confidence score of the accuracy of defect type and defect position recognition corresponding to the target feature image; By comparing the confidence levels of each target feature image, the defect type and defect location corresponding to the target feature image with the highest confidence level are output as the defect identification result of the transmission line image.

6. The method according to any one of claims 1 to 4, characterized in that, The defect identification results of each frame of the transmission line image are input into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line, including: If multiple frames of transmission line images with a number exceeding a preset threshold consistently exhibit the same defect type and location in their defect identification results, then a hazard alarm result for the target transmission line is obtained based on the defect identification results of the multiple frames of transmission line images.

7. The method according to any one of claims 1 to 4, characterized in that, The loss function used in the training process of the line defect identification model is the cross-entropy loss function.

8. A device for identifying potential hazards in power transmission lines, characterized in that, The device includes: The acquisition module is used to acquire multiple frames of transmission line images of the target transmission line and input each frame of the transmission line image into the line defect identification model; The feature module is used to extract features from each frame of the transmission line image using the feature extraction module in the line defect identification model, obtaining multiple target feature images containing different defect features, and generating anchor boxes for each pixel of each target feature image; the defect classification module in the line defect identification model performs defect identification based on each target feature image and the anchor boxes, obtaining the defect identification result of the transmission line image; the defect identification result includes defect type and defect location; The alarm module is used to input the defect identification results of each frame of the transmission line image into the alarm analysis model for joint analysis to obtain the hidden danger alarm results of the target transmission line; the hidden danger alarm results are used to indicate whether there are hidden dangers to be repaired in the target transmission line.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images