Heating defect detection method, device and equipment of power transmission line and storage medium

Abstract

The invention provides a heating defect detection method, device and equipment for a power transmission line and a storage medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining an infrared image and a visible light image of the power transmission line; extracting the features of the infrared image and the visible light image to obtain the features of the infrared image and the features of the visible light image; performing high-heat target detection on the infrared image to obtain a high-heat target significance weight; calculating an infrared image adaptive weight and a visible light image adaptive weight according to the high-heat target saliency weight; according to the infrared image adaptive weight and the visible light image adaptive weight, performing weighted fusion on the infrared image features and the visible light image features to obtain a fused feature image; and performing heating defect target detection on the fused feature image by using a deformation attention detection model to obtain a defect target detection image. According to the invention, the robustness of target detection can be improved, and the accuracy of power transmission line heating defect detection is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and particularly to a method, device, equipment and storage medium for detecting heating defects of transmission lines. Background Art

[0002] In the power system, as a key link for power transmission, the stability and safety of transmission lines are of crucial importance. However, due to the long-term exposure of power equipment to complex and changeable outdoor environments, such as extreme conditions like high voltage, high temperature and high load, the defects that may exist inside these equipment often deteriorate gradually over time and may eventually evolve into faults, posing a serious threat to the normal operation of the power system. With the rapid development of unmanned aerial vehicle (UAV) technology, its applications in various fields are becoming increasingly widespread, especially showing great potential in the inspection of transmission lines. With its flexible, mobile, efficient and fast characteristics, the UAV can easily reach areas that are difficult to access by traditional inspection methods. By taking high-definition images and transmitting them back in real time, it provides unprecedented convenience for power inspection work.

[0003] During the UAV inspection process, the acquired image information mainly includes two types: visible light images and infrared images. Visible light images can intuitively display the appearance of power equipment, such as obvious defects like corroded fittings and foreign objects in bird nests; while infrared images can reflect the thermal distribution of the equipment and have unique advantages in detecting potential faults such as abnormal heating. However, a single type of image often only provides limited information and is difficult to comprehensively reflect the true condition of the equipment. Especially in complex scenarios, the current methods for detecting transmission line defects have the problem of low accuracy in target recognition. Summary of the Invention

[0004] In view of the above deficiencies of the prior art, the present invention provides a method, device, equipment and storage medium for detecting heating defects of transmission lines, effectively solving the problem of low accuracy in target recognition existing in the current methods for detecting transmission line defects.

[0005] In a first aspect, the present invention provides a method for detecting heating defects of transmission lines, the method comprising: Obtaining an infrared image and a visible light image of a transmission line; Extracting features of the infrared image and the visible light image to obtain infrared image features and visible light image features; Performing high-temperature target detection on the infrared image to obtain a high-temperature target saliency weight; Calculating fusion weights of the infrared image features and the visible light image features according to the high-temperature target saliency weight to obtain an infrared image adaptive weight and a visible light image adaptive weight; According to the infrared image adaptive weight and the visible light image adaptive weight, perform weighted fusion on the infrared image features and the visible light image features to obtain a fused feature image; Use a deformable attention detection model to perform thermal defect target detection on the fused feature image to obtain a defect target detection image.

[0006] Further, the extracting the features of the infrared image and the visible light image to obtain infrared image features and visible light image features includes: Construct a convolutional feature extraction network based on a residual network model and a feature pyramid network; Use the convolutional feature extraction network to extract the features of the infrared image to obtain the infrared image features; Use the convolutional feature extraction network to extract the features of the visible light image to obtain the visible light image features.

[0007] Further, the performing high-temperature target detection on the infrared image to obtain a high-temperature target saliency weight includes: Obtain the temperature information of each coordinate point of the infrared image and the average temperature information of the whole image; Perform calculations based on the temperature information and the average temperature information of the whole image to obtain the local temperature difference of the infrared image; Calculate a temperature difference saliency index based on the local temperature difference; Extract the high-temperature target area according to the temperature difference saliency index, and calculate the high-temperature target saliency weight of the high-temperature target area.

[0008] Further, the calculating the fusion weights of the infrared image features and the visible light image features according to the high-temperature target saliency weight to obtain an infrared image adaptive weight and a visible light image adaptive weight includes: Calculate the fusion weight of the infrared image features according to the high-temperature target saliency weight to obtain the infrared image adaptive weight; Calculate the fusion weight of the visible light image features according to the infrared image adaptive weight to obtain the visible light image adaptive weight.

[0009] Further, the using a deformable attention detection model to perform thermal defect target detection on the fused feature image to obtain a defect target detection image includes: Construct an initial deformable attention detection model; Train the initial deformable attention detection model to obtain the deformable attention detection model; Input the fused feature image into the deformable attention detection model for thermal defect target detection to obtain the defect target detection image.

[0010] Further, training the initial deformation attention detection model to obtain the deformation attention detection model includes: Obtain an image dataset and perform image preprocessing to obtain an initial dataset; Perform forward propagation on the initial deformation attention detection model using the initial dataset; Set the loss function of the initial deformation attention detection model; Set the initial weights, optimizer, and learning rate to train the initial deformation attention detection model to obtain the deformation attention detection model.

[0011] Further, the deformation attention detection model includes a backbone network, a deformation attention unit, and a detection head unit. Inputting the fused feature image into the deformation attention detection model for heat defect target detection to obtain the defect target detection image includes: Extract the features of the fused feature image through the backbone network to obtain a feature image; Dynamically select the neighborhood of the defect query points in the feature image through the deformation attention unit for calculation; Output the object category corresponding to the defect query point through the detection head unit, and predict the bounding box of the object corresponding to the defect query point.

[0012] In a second aspect, the present invention provides a heat defect detection device for a transmission line, and the device includes: An image acquisition module for acquiring infrared images and visible light images of the transmission line; A feature extraction module for extracting the features of the infrared image and the visible light image to obtain infrared image features and visible light image features; A weight detection module for performing high-temperature target detection on the infrared image to obtain a high-temperature target saliency weight; A weight calculation module for calculating the fusion weights of the infrared image features and the visible light image features according to the high-temperature target saliency weight to obtain an infrared image adaptive weight and a visible light image adaptive weight; A feature fusion module for performing weighted fusion on the infrared image features and the visible light image features according to the infrared image adaptive weight and the visible light image adaptive weight to obtain a fused feature image; A defect detection module for using a deformation attention detection model to perform heat defect target detection on the fused feature image to obtain a defect target detection image.

[0013] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the method for detecting heat defects in a power transmission line as described in the first aspect of the present invention.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method for detecting heat defects in a power transmission line as described in the first aspect of the present invention.

[0015] The method, device, equipment, and storage medium for detecting heat defects in a power transmission line provided by the present invention perform image fusion by adaptively and dynamically adjusting the weights of infrared images and visible light images, so that the resulting image combines the comprehensive features of the two images. The visible light image provides high-resolution texture and shape information, and the infrared image provides the thermal features of the target in low-light or occluded environments, which can enhance the robustness of target detection and greatly improve the accuracy of defect detection. In an environment with good lighting conditions, the visible light image can provide color and details, and the infrared image provides temperature difference information. The combination of the two helps to distinguish targets with similar appearances and can improve the target discrimination ability. In a complex and variable environment, multi-modal fusion can significantly enhance the environmental perception ability of the unmanned aerial vehicle, greatly expanding the application scenarios of unmanned aerial vehicle inspection. The deformable attention detection model focuses on local regions rather than global regions in the image, thereby enhancing the detection ability for small targets and improving the training efficiency. Due to its strong global information modeling ability and flexible attention mechanism, it can improve the accuracy of target recognition in complex scenarios and is applicable to the task of identifying heat defects in power transmission line fittings. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 It is the first schematic diagram of the flow of the method for detecting heat defects in a power transmission line provided by an embodiment of the present invention; Figure 2 It is the second schematic diagram of the flow of the method for detecting heat defects in a power transmission line provided by an embodiment of the present invention; Figure 3 It is the third schematic diagram of the flow of the method for detecting heat defects in a power transmission line provided by an embodiment of the present invention; Figure 4 It is the fourth schematic diagram of the flow of the method for detecting heat defects in a power transmission line provided by an embodiment of the present invention; Figure 5 It is the fifth schematic diagram of the flow of the method for detecting heat defects in a transmission line provided by an embodiment of the present invention; Figure 6 It is the sixth schematic diagram of the flow of the method for detecting heat defects in a transmission line provided by an embodiment of the present invention; Figure 7 It is the seventh schematic diagram of the flow of the method for detecting heat defects in a transmission line provided by an embodiment of the present invention; Figure 8 It is a schematic structural diagram of a device for detecting heat defects in a transmission line provided by an embodiment of the present invention; Figure 9 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.

[0018] Main element symbol description: 800, device for detecting heat defects in a transmission line; 810, image acquisition module; 820, feature extraction module; 830, weight detection module; 840, weight calculation module; 850, feature fusion module; 860, defect detection module; 900, electronic device; 910, processor; 920, communication interface; 930, memory; 940, communication bus. Specific embodiments

[0019] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be further described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0020] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality" means two or more, unless otherwise specifically defined.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0022] With the rapid development of drone technology, it has shown great potential in the inspection of transmission lines. Drones, with their flexible and mobile, efficient and fast characteristics, can easily reach areas that are difficult to access by traditional inspection methods. By taking high-definition images and transmitting them in real time, they provide unprecedented convenience for power inspection work. During the drone inspection process, the image information obtained mainly includes two types: visible light images and infrared images. Visible light images can intuitively display the appearance of power equipment, such as obvious defects like corrosion of fittings and foreign objects in bird nests; while infrared images can reflect the thermal distribution of equipment and have unique advantages in detecting potential faults such as abnormal heating. However, a single type of image often only provides limited information and is difficult to comprehensively reflect the true condition of the equipment. Especially in complex scenarios, the current transmission line defect detection methods have the problem of low accuracy in target recognition.

[0023] Embodiment 1 The embodiment of the present invention provides a method for detecting heating defects in transmission lines, effectively solving the problem of low accuracy in target recognition existing in the current transmission line defect detection methods. Figure 1 It is the first schematic diagram of the flow of the method for detecting heating defects in transmission lines provided by the embodiment of the present invention, as Figure 1 shown, the method includes the following steps: S100. Obtain infrared images and visible light images of the transmission line.

[0024] In the embodiment of the present invention, infrared images and visible light images are taken by a drone and the images are transmitted back for the detection and identification of heating defects in transmission lines. Infrared images have strong environmental adaptability and can work in lightless or low-light environments, especially suitable for monitoring at night or in bad weather conditions. At the same time, infrared images have strong penetrability, can penetrate slight obstacles, have relatively good detection ability for the occluded targets, and can perform temperature difference detection to accurately locate the heating points in the photographed objects. In the drone power transmission inspection task, by taking infrared images of the transmission line with a drone, it can provide object features with infrared information, facilitating the discovery of defects such as fitting heating and insulator heating existing in the transmission line.

[0025] Visible light images have high spatial resolution and can capture rich detail information, such as information on color, texture, and shape, etc., and are suitable for target detection and classification in daily lighting environments. At the same time, they have strong intuitiveness, and the captured images are consistent with the human visual system, facilitating direct understanding and analysis. In the drone inspection of transmission lines, infrared images can accurately capture transmission line information and fitting feature information for subsequent identification and analysis tasks, such as fitting defect identification, line conductor disconnection defect identification, insulator component rupture defect identification, etc., thus providing rich image details for target detection.

[0026] S200. Extract the features of the infrared image and the visible light image to obtain the infrared image features and the visible light image features.

[0027] Figure 2 It is the second schematic diagram of the process of the heating defect detection method for transmission lines provided by the embodiments of the present invention. As Figure 2 shown, the feature extraction specifically includes the following steps: S210. Construct a convolutional feature extraction network based on the residual network model and the feature pyramid network.

[0028] In the embodiments of the present invention, a convolutional feature extraction network is constructed according to the ResNet101 model of the residual network model and the FPN network of the feature pyramid network. The ResNet101 model adopts a residual learning framework to alleviate the problems of gradient disappearance and gradient explosion during the training process of the deep neural network. The network of the model is divided into multiple convolutional layer groups, and each group contains a certain number of convolutional layers and residual connections. This network structure enables the ResNet101 model to extract deep features in the image. As the network deepens, the extracted features gradually transition from low-level detail information such as edges and textures to high-level semantic information.

[0029] The FPN network combines high-level semantic information with low-level detail information by establishing connections between different levels, thereby generating feature maps with rich multi-scale information. The FPN network includes a bottom-up path and a top-down path. The bottom-up path is the forward propagation process of the convolutional network, which captures low-level detail information. The top-down path then transfers high-level semantic information to the low level through upsampling and lateral connections, thereby enhancing the information exchange between the pyramid levels of the features.

[0030] The convolutional feature extraction network in the embodiments of the present invention combines the ResNet101 model and the FPN network to construct a powerful feature extraction network, which utilizes both the deep feature extraction ability of the ResNet101 model and the multi-scale feature fusion ability of the FPN network. The ResNet101 model serves as the backbone network for feature extraction, responsible for extracting deep features of the image, and the FPN network fuses these features between different levels to generate feature maps with multi-scale information.

[0031] S220. Use the convolutional feature extraction network to extract the features of the infrared image to obtain the infrared image features.

[0032] In the embodiments of the present invention, the convolutional feature extraction network is used to extract features from the infrared image to obtain the infrared image features. The formula for feature extraction is as follows:

[0033] In the above formula,F infrared represents the infrared image features obtained through the convolutional feature extraction network conv_layers represents the convolutional feature extraction network I infrared represents the input infrared image

[0034] S230. Use the convolutional feature extraction network to extract the features of the visible light image to obtain the visible light image features

[0035] In the embodiment of the present invention, the convolutional feature extraction network is used to extract features from the visible light image to obtain the visible light image features. The formula for feature extraction is as follows

[0036] In the above formula F visible represents the visible light image features obtained through the convolutional feature extraction network I visible represents the input visible light image

[0037] S300. Perform high-temperature target detection on the infrared image to obtain the high-temperature target saliency weight

[0038] In the embodiment of the present invention, high-temperature targets are detected based on the temperature difference characteristics of the infrared image Figure 3 is the third schematic diagram of the flow of the method for detecting heat defects in transmission lines provided by the embodiment of the present invention. As Figure 3 shown, the detection of high-temperature targets specifically includes the following steps S310. Obtain the temperature information of each coordinate point of the infrared image and the average temperature information of the whole image

[0039] Optionally, the software development kit (SDK) provided by the official drone can be used to obtain the temperature of the specified point coordinates and the average temperature of the specified area, so as to obtain the temperature information of each coordinate point of the entire infrared image and the average temperature information of the whole image

[0040] S320. Calculate according to the temperature information and the average temperature information of the whole image to obtain the local temperature difference of the infrared image

[0041] In the embodiment of the present invention, an absolute value difference calculation is performed according to the temperature information and the average temperature information of the whole image to obtain the local temperature difference of the infrared image. The calculation formula is as follows

[0042] In the above formula, ∇ T represents the calculated local temperature difference Mean represents the averaging operation, which is used to estimate the global average temperature

[0043] S330. Calculate the temperature difference significance index based on the local temperature difference.

[0044] In the embodiment of the present invention, the formula for calculating the temperature difference significance index is as follows:

[0045] In the above formula, S map ( x, y ) represents the obtained significance index, represents the temperature difference threshold, and can take the lowest temperature among the detectable high-temperature targets.

[0046] S340. Extract the high-temperature target area according to the temperature difference significance index, and calculate the high-temperature target significance weight of the high-temperature target area.

[0047] In the embodiment of the present invention, use the temperature difference significance index S map ( x, y ) to extract the high-temperature target area and obtain an infrared target mask. The generation formula of the infrared target mask is as follows:

[0048] In the above formula, M infrared represents the generated infrared target mask.

[0049] Calculate the high-temperature target significance weight of the high-temperature target area according to the infrared target mask. The calculation formula is as follows:

[0050] In the above formula, represents the calculated high-temperature target significance weight.

[0051] S400. Calculate the fusion weights of the infrared image features and the visible light image features according to the high-temperature target significance weight, and obtain the infrared image adaptive weight and the visible light image adaptive weight.

[0052] Figure 4 is the fourth schematic diagram of the process of the method for detecting the heating defect of the transmission line provided by the embodiment of the present invention. As Figure 4 shown, calculating the adaptive weight specifically includes the following steps: S410. Calculate the fusion weight of the infrared image features according to the high-temperature target significance weight, and obtain the infrared image adaptive weight.

[0053] In the embodiment of the present invention, the calculation formula of the infrared image adaptive weight is as follows:

[0054] In the above formula, ω infrared represents the adaptive weight of the infrared image, Sigmoid represents the sigmoid function, represents an adjustable weight ratio coefficient.

[0055] S420. Calculate the fusion weight of the visible light image features according to the adaptive weight of the infrared image to obtain the adaptive weight of the visible light image.

[0056] In the embodiment of the present invention, the calculation formula of the adaptive weight of the visible light image is as follows:

[0057] In the above formula, ω visible represents the adaptive weight of the infrared image.

[0058] S500. Perform weighted fusion on the infrared image features and the visible light image features according to the adaptive weight of the infrared image and the adaptive weight of the visible light image to obtain a fused feature image.

[0059] In the embodiment of the present invention, the adaptive weight of the infrared image and the adaptive weight of the visible light image calculated dynamically are used to perform weighted fusion on the infrared image features and the visible light image features to generate a fused feature image with both details and thermal information. The calculation formula of the weighted fusion is as follows:

[0060] In the above formula, F fused represents the fused feature image.

[0061] S600. Use the deformed attention detection model to detect the heating defect target in the fused feature image to obtain a defect target detection image.

[0062] Figure 5 is the fifth schematic diagram of the process of the heating defect detection method for the transmission line provided by the embodiment of the present invention. As Figure 5 shown, the specific steps of the heating defect target detection include the following: S610. Construct an initial deformed attention detection model.

[0063] In the embodiment of the present invention, the initial deformable attention detection model adopts the Deformable DETR model, which introduces the deformable attention mechanism. The deformable attention restricts the global attention mechanism to local regions, thereby reducing the computational overhead and enhancing the detection ability for small targets. The deformable attention only focuses on the local features around certain key points, avoiding the traversal calculation of the entire feature map. At the same time, by integrating feature maps of different scales and selectively focusing on features of different scales, the Deformable DETR model can better capture the details of large and small objects in the image, improving the detection ability for small targets.

[0064] Optionally, the Deformable DETR model includes a backbone network, a deformable attention unit, and a detection head unit. The backbone network uses the deep convolutional network ResNet to extract the features of the input image. Through the convolutional layer network, feature maps from low-level to high-level will be gradually extracted for subsequent object detection tasks. The deformable attention unit selectively focuses on the feature points near the target region. By dynamically selecting the neighborhoods of the feature points in the image for calculation, the computational complexity is reduced. At the same time, the deformable attention unit can dynamically select the important positions in the input feature map and allow these positions to deform according to the requirements of specific tasks, so as to capture targets of different shapes and scales. The detection head unit includes a classification head and a regression head.

[0065] S620. Train the initial deformable attention detection model to obtain the deformable attention detection model.

[0066] Figure 6 This is the sixth schematic diagram of the flow of the method for detecting the heating defect of the transmission line provided by the embodiment of the present invention. As Figure 6 shown, the training of the deformable attention detection model specifically includes the following steps: S621. Obtain an image dataset and perform image preprocessing to obtain an initial dataset.

[0067] Obtain an image dataset and perform image preprocessing. The image preprocessing includes, but is not limited to, operations such as image scaling, cropping, and normalization. At the same time, generate corresponding target category labels and bounding boxes for each image to obtain the initial dataset, where these labels and bounding boxes will be used to supervise the training process.

[0068] S622. Perform forward propagation on the initial deformable attention detection model using the initial dataset.

[0069] Optionally, extract the feature map of the input image in the initial dataset through the backbone network, and use the deformable attention unit to process the feature map, selectively focusing on the key regions. Then classify and regress the feature vectors generated for each query to predict the category and bounding box of the target.

[0070] S623. Set the loss function of the initial deformable attention detection model.

[0071] In the embodiments of the present invention, a multi-task loss function is used to optimize the classification and regression tasks simultaneously. The loss function includes, but is not limited to, a classification loss function, a regression loss function, and a matching loss function. The classification loss can adopt a cross-entropy loss function to optimize the prediction of object categories. The regression loss function can use a smooth L1 loss to optimize the target box regression to ensure that the predicted box is close to the true box. The matching loss function is used to handle the matching between multiple queries and targets. The Hungarian algorithm can be used to calculate the matching between the query and the true target, and the matching loss is used for optimization.

[0072] S624. Set the initial weights, optimizer, and learning rate to train the initial deformable attention detection model to obtain the deformable attention detection model.

[0073] In the embodiments of the present invention, the initial weights include setting the weights of each layer of the network. The optimizer can use an Adam optimizer or an AdamW optimizer. At the same time, a learning rate scheduling strategy such as cosine annealing can be adopted to adjust the learning rate to better converge during the training process.

[0074] S630. Input the fused feature image into the deformable attention detection model for detecting the heating defect target to obtain the defect target detection image.

[0075] Figure 7 It is the seventh schematic diagram of the flow of the heating defect detection method for the transmission line provided by the embodiments of the present invention. As Figure 7 shown, the obtaining of the defect target detection image specifically includes the following steps: S631. Extract the features of the fused feature image through the backbone network to obtain the feature image.

[0076] The backbone network uses a deep convolutional network ResNet to extract the features of the fused feature image, and the convolutional layer network will gradually extract the features from low-level to high-level feature images.

[0077] S632. Dynamically select the neighborhood of the defect query points in the feature image through the deformable attention unit for calculation.

[0078] In the embodiments of the present invention, the deformable attention can be implemented through the following steps: First, input the query to generate the query vector of the target, and the query vector can be a matrix with a fixed size. Then select the key points, select the most important key point regions from the feature image for calculation, and allow for more flexible positioning of the region of interest. Finally, calculate the attention, perform the attention mechanism on the selected key points instead of performing global calculation on the entire image, thereby improving the efficiency.

[0079] S633. Output the object category corresponding to the defect query point through the detection head unit, and predict the bounding box of the object corresponding to the defect query point.

[0080] The detection head unit includes a classification head and a regression head. The classification head is used to output the object category corresponding to each query point, and the regression head is used to predict the bounding box of the object corresponding to each query point. The output query vector matches the instance in the thermal defect target detection, and is used to generate the category and location of each thermal defect target, obtaining the defect target detection image.

[0081] The method for detecting thermal defects of transmission lines provided by the embodiments of the present invention performs image fusion by adaptively and dynamically adjusting the weights of infrared images and visible light images, so that the resulting image incorporates the comprehensive features of both images. The visible light image provides high-resolution texture and shape information, and the infrared image provides the thermal features of the target in low-light or occluded environments, which can enhance the robustness of target detection and greatly improve the accuracy of defect detection. The deformable attention detection model focuses on local regions in the image rather than the global region, thereby enhancing the detection ability for small targets and improving the training efficiency. Due to its strong global information modeling ability and flexible attention mechanism, it can improve the accuracy of target recognition in complex scenarios and is applicable to the task of identifying thermal defects of transmission line fittings.

[0082] Embodiment 2 Based on the same technical concept as in Method Embodiment 1 above, the embodiments of the present invention provide a device for detecting thermal defects of transmission lines. Figure 8 It is a schematic structural diagram of the device for detecting thermal defects of transmission lines provided by the embodiments of the present invention. The device 800 for detecting thermal defects of transmission lines includes: An image acquisition module 810, configured to acquire infrared images and visible light images of the transmission line; A feature extraction module 820, configured to extract the features of the infrared image and the visible light image, obtaining infrared image features and visible light image features; A weight detection module 830, configured to perform high-temperature target detection on the infrared image to obtain the significance weight of the high-temperature target; A weight calculation module 840, configured to calculate the fusion weights of the infrared image features and the visible light image features according to the significance weight of the high-temperature target, obtaining the adaptive weight of the infrared image and the adaptive weight of the visible light image; A feature fusion module 850, configured to perform weighted fusion on the infrared image features and the visible light image features according to the adaptive weight of the infrared image and the adaptive weight of the visible light image, obtaining a fused feature image; A defect detection module 860, configured to use the deformable attention detection model to perform thermal defect target detection on the fused feature image, obtaining a defect target detection image.

[0083] The thermal defect detection device for transmission lines provided by the embodiments of the present invention can provide color and details with visible light images and temperature difference information with infrared images in an environment with good lighting conditions. The combination of the two helps to distinguish targets with similar appearances and can improve the target discrimination ability. In a complex and changeable environment, multi-modal fusion can significantly enhance the environmental perception ability of the unmanned aerial vehicle (UAV), greatly expanding the application scenarios of UAV inspection.

[0084] It can be understood that the implementation manners in the thermal defect detection method for transmission lines described in the above-mentioned Embodiment 1 are equally applicable to this embodiment and can achieve the same technical effects, so they will not be described repeatedly here.

[0085] Embodiment 3 Based on the same concept, the embodiments of the present invention also provide an electronic device. Figure 9 FIG. is a schematic structural diagram of an electronic device provided by the embodiments of the present invention. As Figure 9 shown, the electronic device 900 may include: a processor 910, a communication interface 920, a memory 930, and a communication bus 940. Among them, the processor 910, the communication interface 920, and the memory 930 communicate with each other through the communication bus 940. The processor 910 may call the logical instructions in the memory 930 to execute the steps of the thermal defect detection method for transmission lines described in the above embodiments. For example, it includes: S100. Obtain an infrared image and a visible light image of the transmission line; S200. Extract the features of the infrared image and the visible light image to obtain infrared image features and visible light image features; S300. Perform high-temperature target detection on the infrared image to obtain a high-temperature target saliency weight; S400. Calculate the fusion weights of the infrared image features and the visible light image features according to the high-temperature target saliency weight to obtain an infrared image adaptive weight and a visible light image adaptive weight; S500. Perform weighted fusion on the infrared image features and the visible light image features according to the infrared image adaptive weight and the visible light image adaptive weight to obtain a fused feature image; S600. Use a deformed attention detection model to perform thermal defect target detection on the fused feature image to obtain a defect target detection image.

[0086] Among them, the processor 910 may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. chips, or a combination of the above types of chips.

[0087] In addition, when the logical instructions in the above-mentioned memory 930 can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, Read-Only Memories (ROMs), Random Access Memories (RAMs), magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0088] The memory 930 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created by the processor, etc. In addition, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, enterprise intranets, local area networks, mobile communication networks, and combinations thereof.

[0089] Embodiment 4 Based on the same concept, the embodiments of the present invention also provide a computer-readable storage medium. This computer-readable storage medium stores a computer program, and this computer program includes at least one piece of code. This at least one piece of code can be executed by the master device to control the master device to implement the steps of the method for detecting the heating defect of the transmission line as described in the above-mentioned various embodiments. For example, it includes: S100. Obtain the infrared image and visible light image of the transmission line; S200. Extract the features of the infrared image and visible light image to obtain the infrared image features and visible light image features; S300. Perform high-temperature target detection on the infrared image to obtain the significance weight of the high-temperature target; S400. Calculate the fusion weights of the infrared image features and visible light image features according to the significance weight of the high-temperature target to obtain the adaptive weight of the infrared image and the adaptive weight of the visible light image; S500. Perform weighted fusion on the infrared image features and visible light image features according to the adaptive weight of the infrared image and the adaptive weight of the visible light image to obtain the fused feature image; S600. Use the deformed attention detection model to perform heat defect target detection on the fused feature image to obtain the defect target detection image.

[0090] Based on the same technical concept, an embodiment of the present invention also provides a computer program, which is used to implement the above method embodiment when executed by the main control device.

[0091] The computer program can be stored in whole or in part on a computer-readable storage medium packaged together with the processor, or can be stored in whole or in part on a memory not packaged together with the processor.

[0092] Based on the same technical concept, an embodiment of the present invention also provides a processor, which is used to implement the above method embodiment. The above processor can be a chip.

[0093] In summary, the heat defect detection method, device, equipment and storage medium of the transmission line provided by the present invention perform image fusion by adaptively and dynamically adjusting the weights of the infrared image and visible light image, so that the resulting image combines the comprehensive features of the two images. The visible light image provides high-resolution texture and shape information, and the infrared image provides the thermal features of the target in low-light or occluded environments, which can improve the robustness of target detection and greatly improve the accuracy of defect detection. In an environment with good lighting conditions, the visible light image can provide color and details, and the infrared image provides temperature difference information. The combination of the two helps to distinguish targets with similar appearances and can improve the target discrimination ability. In a complex and changeable environment, multi-modal fusion can significantly improve the environmental perception ability of the unmanned aerial vehicle, greatly expanding the application scenarios of unmanned aerial vehicle inspection. The deformed attention detection model focuses on the local area rather than the global area in the image, thereby improving the detection ability of small targets and the training efficiency. Due to its strong global information modeling ability and flexible attention mechanism, it can improve the accuracy of target recognition in complex scenarios and is applicable to the task of identifying heat defects in transmission line fittings.

[0094] Reference to "embodiments" in this document means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appearing in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0095] The above-described embodiments merely represent several implementation manners of the present invention. The description thereof is relatively specific and detailed, but should not be construed as a limitation on the scope of the invention patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent of the present invention shall be subject to the appended claims.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting heating defects in a power transmission line, characterized in that: The method comprises: Acquire infrared and visible light images of power transmission lines; Extracting features of the infrared image and the visible light image to obtain infrared image features and visible light image features; Performing high-heat target detection on the infrared image to obtain a high-heat target significance weight; Calculating the fusion weight of the infrared image feature and the visible light image feature according to the high-heat target saliency weight, and obtaining the infrared image adaptive weight and the visible light image adaptive weight; According to the infrared image adaptive weight and the visible light image adaptive weight, weighted fusion is performed on the infrared image features and the visible light image features to obtain a fused feature image; The deformable attention detection model is used to perform heat defect target detection on the fused feature image to obtain a defect target detection image.

2. The method for detecting heating defects of a power transmission line according to claim 1, characterized in that: The step of extracting the features of the infrared image and the visible light image to obtain the features of the infrared image and the visible light image includes: Construct a convolutional feature extraction network based on the residual network model and feature pyramid network; Using the convolutional feature extraction network to extract features of the infrared image, to obtain features of the infrared image; The convolutional feature extraction network is used to extract features of the visible light image to obtain the visible light image features.

3. The method for detecting heating defects of a power transmission line according to claim 1, characterized in that: The step of performing high-heat target detection on the infrared image to obtain a high-heat target significance weight includes: Obtaining temperature information of each coordinate point of the infrared image and average temperature information of the entire image; Calculating according to the temperature information and the average temperature information of the entire image to obtain a local temperature difference of the infrared image; Calculating a temperature difference significance index according to the local temperature difference; A high heat target area is extracted according to the temperature difference significance index, and the high heat target significance weight of the high heat target area is calculated.

4. The method for detecting heating defects of a power transmission line according to claim 3, characterized in that: The step of calculating the fusion weight of the infrared image feature and the visible light image feature according to the high heat target saliency weight to obtain the infrared image adaptive weight and the visible light image adaptive weight includes: Calculating the fusion weight of the infrared image feature according to the high-heat target significance weight to obtain the infrared image adaptive weight; The fusion weight of the visible light image feature is calculated according to the infrared image adaptive weight to obtain the visible light image adaptive weight.

5. The method for detecting heating defects of a power transmission line according to claim 1, characterized in that: The method of performing heat defect target detection on the fused feature image using the deformation attention detection model to obtain a defect target detection image includes: Construct an initial deformation attention detection model; Training the initial deformation attention detection model to obtain the deformation attention detection model; The fused feature image is input into the deformation attention detection model to perform heating defect target detection to obtain the defect target detection image.

6. The method for detecting heating defects of a power transmission line according to claim 5, characterized in that: The training of the initial deformation attention detection model to obtain the deformation attention detection model includes: Acquire an image data set and perform image preprocessing to obtain an initial data set; Performing forward propagation on the initial deformation attention detection model using the initial data set; Setting a loss function of the initial deformation attention detection model; The initialization weights, optimizer and learning rate are set to train the initial deformation attention detection model to obtain the deformation attention detection model.

7. The method for detecting heating defects of a power transmission line according to claim 6, characterized in that: The deformation attention detection model includes a backbone network, a deformation attention unit and a detection head unit. The step of inputting the fused feature image into the deformation attention detection model for performing heating defect target detection to obtain the defect target detection image includes: Extracting features of the fused feature image through the backbone network to obtain a feature image; Dynamically select the neighborhood of the defect query point in the feature image for calculation by the deformation attention unit; The detection head unit outputs the object category corresponding to the defect query point and predicts the bounding box of the object corresponding to the defect query point.

8. A device for detecting heating defects in a power transmission line, characterized in that: The device comprises: An image acquisition module, used to acquire infrared images and visible light images of the transmission line; A feature extraction module, used to extract features of the infrared image and the visible light image to obtain infrared image features and visible light image features; A weight detection module is used to detect high-heat targets in the infrared image and obtain a saliency weight of the high-heat targets; A weight calculation module, used for calculating the fusion weight of the infrared image feature and the visible light image feature according to the high-heat target significance weight, and obtaining the infrared image adaptive weight and the visible light image adaptive weight; A feature fusion module, used for performing weighted fusion of the infrared image features and the visible light image features according to the infrared image adaptive weight and the visible light image adaptive weight to obtain a fused feature image; The defect detection module is used to perform heating defect target detection on the fused feature image by using a deformation attention detection model to obtain a defect target detection image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the method for detecting heating defects in a power transmission line according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting heating defects in a power transmission line according to any one of claims 1 to 7 is implemented.

Images