Defect detection method and device of power transmission line, computer equipment, readable storage medium and program product

Abstract

The invention relates to a defect detection method and device for a power transmission line, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a visible light image, an infrared image and a depth image of a power transmission line, forming a multi-source heterogeneous data set, and performing adaptive enhancement processing on the images in the set; respectively extracting texture features of the visible light image, temperature features of the infrared image and spatial structure features of the depth image; fusing the texture features, the temperature features and the spatial structure features to obtain initial fusion features; performing dynamic weighted optimization on the initial fusion features based on an attention mechanism of space and a channel; based on the optimized fusion features, identifying the defect type, the defect position and the defect risk level of the power transmission line, and taking the defect type, the defect position and the defect risk level as defect data; and inputting the optimized fusion features and defect data into a visual language model, and outputting a defect detection report. By adopting the method, the accuracy of defect identification can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting defects in power transmission lines. Background Technology

[0002] With the development of artificial intelligence and computer vision technology, it has been widely used in power line inspection, thereby intelligently identifying defects in transmission lines based on images.

[0003] In traditional technologies, images are acquired using novel data acquisition devices such as drones and line inspection robots, and existing image processing and analysis methods are used to identify defects in the acquired images.

[0004] However, existing technologies generally suffer from high feature redundancy and large cross-modal semantic gaps in the process of multimodal data alignment and fusion. This makes it difficult for the fused feature vectors to effectively represent fine-grained defect differences. Furthermore, traditional feature extraction models lack the ability to model the spatial context of key components in power transmission scenarios and cannot distinguish different defect categories with similar appearances. In addition, the defect classification process often relies on manual rules or shallow classifiers, failing to fully combine expert knowledge and deep semantic understanding, resulting in a disconnect between the hazard level determination results and the actual risks. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting defects in transmission lines that can improve the accuracy, granularity, and interpretability of defect identification, in order to address the aforementioned technical problems.

[0006] Firstly, this application provides a method for detecting defects in transmission lines, comprising:

[0007] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0008] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0009] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0010] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0011] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0012] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0013] In one embodiment, prior to performing adaptive enhancement processing on images from a multi-source heterogeneous dataset, the method further includes:

[0014] The image processing in the multi-source heterogeneous dataset performs format standardization conversion, data cleaning, and outlier removal. Data cleaning includes removing duplicate data using perceptual hashing algorithm, filling missing data using neighboring frame interpolation or generative adversarial network repair model, and correcting erroneous data caused by sensor failure. Outlier removal includes identifying outliers using box plot method and replacing them using median filtering.

[0015] In one embodiment, adaptive enhancement processing is performed on images from a multi-source heterogeneous dataset, including:

[0016] The image in the multi-source heterogeneous dataset is analyzed by using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; based on the point spread function, Wiener filtering is used to perform deconvolution operation to complete the restoration and correction of the motion blur image.

[0017] Based on the Retinex theoretical framework, the restored and corrected low-light image is decomposed to obtain the illuminance component and the reflectance component; gamma correction and contrast-limited adaptive histogram equalization are performed on the illuminance component in sequence; and filtering is applied to the reflectance component.

[0018] In the case of a multi-source heterogeneous dataset containing multiple frames of images with different exposures, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures; if it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction; where multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, only the exposure parameters are adjusted.

[0019] The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

[0020] In one embodiment, extracting texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images from a multi-source heterogeneous dataset includes:

[0021] The visible light image is input into the ResNet-50 network, the output feature map of the ResNet-50 network in the fourth feature extraction stage is extracted, and global average pooling is performed to obtain the texture features;

[0022] The infrared image is normalized and then input into the MobileNetV3 network to output temperature features.

[0023] Spatial structural features are obtained by processing point cloud data in depth images using deep learning networks.

[0024] In one embodiment, texture features, temperature features, and spatial structure features are fused to obtain initial fused features, including:

[0025] Texture features, temperature features, and spatial structure features are concatenated to obtain joint features;

[0026] The joint features are input into a multilayer perceptron consisting of three fully connected layers for dimensionality reduction to obtain the initial fused features.

[0027] In one embodiment, based on a spatial and channel-based attention mechanism, the initial fused features are dynamically weighted and optimized, including:

[0028] The initial fused features are subjected to global average pooling. The result of global average pooling is transformed into channel weight vectors through two fully connected layers. The initial fused features are then processed using the channel weight vectors to obtain channel attention features.

[0029] Max pooling and average pooling are performed on the channel attention features along the channel dimension. The results of the two pooling operations are concatenated and then convolved to obtain a spatial weight map. The channel attention features are then processed based on the spatial weight map to obtain the optimized fusion features.

[0030] Secondly, this application also provides a defect detection device for transmission lines, comprising:

[0031] The acquisition module is used to acquire visible light images, infrared images, and depth images of transmission lines, forming a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0032] The processing module is used to perform adaptive enhancement processing on images in a multi-source heterogeneous dataset; and to extract texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images in the multi-source heterogeneous dataset.

[0033] The feature fusion module is used to fuse texture features, temperature features, and spatial structure features to obtain initial fused features;

[0034] An optimization module is used to dynamically weight and optimize the initial fused features based on spatial and channel-based attention mechanisms.

[0035] The identification module is used to identify the defect type, location, and risk level of the transmission line based on the optimized fusion features, as defect data.

[0036] The detection module is used to input the optimized fusion features and defect data into the visual language model and output a defect detection report.

[0037] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0038] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0039] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0040] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0041] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0042] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0043] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0044] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0045] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0046] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0047] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0048] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0049] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0050] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0051] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0052] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0053] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0054] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0055] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0056] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0057] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0058] The aforementioned method, apparatus, computer equipment, computer-readable storage medium, and computer program product for detecting defects in transmission lines improve data quality and effectively address inherent defects in the original data by adaptively enhancing a multi-source heterogeneous data set composed of visible light images, infrared images, and depth images of the transmission line. By extracting texture features from the visible light images, temperature features from the infrared images, and spatial structure features from the depth images in the multi-source heterogeneous data set, and fusing the extracted multimodal features, and then dynamically weighting and optimizing the fused features based on spatial and channel attention mechanisms, feature redundancy is reduced and the problem of large cross-modal semantic gaps is effectively solved. Defects in the transmission line are identified using the optimized fused features, and a defect detection report is output through a visual language model based on the optimized fused features and defect data. This achieves an integrated end-to-end mapping from the original image to defect type, location, hazard level, and natural language description, thereby improving the accuracy, fine granularity, and interpretability of defect identification. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a diagram illustrating the application environment of a defect detection method for transmission lines in one embodiment.

[0061] Figure 2 This is a flowchart illustrating a defect detection method for transmission lines in one embodiment;

[0062] Figure 3 This is a schematic diagram of the process for adaptive enhancement processing of images in a multi-source heterogeneous dataset in one embodiment;

[0063] Figure 4 This is a flowchart illustrating a defect detection method for transmission lines in another embodiment;

[0064] Figure 5 This is a structural block diagram of a defect detection device for a transmission line in one embodiment;

[0065] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0068] The defect detection method for transmission lines provided in this application embodiment can be applied to, for example, Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or located on a cloud or other network server. Specifically, terminal 102 or server 104 completes a method for detecting defects in power transmission lines, which includes:

[0069] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0070] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0071] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0072] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0073] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0074] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0075] Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0076] In one exemplary embodiment, such as Figure 2 As shown, a defect detection method for transmission lines is provided, which can be applied to... Figure 1 Taking the server in the example, the explanation includes the following steps 202 to 212. Wherein:

[0077] Step 202: Collect visible light images, infrared images, and depth images of the transmission line to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels.

[0078] Visible light images can reflect the surface appearance, color, texture, and geometric shape information of an object; infrared images can reflect the temperature distribution of the object's surface; depth images are images that record the physical distance between each pixel in the scene and the camera, thus reflecting the three-dimensional spatial structure, shape, and distance information of the object.

[0079] For example, the drone carries three types of sensors that work synchronously: a visible light camera to capture visible light images of the power transmission line, an infrared thermal imager to capture infrared images of the power transmission line, and a depth camera to capture depth images of the power transmission line. These three sensors achieve millisecond-level time synchronization via hardware trigger signals to ensure strict alignment of multimodal data at the same spatial location. The collected data is then transmitted in real-time via a wireless link to a ground station server or temporarily stored on an onboard solid-state drive. Additionally, various scene data are obtained from the power transmission line monitoring system, inspection reports, and fault record databases. This data, together with the collected data, constitutes a multi-source heterogeneous data set, and the total amount of data in this set needs to reach a preset value, for example, no less than 100,000 images.

[0080] Among them, the visible light camera can use a 20-megapixel CMOS sensor with a frame rate of 30 frames per second and a dynamic range of no less than 120 dB; the infrared thermal imager can have a resolution of 640×512 pixels, a temperature measurement accuracy of ±2 degrees Celsius, and a wavelength response range of 8 to 14 micrometers; the depth camera is based on the principle of flight distance measurement, with an effective ranging range of 0.5 to 50 meters and a depth accuracy of ±1 centimeter.

[0081] For example, the multi-source heterogeneous dataset needs to cover transmission line scenarios with different regions, climate conditions and voltage levels in order to comprehensively collect various types of fault data and simulate and generate complex fault cases with high research value. This includes collecting images of insulator pollution under high humidity during the plum rain season in East China, collecting images of loose fittings under strong wind and sand conditions in the Gobi Desert in Northwest China, and collecting images of conductor galloping caused by icing in the frigid Northeast region. It also includes generating extremely rare but highly dangerous load-type fault scenarios through a physical simulation engine, such as bird nests with lightning strike marks and rusted fittings accompanied by external impacts.

[0082] Step 204: Perform adaptive enhancement processing on the images in the multi-source heterogeneous dataset; extract the texture features of the visible light image, the temperature features of the infrared image, and the spatial structure features of the depth image in the multi-source heterogeneous dataset.

[0083] For example, for motion blur, low light and complex lighting conditions, an adaptive filtering algorithm, multi-scale transformation technology and deep neural network model are integrated to correct image blur, enhance image details and improve overall contrast, thereby performing adaptive enhancement processing operation on images in a multi-source heterogeneous dataset, and extracting texture features of visible light images, temperature features of infrared images and spatial structure features of depth images respectively.

[0084] Step 206: Fuse the texture features, temperature features, and spatial structure features to obtain the initial fused features.

[0085] For example, feature vectors from different modalities, including texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images, are stitched together, and a dimensionality reduction operation is performed on the stitched feature vectors through a fully connected layer to obtain the initial fused features.

[0086] Step 208: Based on the spatial and channel attention mechanism, the initial fused features are dynamically weighted and optimized.

[0087] Among them, the spatial attention module is used to calculate weights based on the spatial location of the feature map, focusing on key component areas such as insulators and fittings; the channel attention module is used to evaluate the importance of different feature channels and adaptively adjust the channel weights.

[0088] For example, spatial attention mechanism and channel attention mechanism are used in series. First, channel attention is adjusted on the initial fused features, and then spatial attention is focused, thereby dynamically strengthening the feature expression of key regions and semantic channels and suppressing interference from irrelevant background.

[0089] Step 210: Based on the optimized fusion features, identify the defect type, defect location, and defect risk level of the transmission line as defect data.

[0090] Among them, the types of defects in transmission lines include bird nest defects, metal equipment corrosion defects, and potential external damage hazards.

[0091] For example, based on the optimized fusion features, a multi-task learning framework is adopted, sharing the underlying feature extraction network, while the upper-level branches handle the identification and classification of different defect types respectively. Specifically, for bird nest defects, the output is the bird nest bounding box coordinates, category confidence, and hazard level score; for metal equipment corrosion defects, the output is the corrosion area segmentation mask and level label; and for external force damage hazards, the output is the location, type, and risk level of the external mechanical target.

[0092] Specifically, end-to-end identification of the hazard level of bird nest defects is achieved by embedding expert knowledge of transmission line defects. This expert knowledge is stored in a rule base, including the size, location, material type, and historical fault data of the bird nests. The rule base defines four hazard levels: Level 1 is a small grass bird nest located in a non-critical position on a crossarm with no historical fault record; Level 2 is a medium-sized mixed-material bird nest near an insulator string; Level 3 is a large tree branch bird nest directly contacting the conductor; and Level 4 is a bird nest containing metal wire located in a high-lightning-risk area. The identification model matches the collected image features with the rule base to output a hazard level score. The model outputs raw scores across four dimensions through a fully connected layer. The probability distribution is obtained after Softmax normalization, and the level corresponding to the highest probability is taken as the final judgment. The normalization calculation can be expressed as:

[0093]

[0094] in, The attention weights are normalized. For the first The original attention score for each spatial location or channel. This represents the total number of position inputs or channels.

[0095] Specifically, the classification of rust defects in metal equipment is based on rust area and color indicators. The rusted area is extracted using image segmentation algorithms, its area percentage is calculated, and the color characteristics of the rusted area, such as hue, saturation, and brightness, are analyzed. Combined with preset thresholds, the degree of rust is divided into multiple levels. Specifically, based on the rust area percentage threshold, it is defined as: less than 5% is considered slight, 5% to 20% is considered moderate, and greater than 20% is considered severe. Based on the color of the rusted area, if its hue value is greater than 30 and its saturation is greater than 0.4, it is judged as active rust, and the risk level of the rust defect in the metal equipment needs to be increased by one level.

[0096] Specifically, the risk level for external force damage is determined by reconstructing the 3D spatial structure and relationships between objects in a scene using a depth prediction model. The depth prediction model uses stereo vision technology or time-of-flight methods to acquire depth information, thereby accurately locating the position of external machinery such as excavators, transport vehicles, and cranes. Then, the risk level is determined based on the distance between the external machinery and the power transmission line, as well as its motion state. The motion state of the external machinery needs to be determined based on continuous frame optical flow analysis. Based on distance thresholds, risks are categorized as follows: less than 5 meters is defined as high risk, 5 to 10 meters as medium risk, and greater than 10 meters as low risk. Based on motion state, if the external machinery is stationary, the risk level is lowered by one level; if the external machinery is in motion, the risk level is raised by one level.

[0097] Step 212: Input the optimized fusion features and defect data into the visual language model and output a defect detection report.

[0098] For example, a visual language pre-trained model is employed. This model architecture is based on a Transformer encoder-decoder structure. The encoder receives fused feature vectors and positional encodings, while the decoder generates descriptive text word by word. Training is performed through a contrastive learning objective, mapping the fused feature vectors and text description vectors to the same semantic space. This enables automatic extraction and natural language description generation of key information such as defect type, location, and severity. The training data contains 100,000 image-text pairs, and the text descriptions follow a template: "[Defect type] was found at [location], with a severity level of [level], and [recommended treatment measures]". During inference, the model outputs a structured JSON report containing fields such as defect ID, type, coordinates, severity level, confidence level, and descriptive text.

[0099] For example, a feature compression algorithm for image recognition defect networks is employed, along with a model reorganization method based on knowledge distillation and an inference acceleration algorithm based on batch parallel processing. This includes pruning and quantization techniques. Pruning removes less contributing connections from the network, quantization converts model weights from 32-bit floating-point numbers to 8-bit integers, and model reorganization transfers knowledge from a large teacher model to a small student model through knowledge distillation. Batch parallel processing divides the inference task into multiple processing units for simultaneous execution, thereby achieving joint hardware and software optimization to accelerate defect identification. Specifically, pruning employs an iterative amplitude pruning strategy, pruning 10% of weights in each round for a total of 5 rounds, retaining the top-50% of important connections. Quantization uses symmetric uniform quantization, and the calibration dataset contains 1000 representative images. In knowledge distillation, the teacher model is the original complete model, and the student model is the MobileNetV2 backbone network. The loss function includes hard-label cross-entropy and soft-label KL divergence. Batch parallel processing is implemented on GPUs, with the batch size dynamically adjusted from 8 to 32, automatically optimized based on GPU memory usage.

[0100] For example, key technologies including image enhancement, feature optimization, defect recognition, and structured understanding are integrated into a unified modular design and encapsulated into a modular model. The unified modular design divides each key technology into an independent module, and the modules communicate through a predefined application programming interface. The encapsulated model undergoes more than 1,000 hours of continuous testing, covering 200 different scenario combinations, including extreme weather, complex terrain, and multiple concurrent defects, to ensure its adaptability and robustness in various scenarios. The model's interface specification adopts the RESTful API standard, and the model's input is Base64 encoded multimodal image data, while the output is a structured report in JSON format.

[0101] For example, the packaged model is deployed to an actual power inspection system using a dedicated tool. This deployment tool provides a graphical user interface and a command-line interface, supporting hot updates and dynamic parameter adjustments. After system integration, it can process real-time video streams and batch image data, with end-to-end processing time for a single image less than 2 seconds. The deployment environment includes edge computing devices (such as NVIDIA Jetson AGX Xavier) and a cloud server cluster. The edge devices handle real-time preliminary screening, while the cloud handles high-precision verification and big data analysis. The system supports integration with existing power production management systems, automatically pushing defect work orders to maintenance personnel's terminals.

[0102] In the aforementioned defect detection method for transmission lines, adaptive enhancement processing of a multi-source heterogeneous data set consisting of visible light images, infrared images, and depth images of the transmission line can improve data quality and effectively address the inherent defects of the original data. By extracting texture features from the visible light images, temperature features from the infrared images, and spatial structure features from the depth images in the multi-source heterogeneous data set, and fusing the extracted multimodal features, and then dynamically weighting and optimizing the fused features based on spatial and channel attention mechanisms, feature redundancy can be reduced and the problem of large cross-modal semantic gaps can be effectively solved. The optimized fused features are used to identify defects in the transmission line, and based on the optimized fused features and defect data, a defect detection report is output through a visual language model. This achieves an integrated end-to-end mapping from the original image to the defect type, location, hazard level, and natural language description, thereby improving the accuracy, fine granularity, and interpretability of defect identification.

[0103] In one embodiment, before performing adaptive enhancement processing on images in a multi-source heterogeneous dataset, the method further includes: performing format normalization conversion, data cleaning, and outlier removal processing on the images in the multi-source heterogeneous dataset. The data cleaning processing includes removing duplicate data using a perceptual hashing algorithm, filling in missing data using neighboring frame interpolation or a generative adversarial network repair model, and correcting erroneous data caused by sensor faults. The outlier removal processing includes identifying outliers using a box plot method and replacing them using median filtering.

[0104] Format standardization refers to converting all images into the lossless compressed PNG format; erroneous data refers to images that are completely black, completely white, or have striped noise due to sensor malfunction.

[0105] For example, images from a multi-source heterogeneous dataset are uniformly converted into lossless compressed PNG format, with infrared temperature data stored as 16-bit grayscale images and depth images stored as 16-bit floating-point matrices in millimeters. A perceptual hash algorithm is used for detection; if the Hamming distance between two images is less than a preset threshold, they are considered duplicate data, and a duplicate data removal operation is performed. The preset threshold can be 5. Missing data is filled using neighboring frame interpolation or a generative adversarial network (GAN) repair model. Images with completely black, completely white, or striped noise caused by sensor malfunctions are automatically identified and removed using pixel statistical histograms. Data cleaning is achieved by performing duplicate data removal, missing data filling, and erroneous data correction operations on images in the multi-source heterogeneous dataset. For outliers in the multi-source heterogeneous dataset, a box plot method is used to calculate the interquartile range for each pixel channel; data points exceeding 1.5 times the upper or lower quartile range are considered outliers and replaced using median filtering.

[0106] After preprocessing, including standardization transformation, data cleaning, and outlier removal, the consistency index of the dataset annotation must exceed 98%. The preprocessed images are annotated by professionals, and three independent annotators are used for comparison and checking to resolve annotation discrepancies. Then, sampling review is conducted.

[0107] In this embodiment, by performing standardization transformation, data cleaning, and outlier removal operations on the data in the multi-source heterogeneous data set, the consistency and comparability of the data are ensured, storage space is reduced, thereby improving data quality and model accuracy, and optimizing computational efficiency.

[0108] In one embodiment, such as Figure 3 As shown, adaptive enhancement processing of images from a multi-source heterogeneous dataset includes steps 302 to 308. Wherein:

[0109] Step 302: Analyze the images in the multi-source heterogeneous dataset using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; based on the point spread function, perform deconvolution operation using Wiener filtering to complete the restoration and correction of the motion blur image.

[0110] For example, motion blur estimation is performed on a visible light image by using a gradient sparse prior model to solve for the point spread function. Then, based on the point spread function, a deconvolution operation is performed through Wiener filtering to restore the high-frequency details of the image, thereby completing the restoration and correction of the motion-blurred image.

[0111] Step 304: Based on the Retinex theoretical framework, the restored and corrected low-light image is decomposed to obtain the illuminance component and the reflectance component; gamma correction and contrast-limited adaptive histogram equalization are performed on the illuminance component in sequence; and filtering is applied to the reflectance component.

[0112] The illuminance component refers to the intensity of light incident on the surface of an object; the reflection component refers to the reflectivity of the surface of an object to light.

[0113] For example, for low-light images in the acquired visible light images, the Retinex theoretical framework is used to decompose the image into illuminance components and reflectance components. Gamma correction and contrast-limited adaptive histogram equalization are performed on the illuminance components, and guided filtering is used to preserve details of the reflectance components, thereby restoring the dark area information of the image.

[0114] Step 306: In the case of multiple frames of images with different exposures in a multi-source heterogeneous dataset, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures; if it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction; where multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, only the exposure parameters are adjusted.

[0115] For example, for complex lighting conditions, such as backlighting or local overexposure, a multi-exposure fusion strategy is adopted. When a single acquisition contains multiple frames of images with different exposures, that is, when the multi-source heterogeneous data set contains multiple images of the same shooting scene and the same target object, and only the exposure parameters are adjusted while other camera parameters remain unchanged, then weighted fusion is performed through pyramid decomposition. When the images in the multi-source heterogeneous data set are only single frames, then the deep learning model is used to predict the image under normal lighting, thereby restoring the true color of the image.

[0116] Step 308: The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

[0117] For example, the training process of the convolutional neural network structure uses over 100,000 power transmission inspection images covering different lighting, weather, and seasonal conditions. The main network is a U-Net architecture, with the encoder using ResNet-34 as the backbone and the decoder fusing multi-scale features through skip connections. The loss function integrates L1 loss, perceptual loss, and adversarial loss, where the perceptual loss is calculated based on the ReLU3_3 layer features of the VGG16 network, and the adversarial loss uses the PatchGAN discriminator. During training, the Adam optimizer is used, with an initial learning rate of 0.0002, a batch size of 16, and 200 training epochs. The processed images are then input into the convolutional neural network structure for optimization. The optimized images must meet the following requirements: peak signal-to-noise ratio of not less than 30 dB and structural similarity index of not less than 0.92.

[0118] In this embodiment, by performing motion blur restoration correction, low-light image restoration correction, and complex lighting condition restoration correction on the images in the original multi-source heterogeneous dataset, the image quality can be enhanced, thereby providing a reliable foundation for subsequent feature extraction.

[0119] In one embodiment, extracting texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images from a multi-source heterogeneous dataset includes: inputting the visible light image into a ResNet-50 network, extracting the output feature map from the ResNet-50 network in the fourth feature extraction stage, and performing global average pooling to obtain texture features; inputting the normalized infrared image into a MobileNetV3 network to output temperature features; and processing the point cloud data in the depth image through a deep learning network to obtain spatial structure features.

[0120] For example, visible light images from a multi-source heterogeneous dataset are input into a ResNet-50 network. The texture feature map output by the ResNet-50 network in the fourth feature extraction stage is extracted. This feature map has a dimension of 14×14×2048, and global average pooling is performed on it to obtain a 2048-dimensional vector. Infrared images from the multi-source heterogeneous dataset are normalized and input into a lightweight MobileNetV3 model, outputting a temperature feature vector with a dimension of 1024. Depth images from the multi-source heterogeneous dataset are processed through a PointNet++ network, converting the point cloud data into a 1024-dimensional global spatial structure feature vector. The extracted 2048-dimensional visible light feature vector is concatenated with the 1024-dimensional infrared feature vector and the 1024-dimensional depth feature vector to form a 4096-dimensional joint feature vector.

[0121] In this embodiment, by extracting the texture features of the visible light image, the temperature features of the infrared image, and the spatial structure features of the depth image respectively, and concatenating the feature vectors of different modes, the robustness and discrimination ability of the feature vectors of the transmission line can be improved.

[0122] In one embodiment, the fusion of texture features, temperature features, and spatial structure features to obtain initial fused features includes: concatenating texture features, temperature features, and spatial structure features to obtain joint features; and inputting the joint features into a multilayer perceptron consisting of three fully connected layers for dimensionality reduction to obtain initial fused features.

[0123] For example, feature vectors of different modalities extracted from multi-source heterogeneous datasets include texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images. These feature vectors of different modalities are concatenated using a concatenation operation to obtain joint features. Then, a multilayer perceptron consisting of three fully connected layers is used to reduce the dimensionality and fuse these joint features to obtain initial fused features. This multimodal feature fusion can be represented as:

[0124]

[0125] in, This represents a feature vector extracted from a visible light image. This represents a feature vector extracted from an infrared image. This represents a feature vector extracted from a depth image. This indicates a splicing operation. This indicates that the multilayer perceptron is used for dimensionality reduction and fusion.

[0126] The multilayer perceptron contains three fully connected layers with 2048, 1024, and 512 hidden units, respectively, and uses the LeakyReLU activation function to output a 512-dimensional initial fused feature vector.

[0127] In this embodiment, by performing splicing, dimensionality reduction and fusion operations on texture features, temperature features and spatial structure features, the integrity of information can be guaranteed, feature redundancy can be removed, and the correlation between multimodal features can be enhanced.

[0128] In one embodiment, based on the spatial and channel attention mechanism, the initial fused features are dynamically weighted and optimized, including: performing global average pooling on the initial fused features; converting the global average pooling result into a channel weight vector through two fully connected layers; processing the initial fused features using the channel weight vector to obtain channel attention features; performing max pooling and average pooling on the channel attention features along the channel dimension; concatenating the results of the two pooling operations and then performing convolution to obtain a spatial weight map; and processing the channel attention features based on the spatial weight map to obtain the optimized fused features.

[0129] For example, after the initial fusion of feature maps, global average pooling is performed to obtain the channel dimensions. The obtained channel dimensions are then compressed based on the first fully connected layer (i.e., the compression layer) with a compression ratio of 16, resulting in the dimensionality-reduced channel dimensions. The dimensionality-reduced channel dimensions are then expanded based on the second fully connected layer (i.e., the expansion layer) with the same compression ratio of 16, resulting in the expanded channel dimensions. The number of channel dimensions is the same as the number of initially obtained channel dimensions. The dimensionality vector of the expanded channel dimensions is then mapped using the Sigmoid function to obtain the channel weight vector. The initial fused features are then processed based on the channel weight vector to obtain the channel attention features.

[0130] For example, the feature maps after initial fusion are subjected to max pooling and average pooling along the channel dimension to obtain two two-dimensional spatial description maps. The two obtained two-dimensional spatial description maps are then concatenated, and a 7×7 convolutional layer is used to perform a convolution operation on the concatenated fusion map to generate a spatial weight map. Based on the spatial weight map, the channel attention features are processed to obtain the optimized fusion features.

[0131] In this embodiment, the initial fused features are dynamically weighted and optimized by spatial attention and channel attention. Channel attention is adjusted first, and then spatial attention is focused, which can reduce feature redundancy and effectively suppress irrelevant background interference.

[0132] like Figure 4 As shown, a specific embodiment illustrates a method for detecting defects in transmission lines, including steps 402 to 416. Wherein,

[0133] Step 402: Collect visible light images, infrared images, and depth images of the transmission line to form a multi-source heterogeneous data set.

[0134] Specifically, the drone carries three types of sensors that work simultaneously: a visible light camera to capture visible light images of the transmission lines, an infrared thermal imager to capture infrared images of the transmission lines, and a depth camera to capture depth images of the transmission lines. The collected data is transmitted in real-time via wireless link to a ground station server or temporarily stored on an onboard solid-state drive. In addition, various scene data are obtained from the transmission line monitoring system, inspection reports, and fault record databases. These, along with the collected data, form a multi-source heterogeneous data set. This data set contains no fewer than 100,000 images and needs to cover transmission line scenarios across different regions, climate conditions, and voltage levels.

[0135] Step 404: Perform preprocessing operations and adaptive enhancement processing on the images in the multi-source heterogeneous data set, and extract the texture features of the visible light image, the temperature features of the infrared image, and the spatial structure features of the depth image.

[0136] Specifically, preprocessing operations such as format standardization, data cleaning, and outlier removal are performed on images from multi-source heterogeneous datasets. Based on the preprocessed images, adaptive filtering algorithms, multi-scale transformation techniques, and deep neural network models are integrated to correct image blur, enhance image details, and improve overall contrast, thus performing adaptive enhancement processing on multimodal images from multi-source heterogeneous datasets, addressing motion blur, low light, and complex lighting conditions. Based on the enhanced multimodal image data, texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images are extracted.

[0137] Step 406: Fuse and optimize texture features, temperature features, and spatial structure features.

[0138] Specifically, feature vectors from different modalities, including texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images, are concatenated. A fully connected layer then performs dimensionality reduction on the concatenated feature vectors to obtain initial fused features. Finally, based on spatial and channel attention mechanisms, the initial fused features are dynamically weighted and optimized to obtain optimized fused features.

[0139] Step 408: Based on the optimized fusion features, identify the defect type, defect location, and defect risk level of the transmission line as defect data.

[0140] Specifically, based on the optimized fusion features, a multi-task learning framework is used to identify defects in bird nests, corrosion defects in metal equipment, and potential external damage hazards, and outputs defect data according to different defect types. For bird nest defects, the coordinates of the bird nest bounding box, category confidence, and hazard level score are output; for corrosion defects in metal equipment, the rust area segmentation mask and level label are output; and for potential external damage hazards, the location, type, and risk level of the external mechanical target are output.

[0141] Step 410: Based on the optimized fusion features and defect data, output a defect detection report through a visual language model.

[0142] Specifically, a visual language pre-trained model is used to map fused feature vectors and text description vectors to the same semantic space, enabling automatic extraction and natural language description generation of key information such as defect type, location, and severity. The training data contains 100,000 image-text pairs, with text descriptions following a template: "[Defect type] found at [location], severity level [level], recommended [treatment measures]". During inference, the model outputs a structured JSON report containing fields such as defect ID, type, coordinates, severity level, confidence level, and descriptive text.

[0143] Step 412 involves joint hardware and software optimization, using pruning, quantization, and knowledge distillation techniques to compress model parameters, and improving inference speed through batch parallel processing.

[0144] Specifically, a feature compression algorithm for image recognition defect networks is adopted, along with a model reorganization method based on knowledge distillation and an inference acceleration algorithm based on batch parallel processing. This includes pruning and quantization techniques. The pruning process removes connections that contribute less to the network, and quantization converts the model weights from 32-bit floating-point numbers to 8-bit integers. The model reorganization transfers knowledge from a large teacher model to a small student model through knowledge distillation. Batch parallel processing divides the inference task into multiple processing units for synchronous execution, thereby achieving joint hardware and software optimization to accelerate defect identification.

[0145] Step 414: Integrate key technologies and encapsulate them into a modular model, defining standardized input and output interfaces.

[0146] Specifically, key technologies, including image enhancement, feature optimization, defect recognition, and structured understanding, are integrated into a unified modular design and encapsulated into a modular model. This unified modular design divides each key technology into an independent module, and these modules communicate through a predefined application programming interface (API). The encapsulated model underwent over 1000 hours of continuous testing, covering 200 different scenario combinations, including extreme weather, complex terrain, and multiple concurrent defects, ensuring its adaptability and robustness across various scenarios. The model achieves an average recognition accuracy of 96.7% and a false positive rate of less than 2.1%. Furthermore, the model's interface specification adopts the RESTful API standard, with Base64-encoded multimodal image data as input and a structured report in JSON format as output.

[0147] Step 416: Deploy the packaged model to the actual power inspection system.

[0148] Specifically, the packaged model is deployed to the actual power inspection system using specialized tools. These tools provide both a graphical user interface and a command-line interface, supporting hot updates and dynamic parameter adjustments. After system integration, it can process real-time video streams and batch image data, with end-to-end processing time for a single image less than 2 seconds. Furthermore, the system supports integration with existing power production management systems, automatically pushing defect work orders to maintenance personnel's terminals.

[0149] A specific application scenario can be constructed and validated based on the model. For example, in a 500 kV transmission line inspection task, a drone equipped with a multimodal sensor flies along the line and collects a sequence of images containing insulator strings. Visible light images show suspected contamination on the insulator surface, infrared images show an abnormal local temperature increase of 3 degrees Celsius, and depth images show minor deformation in the area. After image enhancement, texture features from the visible light image, temperature features from the infrared image, and spatial structure features from the depth image are extracted respectively. These three modal features are then fused and focused on the abnormal area using an attention mechanism. Based on the optimized fused features, the abnormal area is identified as "insulator contamination accompanied by local heating," and the hazard level is determined to be level three. The identification result is used to generate a description through a visual language model: "Severe contamination was found on the 5th insulator of phase A on tower #23. The surface temperature is 3 degrees Celsius higher than the normal area. The hazard level is level three. It is recommended to arrange a power outage for cleaning," and the description is pushed to the operation and maintenance system.

[0150] A specific application scenario can be constructed and validated based on the model. For example, during a 220 kV line inspection, a large bird's nest was found on a crossarm. The nest was identified as measuring 40 cm × 30 cm, made of metal wire, and located adjacent to a jumper wire. Historical records showed that two short-circuit faults had occurred in this area. Combining this with the expert rule base, the hazard level was determined to be level four, an emergency alarm was generated, and the nearest troubleshooting team was automatically dispatched to handle the situation. The system also records this case for subsequent model iteration and optimization.

[0151] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0152] Based on the same inventive concept, this application also provides a transmission line defect detection device for implementing the above-described transmission line defect detection method. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more transmission line defect detection device embodiments provided below can be found in the limitations of the transmission line defect detection method described above, and will not be repeated here.

[0153] In one exemplary embodiment, such as Figure 5 As shown, a defect detection device 500 for transmission lines is provided, comprising: a data acquisition module 502, a processing module 504, a feature fusion module 506, an optimization module 508, an identification module 510, and a detection module 512, wherein:

[0154] The acquisition module 502 is used to acquire visible light images, infrared images and depth images of transmission lines to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions and voltage levels;

[0155] The processing module 504 is used to perform adaptive enhancement processing on images in a multi-source heterogeneous dataset; and to extract texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images from the multi-source heterogeneous dataset.

[0156] The feature fusion module 506 is used to fuse texture features, temperature features and spatial structure features to obtain initial fused features;

[0157] The optimization module 508 is used to dynamically weight and optimize the initial fused features based on the spatial and channel-based attention mechanism.

[0158] The identification module 510 is used to identify the defect type, defect location and defect risk level of the transmission line based on the optimized fusion features, as defect data;

[0159] The detection module 512 is used to input the optimized fusion features and defect data into the visual language model and output a defect detection report.

[0160] In one embodiment, before performing adaptive enhancement processing on the images in the multi-source heterogeneous data set, the processing module further includes: performing format standardization conversion, data cleaning, and outlier removal processing on the images in the multi-source heterogeneous data set. The data cleaning processing includes removing duplicate data using a perceptual hashing algorithm, filling in missing data using neighboring frame interpolation or a generative adversarial network repair model, and correcting erroneous data caused by sensor faults. The outlier removal processing includes identifying outliers using a box plot method and replacing them using median filtering.

[0161] In one embodiment, the processing module is further configured to perform adaptive enhancement processing on images in a multi-source heterogeneous dataset, including: analyzing the images in the multi-source heterogeneous dataset using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; performing deconvolution operation using Wiener filtering based on the point spread function to complete the restoration and correction of the motion-blurred image; decomposing the restored and corrected low-light image based on the Retinex theoretical framework to obtain illuminance and reflectance components; sequentially performing gamma correction and contrast-limited adaptive histogram equalization on the illuminance component; and performing gamma correction and contrast-limited adaptive histogram equalization on the reflectance component. The algorithm employs filtering to process multiple frames of images with different exposures in a multi-source heterogeneous dataset. A pyramid decomposition weighted fusion strategy is used to process these images. If only a single frame is captured, it is input into a pre-defined deep learning model for normal illumination prediction. Multiple frames with different exposures refer to multiple images taken for the same shooting scene and the same target object, with all other camera parameters unchanged except for the exposure parameters. The processed images are then input into a convolutional neural network model for optimization. This model uses a U-Net architecture, with a ResNet-34 backbone encoder and a decoder that fuses multi-scale features via skip connections.

[0162] In one embodiment, the processing module is further configured to extract texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images from a multi-source heterogeneous dataset. This includes: inputting the visible light image into a ResNet-50 network, extracting the output feature map of the ResNet-50 network in the fourth feature extraction stage, and performing global average pooling to obtain texture features; inputting the normalized infrared image into a MobileNetV3 network to output temperature features; and processing the point cloud data in the depth image through a deep learning network to obtain spatial structure features.

[0163] In one embodiment, the feature fusion module is further used to fuse texture features, temperature features and spatial structure features to obtain initial fused features, including: splicing texture features, temperature features and spatial structure features to obtain joint features; and inputting the joint features into a multilayer perceptron composed of three fully connected layers for dimensionality reduction to obtain initial fused features.

[0164] In one embodiment, the optimization module is further configured to dynamically weight and optimize the initial fused features based on spatial and channel attention mechanisms, including: performing global average pooling on the initial fused features; converting the global average pooling result into a channel weight vector through two fully connected layers; processing the initial fused features using the channel weight vector to obtain channel attention features; performing max pooling and average pooling on the channel attention features along the channel dimension; concatenating the results of the two pooling operations and then performing convolution to obtain a spatial weight map; and processing the channel attention features based on the spatial weight map to obtain the optimized fused features.

[0165] Each module in the aforementioned defect detection device for transmission lines can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0166] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting defects in power transmission lines.

[0167] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0168] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0169] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0170] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0171] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0172] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0173] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0174] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0175] In one embodiment, when the processor executes the computer program, it further implements the following steps: before performing adaptive enhancement processing on the images in the multi-source heterogeneous data set, it further includes performing format normalization conversion, data cleaning, and outlier removal processing on the images in the multi-source heterogeneous data set. The data cleaning processing includes removing duplicate data using a perceptual hashing algorithm, filling in missing data using neighboring frame interpolation or a generative adversarial network repair model, and correcting erroneous data caused by sensor failure. The outlier removal processing includes identifying outliers using a box plot method and replacing them using median filtering.

[0176] In one embodiment, when the processor executes the computer program, it further performs the following steps: adaptive enhancement processing on images in a multi-source heterogeneous dataset, including: analyzing the images in the multi-source heterogeneous dataset using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; performing deconvolution operation using Wiener filtering based on the point spread function to complete the restoration and correction of the motion blur image; decomposing the restored and corrected low-light image based on the Retinex theoretical framework to obtain the illuminance component and the reflectance component; and sequentially performing gamma correction and contrast-limited adaptive histogram equalization on the illuminance component. The reflection component is filtered. In the case of multiple frames of images with different exposures in a multi-source heterogeneous dataset, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures. If it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction. Here, multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, but only the exposure parameters are adjusted. The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

[0177] In one embodiment, when the processor executes the computer program, it further performs the following steps: extracting texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images from a multi-source heterogeneous dataset, including: inputting the visible light image into a ResNet-50 network, extracting the output feature map output by the ResNet-50 network in the fourth feature extraction stage, and performing global average pooling to obtain texture features; inputting the normalized infrared image into a MobileNetV3 network to output temperature features; and processing the point cloud data in the depth image through a deep learning network to obtain spatial structure features.

[0178] In one embodiment, when the processor executes the computer program, it further performs the following steps: fusing texture features, temperature features, and spatial structure features to obtain initial fused features, including: concatenating texture features, temperature features, and spatial structure features to obtain joint features; and inputting the joint features into a multilayer perceptron composed of three fully connected layers for dimensionality reduction to obtain initial fused features.

[0179] In one embodiment, when the processor executes the computer program, it further implements the following steps: dynamically weighting and optimizing the initial fused features based on spatial and channel attention mechanisms, including: performing global average pooling on the initial fused features, converting the global average pooling result into a channel weight vector through two fully connected layers, processing the initial fused features through the channel weight vector to obtain channel attention features; performing max pooling and average pooling on the channel attention features along the channel dimension, concatenating the results of the two pooling operations, and then performing convolution processing to obtain a spatial weight map; processing the channel attention features based on the spatial weight map to obtain optimized fused features.

[0180] The implementation principle and technical effects of the above embodiments are similar to those of the above method embodiments, and will not be repeated here.

[0181] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0182] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0183] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0184] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0185] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0186] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0187] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0188] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: before performing adaptive enhancement processing on the images in the multi-source heterogeneous data set, it further includes performing format normalization conversion, data cleaning, and outlier removal processing on the images in the multi-source heterogeneous data set. The data cleaning processing includes removing duplicate data using a perceptual hashing algorithm, filling in missing data using neighboring frame interpolation or a generative adversarial network repair model, and correcting erroneous data caused by sensor failure. The outlier removal processing includes identifying outliers using a box plot method and replacing them using median filtering.

[0189] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: adaptive enhancement processing of images in a multi-source heterogeneous dataset, including: analyzing the images in the multi-source heterogeneous dataset using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; performing deconvolution operation using Wiener filtering based on the point spread function to complete the restoration and correction of the motion blur image; decomposing the restored and corrected low-light image based on the Retinex theoretical framework to obtain the illuminance component and the reflectance component; and sequentially performing gamma correction and contrast-limited adaptive histogram equalization on the illuminance component. The reflection component is filtered. In the case of multiple frames of images with different exposures in a multi-source heterogeneous dataset, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures. If it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction. Here, multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, but only the exposure parameters are adjusted. The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

[0190] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images from a multi-source heterogeneous dataset, including: inputting the visible light image into a ResNet-50 network, extracting the output feature map output by the ResNet-50 network in the fourth feature extraction stage, and performing global average pooling to obtain texture features; inputting the normalized infrared image into a MobileNetV3 network to output temperature features; and processing the point cloud data in the depth image through a deep learning network to obtain spatial structure features.

[0191] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: fusing texture features, temperature features, and spatial structure features to obtain initial fused features, including: concatenating texture features, temperature features, and spatial structure features to obtain joint features; and inputting the joint features into a multilayer perceptron consisting of three fully connected layers for dimensionality reduction to obtain initial fused features.

[0192] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: dynamically weighting and optimizing the initial fused features based on spatial and channel attention mechanisms, including: performing global average pooling on the initial fused features, converting the global average pooling result into a channel weight vector through two fully connected layers, processing the initial fused features through the channel weight vector to obtain channel attention features; performing max pooling and average pooling on the channel attention features along the channel dimension, concatenating the results of the two pooling operations and then performing convolution processing to obtain a spatial weight map; and processing the channel attention features based on the spatial weight map to obtain optimized fused features.

[0193] The implementation principle and technical effects of the above embodiments are similar to those of the above method embodiments, and will not be repeated here.

[0194] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0195] Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels;

[0196] Adaptive enhancement processing is performed on images from multi-source heterogeneous datasets; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from multi-source heterogeneous datasets.

[0197] Texture features, temperature features, and spatial structure features are fused to obtain initial fused features;

[0198] Based on the attention mechanism of space and channel, the initial fused features are dynamically weighted and optimized;

[0199] Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data.

[0200] The optimized fusion features and defect data are input into the visual language model, and a defect detection report is output.

[0201] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: before performing adaptive enhancement processing on the images in the multi-source heterogeneous data set, it further includes performing format normalization conversion, data cleaning, and outlier removal processing on the images in the multi-source heterogeneous data set. The data cleaning processing includes removing duplicate data using a perceptual hashing algorithm, filling in missing data using neighboring frame interpolation or a generative adversarial network repair model, and correcting erroneous data caused by sensor failure. The outlier removal processing includes identifying outliers using a box plot method and replacing them using median filtering.

[0202] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: adaptive enhancement processing of images in a multi-source heterogeneous dataset, including: analyzing the images in the multi-source heterogeneous dataset using a gradient sparse prior model to obtain the point spread function corresponding to motion blur; performing deconvolution operation using Wiener filtering based on the point spread function to complete the restoration and correction of the motion blur image; decomposing the restored and corrected low-light image based on the Retinex theoretical framework to obtain the illuminance component and the reflectance component; and sequentially performing gamma correction and contrast-limited adaptive histogram equalization on the illuminance component. The reflection component is filtered. In the case of multiple frames of images with different exposures in a multi-source heterogeneous dataset, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures. If it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction. Here, multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, but only the exposure parameters are adjusted. The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

[0203] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: extracting texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images from a multi-source heterogeneous dataset, including: inputting the visible light image into a ResNet-50 network, extracting the output feature map output by the ResNet-50 network in the fourth feature extraction stage, and performing global average pooling to obtain texture features; inputting the normalized infrared image into a MobileNetV3 network to output temperature features; and processing the point cloud data in the depth image through a deep learning network to obtain spatial structure features.

[0204] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: fusing texture features, temperature features, and spatial structure features to obtain initial fused features, including: concatenating texture features, temperature features, and spatial structure features to obtain joint features; and inputting the joint features into a multilayer perceptron consisting of three fully connected layers for dimensionality reduction to obtain initial fused features.

[0205] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: dynamically weighting and optimizing the initial fused features based on spatial and channel attention mechanisms, including: performing global average pooling on the initial fused features, converting the global average pooling result into a channel weight vector through two fully connected layers, processing the initial fused features through the channel weight vector to obtain channel attention features; performing max pooling and average pooling on the channel attention features along the channel dimension, concatenating the results of the two pooling operations and then performing convolution processing to obtain a spatial weight map; and processing the channel attention features based on the spatial weight map to obtain optimized fused features.

[0206] The implementation principle and technical effects of the above embodiments are similar to those of the above method embodiments, and will not be repeated here.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0208] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0209] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0210] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for detecting defects in power transmission lines, characterized in that, The method includes: Visible light images, infrared images, and depth images of transmission lines are collected to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels; Adaptive enhancement processing is performed on the images in the multi-source heterogeneous data set; texture features of visible light images, temperature features of infrared images, and spatial structure features of depth images are extracted from the multi-source heterogeneous data set; The texture features, temperature features, and spatial structure features are fused to obtain an initial fused feature; Based on the spatial and channel-based attention mechanism, the initial fusion features are dynamically weighted and optimized; Based on the optimized fusion features, the defect type, defect location, and defect risk level of the transmission line are identified as defect data. The optimized fusion features and the defect data are input into the visual language model to output a defect detection report.

2. The method according to claim 1, characterized in that, Before performing adaptive enhancement processing on the images in the multi-source heterogeneous data set, the method further includes: The images in the multi-source heterogeneous data set are subjected to format standardization conversion, data cleaning and outlier removal. The data cleaning process includes removing duplicate data by perceptual hashing algorithm, filling missing data by neighboring frame interpolation or generative adversarial network repair model, and correcting erroneous data caused by sensor failure. The outlier removal process includes identifying outliers by box plot method and replacing them by median filtering.

3. The method according to claim 1, characterized in that, The adaptive enhancement processing of images in the multi-source heterogeneous data set includes: The images in the multi-source heterogeneous dataset are analyzed using a gradient sparse prior model to obtain the point spread function corresponding to motion blur. Based on the point spread function, Wiener filtering is used to perform deconvolution operation to complete the restoration and correction of the motion-blurred image. Based on the Retinex theoretical framework, the restored and corrected low-light image is decomposed to obtain the illuminance component and the reflectance component; gamma correction and contrast-limited adaptive histogram equalization are performed on the illuminance component in sequence; and filtering is applied to the reflectance component. When the multi-source heterogeneous data set contains multiple frames of images with different exposures, a pyramid decomposition weighted fusion strategy is used to process the multiple frames of images with different exposures; if it is only a single frame image, it is input into a preset deep learning model for normal illumination prediction; where multiple frames of images with different exposures refer to multiple images taken for the same shooting scene and the same target object, with other camera parameters unchanged, only the exposure parameters are adjusted. The processed image is input into a convolutional neural network model for optimization. The convolutional neural network model adopts the U-Net architecture, with the encoder being a ResNet-34 backbone network and the decoder fusing multi-scale features through skip connections.

4. The method according to claim 1, characterized in that, The extraction of texture features from visible light images, temperature features from infrared images, and spatial structure features from depth images from the multi-source heterogeneous dataset includes: The visible light image is input into the ResNet-50 network, the output feature map output by the ResNet-50 network in the fourth feature extraction stage is extracted, and global average pooling is performed to obtain the texture features; The infrared image is normalized and then input into the MobileNetV3 network to output temperature features. Spatial structural features are obtained by processing point cloud data in depth images using deep learning networks.

5. The method according to claim 1, characterized in that, The process of fusing the texture features, temperature features, and spatial structure features to obtain initial fused features includes: The texture features, temperature features, and spatial structure features are concatenated to obtain a joint feature; The joint features are input into a multilayer perceptron consisting of three fully connected layers for dimensionality reduction to obtain initial fused features.

6. The method according to claim 1, characterized in that, The spatial and channel-based attention mechanism dynamically weights and optimizes the initial fused features, including: The initial fused features are subjected to global average pooling. The result of global average pooling is transformed into a channel weight vector through two fully connected layers. The initial fused features are then processed using the channel weight vector to obtain channel attention features. Max pooling and average pooling are performed on the channel attention features along the channel dimension. The results of the two pooling operations are concatenated and then convolved to obtain a spatial weight map. The channel attention features are then processed based on the spatial weight map to obtain the optimized fusion features.

7. A defect detection device for power transmission lines, characterized in that, The device includes: The acquisition module is used to acquire visible light images, infrared images, and depth images of transmission lines to form a multi-source heterogeneous data set; the multi-source heterogeneous data set covers transmission line scenarios with different regions, climate conditions, and voltage levels. The processing module is used to perform adaptive enhancement processing on the images in the multi-source heterogeneous data set; and to extract the texture features of the visible light image, the temperature features of the infrared image, and the spatial structure features of the depth image in the multi-source heterogeneous data set. The feature fusion module is used to fuse the texture features, the temperature features, and the spatial structure features to obtain an initial fused feature; An optimization module is used to dynamically weight and optimize the initial fused features based on a spatial and channel-based attention mechanism. The identification module is used to identify the defect type, location, and risk level of the transmission line based on the optimized fusion features, as defect data. The detection module is used to input the optimized fusion features and the defect data into the visual language model and output a defect detection report.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images