Self-supervised pre-training method for adaptive inspection of distribution network lines based on drones

Through the integration of self-supervised pre-training methods and multi-level features, the problem of insufficient accuracy and generalization capabilities of traditional detection models in complex distribution network circuit environments is solved, and higher detection accuracy and adaptability are achieved, suitable for complex power UAV patrol for line environment.

CN118587563BActive Publication Date: 2024-11-15ZHONGKE FANGCUN ZHIWEI (NANJING) TECH CO LTD +2
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411074879.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-15
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

The traditional UAV distribution network line detection model relies on ImageNet data and has problems such as overfitting or underfitting, resulting in insufficient detection accuracy and generalization capabilities in complex distribution network line environments, which cannot meet the requirements of adaptive patrol. .

Method used

Using a self-supervised pre-training method based on drone distribution network lines, through multi-level feature extraction and fusion, combined with improved Transformer structure and autoencoder model, the occlusion features are processed using Mask matrix and random matrix to avoid data overfitting. Enhance the robustness and adaptability of the model.

Benefits of technology

It improves the accuracy and effect of inspection tasks, enhances the generalization ability of the model, can better deal with noise, interference and occlusion, adapt to complex power line environments, and improves the accuracy and adaptability of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587563B_ABST
    Figure CN118587563B_ABST
Patent Text Reader

Abstract

The present invention discloses a self-supervised pre-training method in adaptive inspection of distribution network lines based on unmanned aerial vehicles, including obtaining the original image of the distribution network overhead line and preprocessing it; extracting primary multi-level features, using an improved Transformer structure to generate advanced features; fusing the primary multi-level features and advanced features to obtain multi-level mixed features; constructing a random matrix, sorting the random matrix, generating a Mask matrix, and calculating multi-level occlusion features; constructing an autoencoder model, calculating the reconstruction loss of multiple layers of pixels; training the autoencoder model based on the reconstruction loss to obtain a trained autoencoder model and a final reconstruction loss. The present invention improves the accuracy of the inspection task, enhances the robustness of the model, enables it to better cope with noise, interference, occlusion and other situations that may exist in the inspection of distribution network overhead lines, and improves the generalization ability and adaptability of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of target detection and discloses a self-supervised pre-training method in adaptive inspection of distribution network lines based on unmanned aerial vehicles. Background Art

[0002] As the scale of power systems continues to expand and the requirements for power supply reliability continue to increase, the complexity of distribution network overhead lines, as a key component of the transmission network, can be described as multi-dimensional and deep. Since these lines often span vast areas, the traditional manual inspection process needs to deal with a variety of complex terrain and climatic conditions, from flat plains to rugged mountains, from hot deserts to cold glaciers, which poses certain safety risks. The introduction of drone technology, especially the drone adaptive inspection system for distribution network overhead lines, provides a revolutionary solution to this challenge.

[0003] In recent years, drone inspection systems have been widely used in the power industry due to their high efficiency, flexibility, and cost-effectiveness. They can quickly collect high-definition images and various sensor data, effectively monitor line status, and promptly detect potential faults such as wire damage, insulator damage, and tree intrusion. However, with the increase in the complexity of inspection tasks, the requirements for the intelligence level of inspection systems have also increased, especially the accuracy and generalization ability of detection models have become key factors in determining inspection efficiency. Traditional detection models generally use classification models trained with image network (ImageNet) data as pre-training models. ImageNet data is very different from distribution network line data. In the complex and changeable natural environment and line conditions of power distribution, the model lacks the ability to understand and represent data, and often faces overfitting or underfitting problems, resulting in low recall of the model in complex distribution network line environments, which cannot meet the requirements of adaptive inspection for the accuracy and generalization ability of the detection model. Summary of the invention

[0004] The purpose of the invention is to provide a self-supervised pre-training method based on UAV distribution network line adaptive inspection to solve the above-mentioned problems existing in the prior art.

[0005] The technical solution is based on the self-supervised pre-training method in the adaptive inspection of UAV distribution network lines, including the following steps:

[0006] S1. Obtaining the original image of the overhead line of the distribution network collected by the drone, and preprocessing it, including dynamic range compression and data enhancement, to obtain the preprocessed original image;

[0007] S2. Extract primary multi-level features based on the preprocessed original image, generate high-level features based on the primary multi-level features using an improved Transformer structure, and fuse the primary multi-level features with the high-level features to obtain multi-level hybrid features.

[0008] S3. Based on the multi-level mixed features, a random matrix is ​​constructed, and the random matrix is ​​sorted to generate a Mask matrix; based on the Mask matrix and the multi-level mixed features, a multi-level occlusion feature is calculated;

[0009] S4. Construct an autoencoder model, and calculate the reconstruction loss of multiple layers of pixels based on multi-level occlusion features and multi-level mixed features; train the autoencoder model based on the reconstruction loss to obtain a trained autoencoder model and a final reconstruction loss.

[0010] Beneficial effect: The present invention avoids overfitting of data and improves the accuracy and effect of inspection tasks through multi-level feature extraction and fusion; by using the Mask matrix, the model can learn to deal with the situation where some information is missing, which helps to enhance the robustness of the model for expressing input information, so that it can better cope with the noise, interference and occlusion that may exist in the inspection of distribution network overhead lines, thereby improving the generalization ability and adaptability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flow chart of the present invention.

[0012] Figure 2 This is a flow chart of step S1 of the present invention.

[0013] Figure 3 This is a flow chart of step S2 of the present invention.

[0014] Figure 4 This is a flow chart of step S3 of the present invention.

[0015] Figure 5 This is a flow chart of step S4 of the present invention.

[0016] Figure 6 is a flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0017] like Figure 1 As shown, the present application proposes a self-supervised pre-training method based on adaptive inspection of UAV distribution network lines, comprising the following steps:

[0018] S1. Obtaining the original image of the overhead line of the distribution network collected by the drone, and preprocessing it, including dynamic range compression and data enhancement, to obtain the preprocessed original image;

[0019] S2. Extract primary multi-level features based on the preprocessed original image, generate high-level features based on the primary multi-level features using an improved Transformer structure, and fuse the primary multi-level features with the high-level features to obtain multi-level hybrid features.

[0020] S3. Based on the multi-level mixed features, a random matrix is ​​constructed, and the random matrix is ​​sorted to generate a Mask matrix; based on the Mask matrix and the multi-level mixed features, a multi-level occlusion feature is calculated;

[0021] S4. Construct an autoencoder model, and calculate the reconstruction loss of multiple layers of pixels based on multi-level occlusion features and multi-level mixed features; train the autoencoder model based on the reconstruction loss to obtain a trained autoencoder model and a final reconstruction loss.

[0022] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0023] S11, obtaining an original image of the overhead line of the distribution network collected by the drone, performing adaptive exposure control processing on the original image, and obtaining a first intermediate image after brightness adjustment;

[0024] S12, generating a second intermediate image with enhanced details by using a dynamic range compression algorithm based on the first intermediate image;

[0025] S13, performing multi-scale image segmentation and feature enhancement based on the second intermediate image to obtain a third intermediate image;

[0026] S14, performing adaptive cropping and scaling based on the third intermediate image to obtain a fourth intermediate image;

[0027] S15. Based on the fourth intermediate image, use image enhancement and noise suppression technology to obtain a fifth intermediate image;

[0028] S16. Based on the fifth intermediate image, perform data enhancement and feature extraction preparation to generate a masked enhanced image, that is, the preprocessed original image.

[0029] In this embodiment, an original image captured by a drone is obtained; an adaptive exposure control algorithm is applied to the original image, and the brightness value of each pixel is dynamically adjusted by analyzing the brightness distribution of the image to obtain a first intermediate image with balanced brightness; a dynamic range compression algorithm is applied to the first intermediate image, and the image details are enhanced by separating and recombining different frequency components of the image to obtain a second intermediate image with enhanced details.

[0030] An improved superpixel segmentation algorithm is applied to the second intermediate image, and the image is segmented into multiple semantically similar regions through iterative optimization to obtain a segmentation result; a local contrast enhancement algorithm is applied to the segmentation result, and the contrast is adaptively adjusted according to the statistical characteristics of each segmented region to obtain a feature-enhanced third intermediate image.

[0031] An attention-guided cropping algorithm is applied to the third intermediate image to determine the most informative area by calculating the saliency map of the image, thereby obtaining a cropped image. A content-aware scaling algorithm is applied to the cropped image to adjust the image size by retaining important image structures while removing redundant information, thereby obtaining a scaled fourth intermediate image.

[0032] An adaptive histogram equalization algorithm is applied to the fourth intermediate image to enhance the local contrast of the image by adjusting the histogram of the local area, thereby obtaining a contrast-enhanced image. A non-local mean denoising algorithm is applied to the contrast-enhanced image to effectively suppress noise while retaining image details by searching for similar image blocks in the entire image, thereby obtaining a denoised fifth intermediate image.

[0033] A hybrid data enhancement strategy is applied to the fifth intermediate image to perform enhancement processing, including random rotation, flipping, brightness adjustment and noise addition, to obtain an enhanced image; a masking matrix is ​​constructed and applied to the enhanced image to achieve random masking of part of the information, to obtain a masked enhanced image.

[0034] According to one aspect of the present application, step S11 is further:

[0035] S111, obtaining an original image collected by the drone, and calculating an overall brightness histogram of the original image; based on the overall brightness histogram, calculating an average brightness value and a standard deviation of the original image;

[0036] S112, constructing a target brightness range, and calculating a brightness adjustment factor based on the target brightness range and the average brightness value and standard deviation of the original image;

[0037] S113, adjusting the brightness of each pixel in the original image based on the brightness adjustment factor to obtain a new pixel value;

[0038] S114: Clip the new pixel value to a predetermined range to obtain a first intermediate image after brightness adjustment.

[0039] According to one aspect of the present application, step S12 is further:

[0040] S121, converting the first intermediate image into a logarithmic domain to obtain a logarithmic image; based on the logarithmic image, using bilateral filtering to obtain a base layer;

[0041] S122, based on the base layer and the logarithmic image, a detail layer is calculated; based on the base layer, a compression function is used to calculate a compressed base layer;

[0042] S123, performing image reconstruction based on the compressed base layer, detail layer and predefined detail enhancement factor to obtain a reconstructed image;

[0043] S124, converting the reconstructed image from the logarithmic domain back to the linear domain to obtain a second intermediate image with enhanced details.

[0044] According to one aspect of the present application, step S15 is further:

[0045] S151, dividing the fourth intermediate image into grids of a predetermined size, and calculating a local histogram of each grid;

[0046] S152, based on the local histogram of each pixel in the fourth intermediate image and the surrounding grids, using a bilinear interpolation method, calculate an interpolated pixel value;

[0047] S153, performing contrast limitation on the interpolated pixel values ​​to obtain contrast-limited pixel values; and forming an enhanced image based on the contrast-limited pixel values;

[0048] S154. Based on the enhanced image, a non-local mean denoising algorithm is used to calculate a fifth intermediate image.

[0049] In one embodiment of the present application, the original image I collected by the drone is obtained. raw (size is 5000×3000 pixels), for the original image I raw Apply the adaptive exposure control algorithm (AEC) to obtain the first image I after brightness adjustment. aec , using the dynamic range compression (DRC) algorithm to compress the first image I aec Processing is performed to obtain a second image I with enhanced details drc ; Output image I after adaptive exposure control and dynamic range compression drc .

[0050] The specific processing process of the adaptive exposure control algorithm is as follows: Calculate the original image I raw The overall brightness histogram H of the image is used to calculate the average brightness value μ and standard deviation σ of the image according to the overall brightness histogram H; the target brightness range [L min , L max ], and set it to the normalized value of [0.3, 0.7], and calculate the brightness adjustment factor α = (L target - μ) / σ, where L target is the target brightness, L target=(L min + L max ) / 2; for the original image I raw The brightness of each pixel p in is adjusted to obtain a new pixel value p new = p + α * (p - μ); Clip the adjusted pixel values ​​to the range of [0, 255] to obtain the first image I after brightness adjustment aec .

[0051] The specific processing process of the dynamic range compression algorithm is as follows: aec Convert to logarithmic domain and get logarithmic image I log =log(I aec + 1); for the logarithmic image I log Apply bilateral filtering to obtain the base layer B, B = BilateralFilter(I log , σs, σr) where the parameters σs and σr are used as the spatial and range standard deviations respectively, and BilateralFilter( ) represents the bilateral filter function; the detail layer D = I log - B. Then, the compression function is applied to the base layer B to obtain the compressed base layer B compressed = (B - min(B)) / (max(B) - min(B)) * compression factor , where compression factor Indicates the compression factor; based on the compressed base layer B compressed and detail enhancement factor detail enhancementfactor Reconstructed Image I reconstructed = B compressed + D * detail enhancementfactor ; Convert the reconstructed image from the logarithmic domain back to the linear domain to obtain a second image I with enhanced details drc = exp(I reconstructed ) – 1, where exp( ) represents the exponential function.

[0052] Obtain a second image I with enhanced details drc , for image I drc The improved superpixel segmentation algorithm (ISLIC) is used to process the segmentation results Segments, and the local contrast enhancement (LCE) algorithm is used to process the segmentation results Segments to obtain the enhanced image third image I lce .

[0053] The processing of the superpixel segmentation algorithm is as follows: initialize the cluster center grid with a grid size of S×S (usually S=16); for each cluster center c i , calculate its eigenvector in the Lab (LAB) color space [l i , a i , b i , x i , y i ]; for each pixel p in the image, calculate its distance D = sqrt((l p - l i ) 2 + (a p - a i ) 2 + (b p - b i ) 2 + m 2 / S 2 * ((x p -x i ) 2 + (y p -y i ) 2 ), where m is the weight factor of the spatial distance, l p and l i Represent pixel p and cluster center c respectively i The brightness component in the LAB color space, a p and a i Represent pixel p and cluster center c respectively i The green-red component in the LAB color space, b p and b i Represent pixel p and cluster center c respectively i The blue-yellow component in LAB color space, x p and x i Represent pixel p and cluster center c respectively i The horizontal coordinate in the image, y p and i Represent pixel p and cluster center c respectively i The vertical coordinate in the image; assign the pixel to the cluster center with the smallest distance; update the cluster center to the average of all pixels assigned to it; repeat the above process until convergence or the maximum number of iterations is reached.

[0054] The processing process of the local contrast enhancement algorithm is as follows: for each superpixel area S i , calculate its average brightness μ i and standard deviation σ i; Define contrast enhancement function f(x) = α* (x - μ i ) +μ i , where α is the enhancement factor; for each pixel p in the region, enhancement is applied: new = f(p); smooth transitions are applied at region boundaries to avoid artifacts.

[0055] Get the third image I lce , for image I lce Apply the attention-guided cropping algorithm (AGC) to obtain the optimal cropping area Crop region , from image I lce Crop the image I to 1024×1024 size crop , I crop = I lce [Crop region . top :Crop region . bottom , Crop region . left :Crop region . right ], where Crop region . top Indicates the upper boundary of the cropping area, Crop region . bottom Indicates the lower boundary of the cropping area, Crop region . left Indicates the left boundary of the cropping area, Crop region . right Indicates the right boundary of the cropping area; the content-aware scaling algorithm (CAS) is used to scale the image I crop After processing, a fourth image I with a size of 224×224 is obtained. resize .

[0056] The processing process of the attention-guided cropping algorithm is as follows: the improved Itti-Koch model is used to calculate the saliency map of the image, including extracting color, intensity and direction feature maps, applying center-periphery difference to each feature map, and combining multi-scale feature maps to obtain the final saliency map; a sliding window is used to search for the best cropping area on the final saliency map, the window size is set to 1024×1024, the saliency score is calculated for each window position, and the window with the highest score is selected as the best cropping area. region .

[0057] The processing process of the content-aware scaling algorithm is as follows: construct an image energy function E(I), which includes a brightness gradient term and an edge preservation term; iteratively remove low-energy pixels or pixel columns, including calculating the cumulative energy of each column (row), removing the column (row) with the smallest cumulative energy, and filling the removed area with interpolation; repeat the above process until the image size reaches 224×224.

[0058] Get the fourth image I resize , for image I resize The adaptive histogram equalization (AHE) algorithm is used to process the image and obtain the image with local contrast enhancement. ahe , the non-local mean denoising (NLMD) algorithm is used to denoise I ahe After processing, the denoised image I is obtained nlmd ; Output the denoised fifth image I nlmd .

[0059] The processing process of the adaptive histogram equalization algorithm is as follows: the image is divided into 8×8 grids; the local histogram is calculated for each grid; for each pixel, bilinear interpolation is performed based on the 4 grids around it; and contrast limitation is applied to prevent noise amplification.

[0060] The specific processing process of the non-local means denoising algorithm is as follows: for each pixel p in the image, search for similar image patches in the search window around it; calculate the weighted average between the current image patch and the searched image patch w(p,q) = exp(-||patch(p) - patch(q)|| 2 / h 2 ), where h is the filter parameter, exp() represents the exponential function, q represents a pixel position in the search window, and patch() represents the function of extracting an image patch from the image; replace the current pixel value with the weighted average.

[0061] Get the fifth image I nlmd , for image I nlmd Apply the hybrid data enhancement strategy (MDA) to obtain the enhanced image I aug , construct the Mask matrix M for information shielding, and upsample the Mask matrix M to the same size as the image to obtain the matrix M upsample , for image I aug Application Matrix M upsample Processing is performed to obtain the shielded enhanced image I masked .

[0062] The processing process of the hybrid data enhancement strategy is as follows: randomly apply a combination of the following enhancement methods: random rotation (-15° to 15°), random horizontal and vertical flipping, random brightness, contrast and saturation adjustment (+20% or -20%), random addition of Gaussian noise (σ = 0.01), and random occlusion (maximum 10% of the image area).

[0063] The construction and application process of the mask matrix M is as follows: create a 14×14 binary matrix M; randomly select 75% of the elements and set them to 0, and the rest to 1; use nearest neighbor interpolation to upsample the binary matrix M from 14×14 to 224×224, and obtain the matrix M upsample ; Apply Mask to the enhanced image: I masked = I aug * M upsample .

[0064] This embodiment improves the image quality and feature extraction effect in the adaptive inspection of overhead lines of UAV distribution network through a combination of a series of image processing technologies. The adaptive exposure control processing technology effectively solves the problem of uneven exposure of images taken by UAV under different lighting conditions by dynamically analyzing and adjusting the brightness value of each pixel, ensuring that images with balanced brightness can be obtained in various complex lighting environments such as cloudy days, sunny days, and dusk, laying the foundation for subsequent processing. The application of dynamic range compression algorithm further enhances the image details, especially for the subtle defects of key components such as insulators and hardware in overhead lines, which can be better retained and highlighted, and improves the accuracy of defect detection. The introduction of multi-scale image segmentation and feature enhancement technology enables the system to adaptively process targets of different scales, from large-scale pole tower structures to small-scale line accessories, which can be accurately segmented and enhanced, which is crucial for identifying power equipment in complex backgrounds. The application of adaptive cropping and scaling solves the problem of image size and proportion caused by inconsistent shooting angles and distances of UAVs, ensuring the consistency of subsequent processing. The use of image enhancement and noise suppression technology effectively improves the signal-to-noise ratio of images, especially images taken under adverse weather conditions (such as haze, rain and snow), which can improve their quality and enhance the all-weather adaptability of inspections. The steps of data enhancement and feature extraction preparation increase the model's adaptability to various complex situations, such as partial occlusion, lighting changes, and perspective changes, by introducing random transformations and masking, which is crucial to improving the robustness in actual inspections.

[0065] This embodiment not only improves the image quality, but also provides high-quality input for subsequent feature extraction and analysis, thereby improving the efficiency and accuracy of UAV adaptive inspection of overhead lines in distribution networks.

[0066] like Figure 3As shown, according to one aspect of the present application, step S2 is further:

[0067] S21. Based on the preprocessed original image, an improved ResNet network is used to extract feature maps of predetermined different levels as primary multi-level features;

[0068] S22, sorting feature maps of different levels, selecting the deepest feature map, and using an improved Transformer structure to generate high-level features based on the deepest feature map;

[0069] S23, calling a preconfigured multi-layer hybrid module, upsampling the high-level features to obtain upsampled features; performing hole convolution, downsampling and channel adjustment on the primary multi-level features to obtain features after adjustment of a predetermined group; fusing the features after adjustment of the predetermined group with the upsampled features to obtain fused features of a predetermined layer;

[0070] S24, integrating the fusion features of different layers to obtain integrated multi-level mixed features.

[0071] According to one aspect of the present application, step S22 is further:

[0072] S221, sorting feature maps of different levels, and selecting the deepest feature map;

[0073] S222, reshape the deepest level feature map into a sequence form, and calculate the query, key and value matrices based on the deepest level feature map in the sequence form using a multi-head self-attention mechanism;

[0074] S223, calculating an attention score based on the query, the key and the value matrix; forming a weighted feature based on the attention score and the value matrix;

[0075] S224. Based on the weighted features, a feedforward neural network is used to generate a nonlinear transformation feature representation; based on the nonlinear transformation feature representation, residual connection and layer normalization are used to generate high-level features.

[0076] In this embodiment, an improved convolutional neural network (ResNet) is used to process the masked enhanced image, extract feature maps of multiple scales, and obtain primary multi-level features; the deepest primary features are processed by an improved Transformer structure, and the long-range dependencies in the feature maps are captured through the self-attention mechanism to generate high-level features with a global receptive field; a multi-layer hybrid module is constructed and applied to fuse the primary multi-level features and the high-level features to obtain the final multi-level hybrid features, which are specifically:

[0077] Read the masked enhanced image; input the masked enhanced image into the improved ResNet network, which extracts image features layer by layer through multiple downsampling modules. Specifically, in the five downsampling modules of ResNet, three feature maps of different scales are extracted respectively, including the third feature map, the fourth feature map and the fifth feature map; output three feature maps of different scales as primary multi-level features, where the third feature map has a higher spatial resolution but shallower semantic information, the fourth feature map is centered in spatial resolution and semantic information, and the fifth feature map has a lower spatial resolution but deeper semantic information. Read the fifth feature map and reshape it into a sequence form; apply the multi-head self-attention mechanism, calculate the correlation between features, including query, key and value matrices, calculate the attention score, and obtain weighted features; apply the feedforward neural network to process the weighted features; add residual connections and layer normalization to obtain high-level features with global receptive field after Transformer processing.

[0078] Read the third feature map, the fourth feature map, and the high-level features; construct and apply a multi-layer hybrid module to fuse these features, specifically: upsample the high-level features to match their spatial resolution with the fourth feature map; apply dilated convolution to the fourth feature map to increase its receptive field; downsample and adjust the channels of the third feature map; add the three processed feature maps at the element level to obtain the first layer of fused features. Use transposed convolution to upsample the high-level features to obtain the first upsampled features; apply dilated convolution to process the third feature map to obtain the first dilated convolution features; fuse the first upsampled features, the first dilated convolution features, and the original third feature map to obtain the second layer of fused features; use the high-level features as the third layer of fused features. The first-layer fusion features, the second-layer fusion features, and the third-layer fusion features are finally integrated; the spatial resolution and number of channels of the feature map are adjusted to make its consistency meet the requirements, and the final integrated multi-level mixed features are output, including the first-level features, the second-level features, and the third-level features; the integrated multi-level mixed features contain everything from low-level edge information to high-level semantic information, which retains both details and global context.

[0079] In another embodiment of the present application, the shielded enhanced image I is obtained. masked , for the enhanced image I masked Apply the improved ResNet backbone network for processing to obtain multi-level features c3, c4, and c5; obtain feature c5, apply the improved Transformer structure to feature c5, and obtain feature s5 after Transformer processing; obtain features c3, c4, and s5, use a multi-layer hybrid module to fuse c3, c4, and s5, and obtain multi-level hybrid features s3, s4, and s5.

[0080] The specific processing process of the ResNet backbone network is as follows: masked Input into the improved ResNet network; in the 5 downsampling modules of ResNet, 3 feature maps of different scales are extracted respectively: the first scale feature map c3 = ResNet Block3 (I masked ), size is 28×28×256; the second scale feature map c4 = ResNet Block4 (c3), size is 14×14×512; third scale feature map c5 = ResNet Block5 (c4), size is 7×7×1024, where ResNet Block3 、ResNet Block4 and ResNet Block5 Represents different downsampling modules in the ResNet network, which are used to extract feature maps of different scales.

[0081] The Transformer structure processing process is as follows: reshape the feature map c5 into a sequence form: x = Reshape(c5, [49, 1024]), where Reshape() represents a reshaping operation, converting the feature map from a two-dimensional form to a one-dimensional sequence form; apply a multi-head self-attention mechanism to calculate the query, key, and value matrices: Q, K, V = LinearProjection(x), where LinearProjection() represents a linear projection operation, mapping the input features to the query Q, key K, and value V matrices; calculate the attention score: Attention = Softmax(QK T / sqrt(dk)), where dk represents the dimension of the query and key matrices, which is used to scale the dot product attention score; get the weighted feature: z = Attention * V; apply the feedforward neural network: FFN(z) = ReLU(Linear1(z)) + Linear2(ReLU(Linear1(z))), where ReLU( ) represents the activation function, Linear1( ) and Linear2( ) represent the fully connected layers in the feedforward neural network for further processing of features; add residual connections and layer normalization: z1 = LayerNorm(x + z), z2 = LayerNorm(z1 + FFN(z1)), where LayerNorm( ) represents the layer normalization operation.

[0082] The specific processing process of the multi-layer hybrid module is as follows: Process feature s5: Use transposed convolution to upsample feature s5 to obtain feature c41 = TransposedConv(s5, stride=2), with a size of 14×14×512, where TransposedConv( ) represents transposed convolution (also called deconvolution), which is used to upsample the feature map, and stride represents the stride;

[0083] Processing feature c4: Apply dilated convolution to obtain c42 = DilatedConv(c4, rate=2), with a size of 14×14×512, where DilatedConv( ) represents dilated convolution, which is used to expand the receptive field without increasing the number of parameters, and rate represents the dilation rate;

[0084] Process feature c3: Apply average pooling to obtain c43 = AvgPool(c3), with a size of 14×14×256. Use 1x1 convolution to adjust the number of channels to obtain c43 = Conv1x1(c43, outchannels=512), where AvgPool() represents the average pooling operation for downsampling feature maps, Conv1x1() represents 1×1 convolution, which is used to adjust the number of channels of feature maps, and outchannels represents the number of channels of the output feature map.

[0085] Fusion features: s4 = c41 + c42 + c43, with size of 14×14×512.

[0086] Repeat a similar process to generate s3 features, specifically: use transposed convolution to upsample feature s5 to obtain feature c31 = TransposedConv(s5, stride=4), with a size of 28×28×256;

[0087] Apply dilated convolution to feature c3: c32 = DilatedConv(c3, rate=2), with a size of 28×28×256;

[0088] Fusion feature: s3 = c31 + c32 + c3, size is 28×28×256.

[0089] Output multi-level mixed features s3, s4, and s5, which have the following characteristics respectively: s3: size is 28×28×256, and the receptive field is 1 / 8 of the original image; s4: size is 14×14×512, and the receptive field is 1 / 16 of the original image; s5: size is 7×7×1024, and the receptive field is 1 / 32 of the original image.

[0090] This embodiment combines the improved convolutional neural network and the Transformer structure to achieve efficient extraction and fusion of multi-level features, and improves the feature representation capability and computational efficiency in the adaptive inspection of overhead lines of UAV distribution networks. The improved convolutional neural network can extract multi-scale feature maps from the shielded enhanced image, fully considering the multi-scale characteristics of the target in the power line inspection, and can effectively represent everything from large-scale tower structures to small-scale insulator defects. In particular, by extracting feature maps of three different scales, the system can pay attention to the macroscopic line layout and microscopic component details at the same time, improving the comprehensiveness and accuracy of the inspection. When the improved Transformer structure is applied to the deepest features, the long-range dependencies between features can be captured through the self-attention mechanism, which is crucial for understanding complex power line scenarios. For example, when identifying abnormal sag of conductors or evaluating the inclination of towers, it is necessary to consider the relationship of the overall structure, not just the local features, and the Transformer structure just provides a global receptive field. The construction and application of multi-layer hybrid modules further enhance the expressiveness of features. By fusing features at different levels, the system can simultaneously utilize low-level detail information and high-level semantic information, which is essential for target detection and defect recognition in complex scenarios. For example, when identifying tiny cracks on the surface of an insulator, it is necessary to combine low-level texture information with high-level semantic understanding. In addition, through operations such as upsampling, dilated convolution, and channel adjustment, the problems of feature resolution mismatch and insufficient receptive field can be effectively solved, ensuring that important information is not lost during the fusion process. The multi-level and multi-scale feature fusion method enhances the model's ability to understand complex power line scenes and can better cope with various challenges encountered in actual inspections, such as partial occlusion, illumination changes, and perspective changes. This embodiment improves the expressiveness and robustness of features through network structure and feature fusion strategy, and provides high-quality feature input for subsequent target detection, defect recognition, and state assessment, thereby improving the accuracy and efficiency of adaptive inspection of overhead lines of UAV distribution networks.

[0091] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0092] S31. Based on the multi-level mixed features, a two-dimensional random matrix is ​​constructed, and each element in the two-dimensional random matrix is ​​randomly initialized using uniform distribution;

[0093] S32, sorting each row in the two-dimensional random matrix in ascending order, and recording the sorted index; constructing a Mask matrix based on the sorted index and the predefined occlusion rate;

[0094] S33, adjusting the shape of the Mask matrix based on the multi-level mixed features to obtain a Mask matrix after the shape is adjusted;

[0095] S34. Multiply the mixed features of each level in the multi-level mixed features by the corresponding mask matrix after adjusting the shape to obtain the multi-level occlusion features.

[0096] This embodiment constructs a two-dimensional random matrix for multi-level mixed features; each element in the two-dimensional random matrix is ​​randomly initialized using uniform distribution so that its element value is between 0 and 1; the number of rows in the two-dimensional random matrix is ​​equal to the batch size to be processed, and the number of columns is equal to the spatial dimension of the feature map (width multiplied by height). Sort each row of the random matrix in ascending order and record the sorted index; create an all-1 matrix of the same size as the random matrix; according to the predefined occlusion rate (for example, N*%), set the elements in the all-1 matrix corresponding to the first N*% of the sorted index to 0, so that the binary matrix obtained is the Mask matrix, which indicates which positions of the features should be blocked, where N* is a natural number, generally 75. Adjust the Mask matrix to a shape that matches the feature map of each level in the multi-level mixed features; for each level of the feature map, multiply it with the corresponding Mask matrix to achieve partial occlusion of the feature; obtain three blocked feature maps, corresponding to the original three levels of mixed features.

[0097] In another embodiment of the present application, multi-level mixed features s3, s4, and s5 are obtained, a random matrix R is constructed, an ascending sorting process is applied to the random matrix R to obtain a sorting index I, and a Mask matrix M is generated based on the sorting index I; the Mask matrix M is applied to the multi-level mixed features s3, s4, and s5 to obtain the masked feature s3 masked 、s4 masked 、s5 masked .

[0098] The construction of the random matrix R and the generation process of the mask matrix M are as follows: create a random matrix R of size N×L, where N is the batch size (for example, 512) and L is the number of pixels (196, corresponding to 14×14); initialize R using uniform distribution: R = Uniform(0, 1, shape=[N, L]), where Uniform() represents the uniform distribution initialization function and shape represents the shape parameter; sort each row in R in ascending order to obtain the sorting index I; create an all-1 matrix M* with the same size as the random matrix R; set the occlusion rate mask ratio (e.g. 0.75); set the first L * mask in the all-1 matrix M* ratio elements (according to the order of I) are set to 0.

[0099] The specific process of applying the Mask matrix is ​​as follows: Adjust the Mask matrix M to the same shape as feature s4: M reshaped=Reshape(M, [N, 14, 14, 1]); Mask feature s4: s4 masked = s4 * M reshaped ; resize s3 and s5 accordingly and apply occlusion: s3 masked = s3 * Upsample(M reshaped , scale factor =2); s5 masked = s5*AvgPool(M reshaped , kernel size =2, stride=2), where Upsample() represents the upsampling operation, scale factor Indicates the upsampling multiple parameter, scale factor =2 means doubling the size of the matrix, M reshaped Represents the Mask matrix after adjusting the shape, kernel size Indicates the size parameter of the pooling window.

[0100] This embodiment improves the robustness and generalization ability of the UAV distribution network overhead line adaptive inspection system by introducing random occlusion and Mask matrix generation technology. The method of constructing a random matrix cleverly introduces controllable randomness, which simulates various occlusion and interference situations that may be encountered in actual inspections, such as tree branch occlusion, shadow interference, or local image quality degradation. By using uniformly distributed random initialization matrix elements, the diversity and balance of occlusion patterns are ensured, and overfitting of specific patterns is avoided. In particular, the number of rows of the random matrix is ​​set to the batch size, and the number of columns is set to the spatial dimension of the feature map, so that each sample has a unique occlusion pattern, which increases the diversity of training data. The subsequent sorting process and Mask matrix generation steps achieve partial occlusion of features by setting a predefined occlusion rate. The partial occlusion mechanism simulates the common partial information missing situation in actual inspections, such as partial equipment invisible due to angle problems, or local image blurring due to weather reasons. By forcing the model to learn to infer global features from partial information, the robustness and generalization ability of the model are improved. For example, even if part of the insulator or conductor is obscured, the model can make correct judgments based on the visible part and contextual information. Applying the Mask matrix to the multi-level hybrid features ensures the consistency of the occlusion effect at different scales and semantic levels, which is of great significance for processing multi-scale targets (from large poles to small hardware) in power line inspections. In addition, the occlusion mechanism can also be regarded as a regularization technique, which increases the difficulty of the task and prevents the model from over-relying on certain specific features, thereby improving its performance in complex actual environments. In practical applications, the system's adaptability to partial occlusion, local damage or missing information can be effectively improved. For example, even when some key components are obscured, the system can still make an accurate assessment of the line status. This embodiment enhances the robustness and generalization ability of the model by introducing an intelligent random occlusion mechanism, so that the UAV distribution network overhead line adaptive inspection system can better cope with various complex situations in the actual environment, and improve the reliability and adaptability of the inspection.

[0101] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0102] S41, constructing an autoencoder model based on each level occlusion feature in the multi-level occlusion features, including an encoder and a decoder;

[0103] S42, inputting each level of occlusion feature into the encoder to obtain a compressed potential representation; inputting the potential representation into the decoder to obtain a reconstructed feature map;

[0104] S43, calculating the difference between the reconstructed feature map and the multi-level mixed features, and obtaining the reconstruction loss of the multi-layer pixels;

[0105] S44, using a gradient descent algorithm to calculate the gradient of the reconstruction loss with respect to the parameters of the autoencoder model; based on the gradient, using an optimizer to update the parameters of the autoencoder model;

[0106] S45. Repeat steps S42 to S44 until a predefined number of iterations is reached, and output the trained autoencoder model and the final reconstruction loss.

[0107] According to one aspect of the present application, step S43 is further:

[0108] S431, calculating the mean square error of a predetermined number of levels based on the reconstructed feature map and the multi-level mixed features;

[0109] S432: Perform weighted summation on the mean square errors of the predetermined levels to obtain reconstruction losses of multiple layers of pixels.

[0110] In this embodiment, the multi-level features are processed by the autoencoder structure to obtain the reconstructed features; the reconstructed features are compared with the original multi-level mixed features to calculate the multi-level pixel reconstruction loss; the model parameters are updated based on the loss; the above process is repeated until the termination condition is met to obtain the trained autoencoder model parameters and the final reconstruction loss, specifically: based on the multi-level features after occlusion, an independent autoencoder is constructed for the feature map of each level, including the encoder and decoder parts; the encoder compresses the input features into a latent space, and the decoder reconstructs the original features from this latent space. The occluded feature map of each level is processed by the corresponding autoencoder; the occluded feature map is input into the encoder to obtain the compressed latent representation; the latent representation is input into the decoder to obtain the reconstructed feature map, and this process will generate three reconstructed feature maps, corresponding to the original three-level mixed features. The difference between the reconstructed feature map and the original feature map is calculated to obtain the reconstruction loss; for each level, the mean square error between the reconstructed feature map and the original feature map is calculated to obtain the reconstruction loss of the three levels; the three losses are weighted summed to obtain the total reconstruction loss. Based on the total reconstruction loss, the model parameters of the autoencoder are updated using the gradient descent algorithm; the gradient of the loss to each model parameter is calculated; the model parameters are updated according to these gradients using the optimizer. The above steps are repeated until the predefined number of iterations is reached or the convergence condition is met; in each iteration, the learning rate is fine-tuned; when the training is completed, the final autoencoder model parameters and the loss curve during the training process are output.

[0111] In another embodiment of the present application, the feature s3 after the occlusion in the acquisition step masked 、s4 masked 、s5 masked, apply the autoencoder structure to these features respectively, obtain the reconstructed features o3, o4 and o5, calculate the mean square error loss between the reconstructed features o3, o4, o5 and the original features s3, s4, s5, and obtain the multi-layer pixel reconstruction loss L3, L4, L5; calculate the weighted total loss L total , for the weighted total loss L total Apply gradient descent and optimizer to update model parameters; output updated model parameters θ new and the final reconstruction loss L total Repeat the above steps until the predefined number of iterations is reached or the convergence condition is met, and the trained autoencoder model parameters θ are obtained. trained And the final reconstruction loss L final .

[0112] The specific process of the autoencoder is as follows: create encoder-decoder pairs for features s3, s4, and s5 respectively:

[0113] E3, D3 = CreateAutoencoder(input shape =s3.shape, latent dim =256);

[0114] E4, D4 = CreateAutoencoder(input shape =s4.shape, latent dim =512);

[0115] E5, D5 = CreateAutoencoder(input shape =s5.shape, latent dim =1024);

[0116] CreateAutoencoder() represents a function used to create an autoencoder model. shape Represents the shape of the input feature, latent dim represents the dimension of the latent space, E3 and D3 represent the encoder and decoder of the autoencoder, respectively.

[0117] Encode and decode occluded features:

[0118] o3 = D3(E3(s3 masked )); o4 = D4(E4(s4 masked ));o5 = D5(E5(s5 masked )).

[0119] The specific calculation process of multi-layer pixel reconstruction loss is as follows: Calculate the mean square error loss of each layer:

[0120] L3 = MSE(o3, s3); L4 = MSE(o4, s4); L5 = MSE(o5, s5); where MSE( ) represents the mean square error loss function;

[0121] Calculate the weighted total loss: L total = w3*L3 + w4*L4 + w5*L5, where w3, w4, w5 are predefined weights; calculate the gradient: grads = ▽θ L total ; where ▽ represents the gradient symbol and θ represents the model parameters; use the optimizer to update the model parameters: θ new = Optimizer(θ, grads), where Optimizer( ) represents the optimizer function; the above process is repeated until the predefined number of iterations is reached or the convergence condition is met.

[0122] This embodiment realizes an efficient self-supervised learning mechanism by constructing and applying multi-layer pixel reconstruction and autoencoder structures, and improves the feature learning ability and model performance of the UAV distribution network overhead line adaptive inspection system. An independent autoencoder structure is constructed for the feature map of each level, which fully considers the multi-scale and multi-level characteristics of the target in the power line inspection. For example, low-level features may correspond to texture and edge information in the line, middle-level features may represent the shape and structure of the component, and high-level features may contain more abstract semantic information. By constructing a dedicated autoencoder for each level, the system can more accurately learn and reconstruct these different levels of features, thereby improving the ability to understand the state of complex power equipment and lines. The process of inputting the occluded feature map into the encoder, obtaining the compressed potential representation, and then reconstructing the original feature through the decoder is actually a self-supervised learning mechanism. The mechanism forces the model to learn how to infer and reconstruct complete features from partial information, which is of great significance for dealing with common problems such as occlusion and partial visibility in actual inspections. For example, even if some key components are partially occluded, the system can infer the complete state based on the context and visible parts. Computing the difference between the reconstructed feature map and the original feature map and using the mean squared error as the loss function provides a clear learning goal for the model. By minimizing this reconstruction loss, the model is forced to learn to capture the essential features and structure of the data rather than simply memorizing surface features. This learning method is particularly suitable for tasks such as power line inspection, which have clear structures and regularities. The method of weighted summing the reconstruction losses of the three levels to obtain the total reconstruction loss ensures that the model can accurately reconstruct features at different scales and levels of abstraction, which is crucial for a comprehensive understanding of complex power line scenarios. The process of updating the model parameters of the autoencoder using the gradient descent algorithm based on the total reconstruction loss realizes the automatic optimization of the model parameters. The end-to-end training method enables the model to adaptively learn the most effective feature representation without the need for manual feature design. Through multiple iterations of training and fine-tuning the learning rate, the model is able to gradually improve its feature extraction and reconstruction capabilities, and finally reach a balance point that can effectively compress information while accurately reconstructing the original features. An important effect of this process is that the model learns how to extract the most critical features from limited or partially damaged information, which is crucial for improving fault tolerance and robustness in actual inspections. For example, the system can still make accurate judgments even in certain situations where the camera angle is poor or the image quality is degraded.

[0123] This embodiment improves the model's ability to understand and express features of power line scenarios through self-supervised learning and multi-layer feature reconstruction, laying a solid foundation for subsequent target detection, defect identification, and status assessment tasks, thereby greatly improving the accuracy, reliability, and adaptability of UAV adaptive inspection of overhead lines in distribution networks.

[0124] In another embodiment of the present application, a self-supervised learning method can also be used to calculate the pixel reconstruction loss, specifically: read the multi-level mixed features s3 (28×28×256), s4 (14×14×512) and s5 (7×7×1024). Construct and apply the autoencoder structure to process the multi-level features: Process feature s3:

[0125] Encoder: First, perform a two-dimensional convolution operation on the feature map s3: E3=Conv2D(s3, filters=512), where Conv2D represents two-dimensional convolution and filters represents the number of channels of the output feature map; then perform ReLU activation and perform a two-dimensional convolution operation again, and the number of channels of the output feature map is 768: Conv2D(filters=768);

[0126] Decoder: First, perform a two-dimensional convolution operation on the encoder output E3: D3 = Conv2D(E3, filters=512), then perform ReLU activation, and perform a two-dimensional convolution operation again: Conv2D(filters=3);

[0127] Get the reconstructed feature o3 = Reshape(D3, [512, 784, 768]), where 784 = 28 * 28;

[0128] Process feature s4:

[0129] Encoder: Perform a two-dimensional convolution operation on feature map s4: E4 = Conv2D(s4, filters=768); then perform ReLU activation and perform another two-dimensional convolution operation: Conv2D(filters=768);

[0130] Decoder: First, perform a two-dimensional convolution operation on the encoder output E4: D4 = Conv2D(E4, filters=512); then perform ReLU activation and perform a two-dimensional convolution operation again: Conv2D(filters=3);

[0131] Get the reconstructed feature o4 = Reshape(D4, [512, 196, 768]), where 196 = 14 * 14;

[0132] Process feature s5:

[0133] Encoder: First perform a two-dimensional convolution operation on feature map s5: E5 = Conv2D(s5, filters=1024); then perform ReLU activation and perform another two-dimensional convolution operation: Conv2D(filters=768);

[0134] Decoder: First, perform a two-dimensional convolution operation on the encoder output E5: D5 = Conv2D(E5, filters=512); then perform ReLU activation and perform another two-dimensional convolution operation: Conv2D(filters=3);

[0135] The reconstructed feature o5 = Reshape(D5, [512, 49, 768]) is obtained, where 49 = 7 * 7.

[0136] Read the original enhanced image I aug (224×224×3), construct the target t4, and transform the image I aug Reshape into a tensor t4 of size 512×196×768. Construct target t3 for image I aug Perform bilinear interpolation upsampling: I up = Upsample(I aug , scale factor =2); the upsampled image I up Reshape into a tensor t3 of size 512×784×768. Construct target t5 for image I aug Perform average pooling downsampling: I down = AvgPool(I aug , kernel size =2, stride=2); the downsampled image I down Reshape into tensor t5 of size 512×49×768.

[0137] The target images t3, t4, and t5 are standardized respectively: t3 norm = (t3 - Mean(t3)) / Std(t3);t4 norm = (t4 - Mean(t4)) / Std(t4);t5 norm = (t5 - Mean(t5)) / Std(t5), where Mean( ) represents the function for calculating the mean, and Std( ) represents the function for calculating the standard deviation.

[0138] Calculate the multi-level pixel reconstruction loss: Calculate the L1 loss: L1 = MSE(o3, t3 norm ); Calculate L2 loss: L2 =MSE(o4, t4 norm ); Calculate L3 loss: L3 = MSE(o5, t5 norm); Calculate the weighted total loss: L = w1*L1 + w2*L2+ w3*L3, where w1, w2, w3 are predefined weights that can be adjusted based on experimental results.

[0139] Apply the gradient descent optimization algorithm to update the model parameters: Calculate the gradient of the loss L with respect to the model parameter θ: ▽θ = ΨL / Ψθ, where Ψ represents the partial derivative; Use the Adam optimizer to update the parameters: θ new = Adam(θ, ▽θ).

[0140] Repeat the above steps until the predefined number of iterations or convergence condition is reached, and output the trained model parameters θ trained And the final reconstruction loss L final .

[0141] This embodiment performs pixel reconstruction and self-supervised learning through multi-level mixed features. The self-supervised learning process allows the model to learn how to reconstruct a complete image from partially occluded inputs, thereby improving the model's ability to understand and characterize distribution network overhead line images.

[0142] According to another aspect of the present application, feature extraction and downstream task adaptation are also included. trained Initialize the weights of the feature extractor and freeze all layer parameters except the last few layers. Read the distribution network overhead line image dataset D downstream ; For the image dataset D downstream I for each image in i Apply the preprocessing steps in step S1, including cropping, scaling, enhancement, etc., to obtain the preprocessed image I iprocessed For each processed image I iprocessed Generate features using feature extractor: f i3 , f i4 , f i5 = FeatureExtractor(I iprocessed ), where f i3 , f i4 , f i5 Corresponding to the features of s3, s4, and s5 levels respectively. For the target detection task, build a feature pyramid network (FPN): FPN =BuildFPN(f i3 , f i4 , f i5 ); Add the region proposal network (RPN) and detection head: Detector = BuildDetector(FPN);

[0143] For the semantic segmentation task, build a decoder network: Decoder = BuildDecoder(fi3 , f i4 , f i5 ); add segmentation head: Segmenter = BuildSegmenter(Decoder), where FeatureExtractor() represents the feature extractor function, BuildFPN() represents the function of building a feature pyramid network, which is used to fuse features at different levels for better target detection or semantic segmentation tasks, BuildDetector() represents the function of adding a region proposal network and detection head; BuildDecoder() represents the function of building a decoder network; BuildSegmenter() represents the function of adding a segmentation head.

[0144] Define task-specific loss functions: For object detection: L det = L cls + L reg + L rpn ; For semantic segmentation: L seg = CrossEntropyLoss(pred, gt); Use mini-batch gradient descent to optimize the model, where L det Represents the loss function of the target detection task, including the classification loss L cls , regression loss L reg and the loss L of the region proposal network rpn , L seg represents the loss function of the semantic segmentation task, CrossEntropyLoss() represents the cross entropy loss function, pred represents the prediction result of the model, and gt represents the true value.

[0145] Evaluate model performance on the validation set: For object detection, calculate the mean average precision (mAP); for semantic segmentation, calculate the mean intersection over union (mIoU); if the performance does not meet the requirements, fine-tune the latter layers of the feature extractor and the task-specific layers, and output the final downstream task model and evaluation indicators.

[0146] Read the final downstream task model, apply weights to quantize the model and optimize the computational graph; deploy the model to the drone system: convert the model to a format suitable for edge devices (such as ONNX or TensorRT); load the converted model into the drone's computing unit. Get real-time image stream from the drone camera I stream ; Use the preprocessing steps in step S1 to process each frame of image I frame , get the preprocessed image I processed; Perform model inference based on preprocessed images and post-process the inference results (such as NMS, threshold filtering, etc.). Collect confidence statistics of the inference results. If the confidence is lower than the threshold, trigger adaptive updates; Regularly synchronize the updated model to the central server and output real-time detection or segmentation results and the updated model.

[0147] like Figure 6 As shown, according to another aspect of the present application, based on the self-supervised pre-training method in the adaptive inspection of overhead lines of UAV distribution networks, the overall content of the method is as follows: output three feature layers c3, c4 and c5 through the backbone network of the convolutional neural network (CNN), wherein c3, c4 and c5 are multi-level features (with 8, 16, and 32 times down-sampling receptive fields, respectively). Straighten feature c5 into a sequence and fuse it with the Transformer structure to output feature s5. Unlike the CNN structure, the self-attention mechanism in the Transformer pays more attention to the semantic information of the feature rather than the local details of the space. Therefore, feature s5 has richer semantic features. Construct a random Mask matrix to randomly block part of the information of the input image. Since the Backbone network of the model uses the CNN structure, in order to prevent information leakage, it is necessary to perform a Mask operation on the preprocessed image. By using the constructed multi-level hybrid module, the features of the c3, c4 and s5 layers are fused to generate s3, s4 and s5 features, and the s3, s4 and s5 are processed respectively through the autoencoder structure to complete the multi-scale pixel reconstruction. Since the features integrate the characteristics of the CNN and Transformer structures, the model can effectively understand and represent the data during the multi-scale pixel reconstruction, thus improving the generalization ability of the adaptive inspection model.

[0148] In another embodiment of the present application, the self-supervised pre-training method in the adaptive inspection of overhead lines of a UAV distribution network is specifically performed as follows:

[0149] Step 1: Randomly crop and resize the input image of the model, build a Mask matrix for information shielding, use the ResNet network to extract features, and generate attention semantic features based on the Transformer structure. Through the multi-layer hybrid module, feature fusion and multi-scale pixel reconstruction are performed to realize the processing and reconstruction of the UAV distribution line image;

[0150] Step 2: By fusing the CNN and Transformer structures and building a multi-layer hybrid module, multi-layer features are generated to improve the generalization ability and pixel reconstruction quality of the adaptive inspection model;

[0151] Step 3: After image preprocessing, a random masking operation is performed to construct a random matrix and generate a Mask matrix according to the ascending ranking to shield the information of the input image, avoid information leakage, and ensure that the model can effectively understand and represent the data;

[0152] Step 4: Construct a multi-layer pixel reconstruction loss and use an autoencoder structure to enable the multi-level model to understand and represent data in the pixel reconstruction task and improve the quality of pixel reconstruction.

[0153] Specifically, the main contents of the model structure in step 1 include:

[0154] S11. The images collected by the UAV distribution network line are mostly large-size images of 5000×3000 pixels. The input images are randomly cropped to 1024×1024 size and resized to 224×224 size.

[0155] S12, using ResNet as the model backbone structure, adding the Transformer structure, and the multi-level hybrid module constructed by the present invention, based on the autoencoder framework of the masked autoencoder (MAE), constructing a Mask matrix to randomly block the original input image, and generating multi-level hybrid features by the model to complete multi-layer pixel reconstruction. The specific process is as follows:

[0156] 1. Construct a Mask matrix. According to the output features of the multi-layer hybrid module (size is b×14×14×256), a Mask matrix of size 14×14 is constructed. The Mask operation is performed after the image is preprocessed. The Mask matrix shields 75% of the pixel information, effectively preventing information leakage caused by the CNN structure;

[0157] 2. ResNet is the backbone structure of the model, which consists of 5 downsampling modules with a step size of 2, generating three output features: c3 (b, 28, 28, 256), c4 (b, 14, 14, 256), and c5 (b, 7, 7, 256);

[0158] 3. Input feature c5 with richer semantic features into the Transformer structure, and generate feature s5 by the self-attention mechanism. The self-attention mechanism pays more attention to the semantic information of the whole feature, which helps the model reconstruct pixels.

[0159] 4. Input features c3, c4 and s5 together into a multi-layer hybrid module to generate features s3, s4 and s5. The multi-layer hybrid module consists of deconvolution, dilated convolution and average pooling. Feature s5 is deconvolved to align the feature size with features c3 and c4, and is added to features c3 and c4, integrating the Transformer structure characteristics into features c3 and c4.

[0160] 5. Input features s3, s4 and s5 into the autoencoder structure respectively, and calculate the L2 loss with the target constructed by the image pixels to complete the task of pixel reconstruction, thereby enhancing the model's ability to understand and represent the features of multi-scale images.

[0161] In step 2, in order to generate multi-level features, CNN and transformer structures are fused and multi-layer hybrid modules are constructed so that the multi-level model can effectively understand and represent data and improve the generalization ability of the adaptive inspection model. Specifically:

[0162] S21. Input feature (b×14×14×256) is fed into ResNet to generate multi-level features c3(b, 28, 28, 256), c4(b, 14, 14, 256), c5(b, 7, 7, 256) (with 8, 16, 32 times downsampled receptive fields respectively);

[0163] S22, input with richer semantic features into the Transformer structure, and generate feature s5 by self-attention. Since the self-attention mechanism pays more attention to the overall semantics of the feature, it builds the dependency relationship of the feature semantics and gives feature s5 a global receptive field, which helps the model reconstruct pixels: qkv=linear(x, embed dims *3); q, k, v = qkv[0], qkv[1], qkv[2]; Attention(q, k, v)=softmax(qk T / sqrt(d))v; where x represents feature c5, embed dims represents the dimension of the output feature (256 in the present invention), qkv represents the feature output by c5 through the fully connected layer, q, k, v represent the input query, key, and value of the self-attention module respectively, the constant d=32, and Attention() represents the calculation process of self-attention. First, the inner product of qk is calculated. At the same time, in order to prevent the inner product from being too large, it is divided by the square root of d, and the inner product is normalized by softmax to generate self-attention, and finally multiplied by value;

[0164] S23. In order to fully integrate the multi-level features of CNN with the structural characteristics of Transformer, a multi-layer hybrid module is constructed: feature s5 is sent to the deconvolution with a stride of 2 and a convolution kernel size of 3×3 to generate a feature c41 with a size of 14×14. Feature c41 inherits the characteristics of feature s5 and has a global receptive field; feature c4 uses a dilated convolution with a dilated rate of 2 and a convolution kernel of 3×3 to generate a feature c42 with a size of 14×14. Since the holes separate the pixels, its features can capture more extensive contextual information; based on the average pooling operation, the size of feature c3 is reduced to generate a feature c43 with a size of 14×14, and the features c41, c42, and c43 of the same size are added to obtain feature s4. Similar to the acquisition method of feature s4, s3 is obtained by adding multiple features of the same size: feature s5 is sent as input to the deconvolution with a stride of 4 and a convolution kernel of 3×3 to generate feature c31 of size 28×28; feature c3 is used as input, and a dilated convolution with a dilation rate of 2 and a convolution kernel of 3×3 is used to generate feature c32; features c31, c32, and c3 of the same size are added to obtain feature s3. The multi-layer hybrid module can effectively combine the structural characteristics of CNN and Transformer, and the generated features s3, s4, and s5 all have global receptive fields, which improves the quality of pixel reconstruction.

[0165] Step 3: When performing pixel reconstruction tasks, the input image needs to be randomly masked. Since information in the CNN structure is transmitted during convolution, in order to avoid information leakage and the model's inability to effectively understand and represent data, the present invention chooses to perform a Mask operation after image preprocessing:

[0166] S31, construct a random matrix with a size of N×L, where N is the number of pictures (set to 512 in the present invention) and L is the number of pixels (set to 196 in the present invention);

[0167] S32, sort the random matrix in ascending order to obtain the index of the random matrix after ascending. Then sort the ascending index in ascending order to obtain the ascending ranking corresponding to each element of the random matrix. The ascending ranking records the specific positions that need to be retained and shielded;

[0168] S33, construct a Mask matrix with the same size as the random matrix, and all elements of the Mask matrix are set to 1. The occlusion rate is set to 75%, so the first 147 elements of the 196 elements of the Mask matrix are set to 0. Finally, the ascending ranking corresponding to each element of the random matrix is ​​used as an index to obtain the value in the Mask matrix to obtain the final Mask matrix;

[0169] S34, the size of the Mask matrix is ​​512×196, and the corresponding 2D size can be regarded as 512×14×14. Since the present invention directly performs the Mask operation after image preprocessing, the size of the input image is 512×224×224×3. Therefore, it is necessary to upsample the Mask matrix by 16 times, that is, upsample the size of the Mask matrix to 512×224×224. Finally, the Mask matrix is ​​multiplied by the features after image preprocessing to complete the operation of shielding 75% of the input features.

[0170] Step 4: In order to enable the multi-level model to gain the ability to understand and represent data in the pixel reconstruction task, a multi-layer pixel reconstruction loss is constructed, specifically:

[0171] S41, features s3, s4, and s5 pass through the autoencoder structure to output features o3 (size is 512×784×768), o4 (size is 512×196×768), and o5 (size is 512×49×768);

[0172] S42, construct the target t4 corresponding to the feature o4, and change the dimension of the input image with a size of 512×224×224×3 to a size of 512×196×768;

[0173] S43, interpolate and upsample the input image to generate an input feature of size 512×448×448×3, modify the input feature dimension, and generate target t3 (size 512×784×768). Average pool the input image and downsample it to an input feature of size 512×112×112×3, and modify its dimension to generate target t5 (size 512×49×768);

[0174] S44, standardize the targets t3, t4, and t5 respectively:

[0175] m=mean(t,dim=-1); v=var(t,dim=-1);t=(tm) / (sqrt(v+1.e -6 ));

[0176] Where t represents the target t3, t4, t5, mean() represents the mean of the target t in the last dimension, var() represents the standard deviation of the target t in the last dimension, m represents the mean of the target t in the last dimension, v represents the variance of the target t in the last dimension, dim represents the dimension, 1.e -6 Represents a small constant used to prevent division by zero errors;

[0177] S45. According to the output features o3, o4, o5 and the targets t3, t4, t5, the mean square error loss is constructed respectively. The following formula calculates the mean square error loss of the three layers respectively, and adds L1, L2 and L3 with different weights to obtain the multi-layer pixel reconstruction loss L: L1=l2(t3, o3); L2= l2(t4, o4); L3= l2(t5, o5); L=0.2 L1+0.6 L2+0.2 L3, where l2 represents the mean square error loss.

[0178] This embodiment helps to improve the accuracy and effectiveness of inspection tasks through multi-level feature extraction: by using the Backbone network of CNN to output multiple feature layers, each feature layer has a different receptive field and resolution. Not only can it capture image features of different scales and levels, but it also helps to more comprehensively understand and characterize the information of the overhead lines of the UAV distribution network. The application of feature fusion and Transformer helps to improve the model's understanding and characterization capabilities of the overhead lines of the UAV distribution network: by fusing multi-level features with the Transformer structure, the self-attention mechanism of the Transformer is fully utilized to focus on the semantic information of the features. Compared with the traditional CNN structure, the Transformer pays more attention to semantic features rather than local spatial details. Therefore, the fused features have richer semantic information. The random Mask operation helps to improve the generalization and adaptability of the model: by constructing a random Mask matrix and applying it to the input image, the model can learn to deal with the situation where some information is missing. It helps to enhance the robustness of the model for the expression of input information, so that it can better cope with the noise, interference and occlusion that may exist in the inspection of distribution network overhead lines; by building a multi-level hybrid module, the fused features are generated into multi-scale reconstruction results, which helps to restore the details and structure of the image and improve the image quality. In the adaptive inspection task, accurate pixel reconstruction can help operators better understand and analyze the images captured by the drone, and improve the efficiency and reliability of the inspection.

[0179] This application improves the accuracy and generalization ability of the detection model. Through the self-supervised pre-training method, it effectively solves the domain mismatch problem caused by the traditional detection model's reliance on ImageNet pre-training data. By using massive amounts of unlabeled distribution network overhead line data for self-supervised learning, the model can better understand and characterize the data characteristics unique to the distribution network. Multi-level image processing technology ensures the high quality of input data, and the multi-level feature extraction and fusion mechanism captures comprehensive features from low-level textures to high-level semantics. The random occlusion strategy enhances the robustness of the model, and the autoencoder structure optimizes feature expression through self-supervised learning. Combined, it improves the detection accuracy and generalization ability of the model in complex distribution network line environments, effectively solving the problem of overfitting or underfitting.

[0180] This application enhances the system's adaptability to complex environments at multiple levels. Adaptive exposure control and dynamic range compression algorithms can effectively process images under various lighting conditions, ensuring that high-quality input data can be obtained in different situations such as cloudy days, sunny days or dusk. Multi-scale image segmentation and feature enhancement technology enable the system to simultaneously process large-scale tower structures and small-scale line accessories, and adapt to targets of different scales. The Transformer structure introduces an understanding of the global context, which helps the system accurately identify and locate targets in complex backgrounds. The random occlusion mechanism simulates various occlusions and interferences that may occur in the actual environment, such as branch occlusion, shadow interference, etc., which improves the environmental adaptability of the model. Through multi-level feature extraction and fusion and self-supervised learning mechanisms, the model's ability to recognize various defects and abnormal states is improved. The use of multi-level features ensures that the model can focus on both macroscopic structures and microscopic details, while self-supervised learning enables the model to infer complete features from partial information, both of which help to improve the recall rate of detection. Especially in the case of complex backgrounds or partial occlusions, the model can still make accurate judgments based on limited information, effectively improving the recall rate.

[0181] The present application can effectively utilize a large amount of unlabeled distribution network overhead line data. Through the self-supervised pre-training method, the system can learn valuable feature representations from the raw data without a large amount of manual annotation. The autoencoder structure and reconstruction task provide a clear learning goal for the model, enabling it to extract features that are meaningful to the task from unlabeled data. This not only reduces the dependence on labeled data, but also enables the model to better adapt to actual application scenarios. Each step of the present application reflects the consideration of adaptability. From adaptive exposure control and adaptive cropping, to multi-level feature fusion, to random occlusion strategy, each step enhances the system's adaptability to different situations. The multi-level adaptive construction enables the system to maintain efficient and stable performance under different weather conditions, different lighting environments, and lines of different complexity. Although the present application introduces some complex processing steps, it still maintains a high computational efficiency through clever design. For example, the multi-level feature extraction and fusion process can obtain features of multiple scales at the same time through a single forward propagation, avoiding repeated calculations. The autoencoder structure reduces the computational complexity through compressed representation. This enables the system to achieve efficient adaptive inspection with limited computing resources. This application enhances the interpretability of the model through multi-level feature extraction and reconstruction tasks. Multi-level features can intuitively display the information of different scales and abstract levels that the model focuses on, while the reconstruction task can understand the features learned by the model by visualizing the reconstruction results. This interpretability is very important for improving the credibility of the system and its acceptance in practical applications.

[0182] This application combines a number of advanced technologies to effectively solve the problems of low accuracy, poor generalization, and insufficient recall rate faced by traditional detection models in complex distribution network line environments. It can not only make full use of massive unlabeled data, but also adapt to complex and changeable natural environments and line conditions, improving the performance and reliability of the UAV distribution network overhead line adaptive inspection system. This comprehensive, efficient and robust solution provides new possibilities for the intelligent inspection of distribution network overhead lines, and is expected to improve the operation and maintenance efficiency and safety of power systems.

[0183] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A self-supervised pre-training method for adaptive inspection of UAV distribution network lines, characterized in that: The steps include: S1. Obtaining the original image of the overhead line of the distribution network collected by the drone, and preprocessing it, including dynamic range compression and data enhancement, to obtain the preprocessed original image; S2. Extract primary multi-level features based on the preprocessed original image, generate high-level features based on the primary multi-level features using an improved Transformer structure, and fuse the primary multi-level features with the high-level features to obtain multi-level hybrid features. S3. Based on the multi-level mixed features, a random matrix is ​​constructed, and the random matrix is ​​sorted to generate a Mask matrix; based on the Mask matrix and the multi-level mixed features, a multi-level occlusion feature is calculated; S4. Build an autoencoder model, calculate the reconstruction loss of multiple layers of pixels based on multi-level occlusion features and multi-level mixed features; train the autoencoder model based on the reconstruction loss to obtain a trained autoencoder model and a final reconstruction loss; Step S3 is further as follows: S31. Based on the multi-level mixed features, a two-dimensional random matrix is ​​constructed, and each element in the two-dimensional random matrix is ​​randomly initialized using uniform distribution; S32, sorting each row in the two-dimensional random matrix in ascending order, and recording the sorted index; constructing a Mask matrix based on the sorted index and the predefined occlusion rate; S33, adjusting the shape of the Mask matrix based on the multi-level mixed features to obtain a Mask matrix after the shape is adjusted; S34, multiplying the mixed features of each level in the multi-level mixed features by the corresponding mask matrix after adjusting the shape, to obtain a multi-level occlusion feature; Step S4 is further as follows: S41, constructing an autoencoder model based on each level occlusion feature in the multi-level occlusion features, including an encoder and a decoder; S42, inputting each level of occlusion feature into the encoder to obtain a compressed potential representation; inputting the potential representation into the decoder to obtain a reconstructed feature map; S43, calculating the difference between the reconstructed feature map and the multi-level mixed features, and obtaining the reconstruction loss of the multi-layer pixels; S44, using a gradient descent algorithm to calculate the gradient of the reconstruction loss with respect to the parameters of the autoencoder model; based on the gradient, using an optimizer to update the parameters of the autoencoder model; S45, repeating steps S42 to S44 until a predefined number of iterations is reached, and outputting the trained autoencoder model and the final reconstruction loss; Step S43 is further as follows: S431, calculating the mean square error of a predetermined number of levels based on the reconstructed feature map and the multi-level mixed features; S432: Perform weighted summation on the mean square errors of the predetermined levels to obtain reconstruction losses of multiple layers of pixels.

2. The self-supervised pre-training method for adaptive inspection of UAV distribution network lines according to claim 1 is characterized in that: Step S1 is further as follows: S11, obtaining an original image of the overhead line of the distribution network collected by the drone, performing adaptive exposure control processing on the original image, and obtaining a first intermediate image after brightness adjustment; S12, generating a second intermediate image with enhanced details by using a dynamic range compression algorithm based on the first intermediate image; S13, performing multi-scale image segmentation and feature enhancement based on the second intermediate image to obtain a third intermediate image; S14, performing adaptive cropping and scaling based on the third intermediate image to obtain a fourth intermediate image; S15. Based on the fourth intermediate image, use image enhancement and noise suppression technology to obtain a fifth intermediate image; S16. Based on the fifth intermediate image, perform data enhancement and feature extraction preparation to generate a masked enhanced image, that is, the preprocessed original image.

3. The self-supervised pre-training method in the adaptive inspection of UAV distribution network lines according to claim 2 is characterized in that: Step S2 is further as follows: S21. Based on the preprocessed original image, an improved ResNet network is used to extract feature maps of predetermined different levels as primary multi-level features; S22, sorting feature maps of different levels, selecting the deepest feature map, and using an improved Transformer structure to generate high-level features based on the deepest feature map; S23, calling a preconfigured multi-layer hybrid module, upsampling the high-level features to obtain upsampled features; performing hole convolution, downsampling and channel adjustment on the primary multi-level features to obtain features after adjustment of a predetermined group; fusing the features after adjustment of the predetermined group with the upsampled features to obtain fused features of a predetermined layer; S24, integrating the fusion features of different layers to obtain integrated multi-level mixed features.

4. The self-supervised pre-training method in the adaptive inspection of UAV distribution network lines according to claim 3 is characterized in that: Step S22 is further as follows: S221, sorting feature maps of different levels, and selecting the deepest feature map; S222, reshape the deepest level feature map into a sequence form, and calculate the query, key and value matrices based on the deepest level feature map in the sequence form using a multi-head self-attention mechanism; S223, calculating an attention score based on the query, the key and the value matrix; forming a weighted feature based on the attention score and the value matrix; S224, based on the weighted features, using a feedforward neural network to generate a nonlinear transformation feature representation; Based on nonlinear transformation feature representation, residual connection and layer normalization are used to generate high-level features.

5. The self-supervised pre-training method in the adaptive inspection of UAV distribution network lines according to claim 3 is characterized in that: Step S11 is further as follows: S111, obtaining an original image collected by the drone, and calculating an overall brightness histogram of the original image; based on the overall brightness histogram, calculating an average brightness value and a standard deviation of the original image; S112, constructing a target brightness range, and calculating a brightness adjustment factor based on the target brightness range and the average brightness value and standard deviation of the original image; S113, adjusting the brightness of each pixel in the original image based on the brightness adjustment factor to obtain a new pixel value; S114: Clip the new pixel value to a predetermined range to obtain a first intermediate image after brightness adjustment.

6. The self-supervised pre-training method in the adaptive inspection of UAV distribution network lines according to claim 3 is characterized in that: Step S12 is further as follows: S121, converting the first intermediate image into a logarithmic domain to obtain a logarithmic image; based on the logarithmic image, using bilateral filtering to obtain a base layer; S122, based on the base layer and the logarithmic image, a detail layer is calculated; based on the base layer, a compression function is used to calculate a compressed base layer; S123, performing image reconstruction based on the compressed base layer, detail layer and predefined detail enhancement factor to obtain a reconstructed image; S124, converting the reconstructed image from the logarithmic domain back to the linear domain to obtain a second intermediate image with enhanced details.

7. The self-supervised pre-training method in the adaptive inspection of UAV distribution network lines according to claim 3 is characterized in that: Step S15 is further as follows: S151, dividing the fourth intermediate image into grids of a predetermined size, and calculating a local histogram of each grid; S152, based on the local histogram of each pixel in the fourth intermediate image and the surrounding grids, using a bilinear interpolation method, calculate an interpolated pixel value; S153, performing contrast limitation on the interpolated pixel value to obtain a contrast-limited pixel value; forming an enhanced image based on the contrast-limited pixel values; S154. Based on the enhanced image, a non-local mean denoising algorithm is used to calculate a fifth intermediate image.

Citation Information

Patent Citations

  • Unsupervised optical flow estimation method based on Transform feature pyramid network

    CN114187331A

  • Electrical energy equipment identification method and device and terminal equipment

    CN114399681A

  • Transform-based remote sensing image building extraction method

    CN117351363A

  • Image definition evaluation method and device, equipment and storage medium

    CN117392059A

  • Image compression method and system based on boundary perception segmentation network

    CN117651138A